Breast cancer focus detection and operation boundary simulation method and device based on deep learning

Abstract

The invention relates to a breast cancer detection and operation boundary simulation method and device based on deep learning, and relates to a computer vision and medical image processing technology. The method comprises the steps of obtaining a breast cancer image, performing preprocessing, and performing focus detection by using a pre-trained nnU-Net network to obtain a focus boundary and region. And processing the lesion image by applying an external expansion algorithm, calculating the lesion volume, and displaying a lesion area through a visual interface. According to the invention, the pre-trained nnU-Net network is used for focus detection, and the detection precision and efficiency are improved. The external expansion treatment ensures that the surgical resection range is sufficient. Accurate three-dimensional volume information is provided by focus volume calculation, which is crucial to clinical diagnosis, treatment planning and curative effect evaluation. And a visual interface displays a focus area, so that a doctor can conveniently check and perform diagnosis and surgical planning.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and medical image processing, and in particular to a method and device for breast cancer lesion detection and surgical boundary simulation based on deep learning. Background Art

[0002] The diagnosis and treatment planning of breast cancer mainly rely on the analysis of medical images. In clinical practice, doctors usually use advanced imaging technologies such as magnetic resonance imaging (MRI) and computed tomography (CT) to identify breast cancer lesions. These technologies can provide detailed images of internal structures to help doctors more accurately locate the location, size and possible spread of tumors. With these detailed medical imaging data, doctors can develop more accurate and personalized treatment plans, thereby improving treatment outcomes and patient survival rates.

[0003] However, the existing technology still has the following shortcomings: ① Most of the current lesion detection requires manual labeling by imaging experts, which is not only time-consuming but also easily affected by the subjective judgment of the labeler, resulting in unstable accuracy of the diagnosis results. ② Most of the existing lesion volume calculations are based on rough estimates of two-dimensional images, lacking accurate three-dimensional volume calculation tools, thus affecting the accuracy of treatment plans. ③ Existing tools generally lack integrated automatic detection, volume calculation and ROI expansion functions. Doctors need to use different software to operate them separately, which is complicated and inconvenient. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method and device for breast cancer lesion detection and surgical boundary simulation based on deep learning.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a method for breast cancer lesion detection and surgical boundary simulation based on deep learning, comprising the steps of:

[0007] Acquire a breast cancer image, wherein the breast cancer image includes a multi-layer slice image of a breast cancer lesion area;

[0008] Preprocessing the breast cancer image to obtain a preprocessed breast cancer image;

[0009] Using a pre-trained nnU-Net network to perform lesion detection on the pre-processed breast cancer image to obtain a breast cancer image after lesion detection, wherein the breast cancer image after lesion detection includes a breast cancer lesion boundary and a breast cancer lesion area;

[0010] Performing an expansion process on the breast cancer image after lesion detection by using a preset expansion algorithm to obtain an expanded breast cancer image, wherein the expanded breast cancer image includes a breast cancer lesion boundary and a breast cancer lesion region after the expansion process;

[0011] Calculating the volume of breast cancer lesions according to the breast cancer image after the expansion process to obtain breast cancer lesion volume data;

[0012] Based on the breast cancer lesion volume data and the breast cancer image after the expansion process, the breast cancer lesion area is displayed through a visualization interface.

[0013] The method for breast cancer lesion detection and surgical boundary simulation based on deep learning as described above, further, the step of acquiring the breast cancer image specifically includes:

[0014] Importing an initial image of breast cancer from a medical image database of a breast cancer patient, wherein the initial image of breast cancer includes CT or / and MRI image data of breast cancer;

[0015] storing the initial image of the breast cancer in a digital imaging and communications in medicine format;

[0016] The medical image processing database is called to process and analyze the initial image of breast cancer stored in the medical digital imaging and communication format to obtain an image of breast cancer containing a standardized image matrix.

[0017] The method for breast cancer lesion detection and surgical boundary simulation based on deep learning as described above, further preprocessing the breast cancer image to obtain the preprocessed breast cancer image, specifically includes:

[0018] performing a normalization operation on the breast cancer image of the standardized image matrix to obtain a breast cancer image with standardized pixel values;

[0019] A noise filtering operation is performed on the breast cancer image with standardized pixel values ​​to obtain a preprocessed breast cancer image.

[0020] The method for breast cancer lesion detection and surgical boundary simulation based on deep learning as described above, further, using a pre-trained nnU-Net network to perform lesion detection on the pre-processed breast cancer image to obtain the breast cancer image after lesion detection, specifically includes:

[0021] Using a pre-trained nnU-Net network to perform lesion detection on the pre-processed breast cancer image to obtain the lesion boundary of the breast cancer;

[0022] The breast cancer lesion area is determined according to the breast cancer lesion boundary.

[0023] The method for breast cancer lesion detection and surgical boundary simulation based on deep learning as described above, further, using a preset expansion algorithm to perform expansion processing on the breast cancer image after lesion detection to obtain the breast cancer image after expansion processing, specifically includes:

[0024] Performing an expansion process on the breast cancer lesion boundary using a preset expansion algorithm and a preset expansion distance to obtain the breast cancer lesion boundary after the expansion process;

[0025] The breast cancer lesion area after the expansion process is determined according to the breast cancer lesion boundary after the expansion process.

[0026] The method for breast cancer lesion detection and surgical boundary simulation based on deep learning as described above, further, using a preset expansion algorithm and a preset expansion distance to perform expansion processing on the breast cancer lesion boundary, to obtain the breast cancer lesion boundary after expansion processing, specifically includes:

[0027] Determining the size and shape of the structural element and the preset expansion distance according to the lesion boundary of the breast cancer;

[0028] Sliding the structural element on the breast cancer lesion area, and checking whether the structural element overlaps with the breast cancer lesion area each time the structural element slides, and if there is overlap, setting the corresponding position as the breast cancer lesion area;

[0029] The above process is repeated until the entire breast cancer image after lesion detection is traversed until the preset expansion distance is reached.

[0030] The method for breast cancer lesion detection and surgical boundary simulation based on deep learning as described above, further, calculates the breast cancer lesion volume according to the breast cancer image after the expansion process to obtain breast cancer lesion volume data, specifically including:

[0031] Calculate the actual area of ​​the breast cancer lesion after the expansion process of each slice image;

[0032] The actual area of ​​the breast cancer lesion region after the expansion processing of each slice image is accumulated in the three-dimensional space to obtain the breast cancer lesion volume data.

[0033] The deep learning-based breast cancer lesion detection and surgical boundary simulation method described above can further:

[0034] Calculating the area of ​​the breast cancer lesion region after the expansion processing of each slice image, specifically comprising: counting the number of pixels in the breast cancer lesion region after the expansion processing, and multiplying the number of pixels by the actual physical size of a single pixel to obtain the actual area of ​​the lesion region;

[0035] Accumulating the actual area of ​​the breast cancer lesion region after the expansion processing of each layer of slice image in three-dimensional space to obtain the breast cancer lesion volume data specifically includes: accumulating the actual area of ​​the lesion region of all slices layer by layer, and multiplying it by the thickness of each slice layer to obtain the breast cancer lesion volume data.

[0036] The deep learning-based breast cancer lesion detection and surgical boundary simulation method described above further displays the breast cancer lesion area through a visual interface, and then further includes:

[0037] In each slice image:

[0038] Based on the breast cancer lesion boundary after the expansion process, a breast cancer lesion boundary point set is determined;

[0039] Mapping the breast cancer lesion boundary point set to a pre-established coordinate system to obtain the coordinates of each boundary point in the breast cancer lesion boundary point set;

[0040] Determine the coordinates of pixels in the breast cancer lesion area based on the coordinates of each boundary point in the breast cancer lesion boundary point set;

[0041] Generate multi-layer coordinate data of the breast cancer lesion according to the coordinates of the pixels of the breast cancer lesion area in each layer of the slice image and the coordinates of each boundary point in the breast cancer lesion boundary point set;

[0042] A 3D printing file in a preset format is generated according to the multi-layer coordinate data of the breast cancer lesion and preset printing parameters.

[0043] In a second aspect, the present invention provides a deep learning-based breast cancer lesion detection and surgical boundary simulation device, characterized in that it includes

[0044] An image data acquisition unit, which is used to acquire an image of breast cancer, wherein the image of breast cancer includes a multi-layer slice image of a breast cancer lesion area;

[0045] An image preprocessing unit, which is used to preprocess the breast cancer image to obtain a preprocessed breast cancer image;

[0046] A lesion detection unit, which is used to perform lesion detection on the preprocessed breast cancer image using a pretrained nnU-Net network to obtain the breast cancer image after lesion detection;

[0047] An ROI expansion unit, which is used to perform expansion processing on the breast cancer image after the lesion detection by using a preset expansion algorithm to obtain the breast cancer image after the expansion processing;

[0048] a lesion volume calculation unit, which is used to calculate the lesion volume of breast cancer according to the breast cancer image after the expansion process to obtain the lesion volume data of breast cancer; and

[0049] A user interaction unit is used to display the breast cancer lesion area through a visual interface based on the breast cancer lesion volume data and the breast cancer image after the expansion process.

[0050] In a third aspect, the present invention further provides an electronic device, including a processor and a memory;

[0051] The memory is used to store programs;

[0052] The processor executes the program to implement the method described above.

[0053] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0054] In a fifth aspect, the present invention further provides a computer program product or a computer program, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method.

[0055] Compared with the prior art, the present invention has the following beneficial effects: the image of breast cancer contains multi-layer slice images, which can provide detailed internal structure information, and is helpful for subsequent image analysis, three-dimensional reconstruction and visualization operations. Image preprocessing includes normalizing and noise filtering the image, which can improve image quality and consistency, thereby providing more reliable image input for subsequent lesion detection. Lesion detection is performed using a pre-trained nnU-Net network, which can automatically identify the lesion area and improve detection accuracy and efficiency. The external expansion processing ensures that the range of surgical resection is sufficient through a preset external expansion algorithm and external expansion distance. Lesion volume calculation provides accurate three-dimensional volume information. The volume information of the lesion area of ​​breast cancer is of great significance to clinical diagnosis, treatment planning and efficacy evaluation, and is helpful to formulate a more accurate treatment plan. Finally, the lesion area is displayed through a visual interface, and doctors can easily view the detected lesion area and its volume information, which is convenient for doctors to make diagnosis and surgical planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 Schematic diagram of the process of deep learning-based breast cancer lesion detection and surgical boundary simulation method according to an embodiment of the present invention.

[0058] Figure 2 is a rendering of an image of breast cancer after lesion detection according to an embodiment of the present invention;

[0059] Figure 3 is a rendering of an image of breast cancer after expansion processing according to an embodiment of the present invention;

[0060] Figure 4 This is an embodiment of the present invention that displays the breast cancer lesion area through a visual interface. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] Example:

[0063] It should be noted that the terms "including" and "having" and any variations of the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0064] The diagnosis and treatment planning of breast cancer mainly rely on the analysis of medical images. In clinical practice, doctors usually use advanced imaging technologies such as magnetic resonance imaging and computed tomography to identify breast cancer lesions. These technologies can provide detailed internal structure images to help doctors more accurately locate the location, size and possible spread of tumors. With these detailed medical imaging data, doctors can develop more accurate and personalized treatment plans, thereby improving treatment effects and patient survival rates. However, the existing technologies still have the following shortcomings: Most of the current lesion detection requires manual annotation by imaging experts, which is not only time-consuming, but also easily affected by the subjective judgment of the annotator, resulting in unstable accuracy of the diagnosis results. Most of the existing lesion volume calculations are based on rough estimates of two-dimensional images, and lack accurate three-dimensional volume calculation tools, which affects the accuracy of treatment plans. Existing tools generally lack integrated automatic detection, volume calculation and ROI expansion functions. Doctors need to use different software to operate them separately, which is complicated and inconvenient.

[0065] Figure 1 FIG. 1 is a flow chart of a method for breast cancer lesion detection and surgical boundary simulation based on deep learning according to an embodiment of the present invention. Figure 1 As shown, the deep learning-based breast cancer lesion detection and surgical boundary simulation method provided by the embodiment of the present invention may specifically include the following steps:

[0066] Step 100: Acquire a breast cancer image, wherein the breast cancer image includes a multi-layer slice image of a breast cancer lesion area.

[0067] In this step, the image of breast cancer can be obtained by importing the image of breast cancer from medical image data of breast cancer patients. Medical image data of breast cancer patients can include medical image data obtained and stored by mammography (MG), computed tomography (CT), magnetic resonance imaging (MRI), ultrasound and PET-CT examination. Multi-layer slice images are images with the number of image layers added on the basis of single-layer images. Multi-layer slice images are used to represent multiple sections or layers of a three-dimensional object or scene, so as to facilitate operations such as three-dimensional reconstruction, image analysis or visualization.

[0068] In some embodiments, the step of acquiring an image of breast cancer specifically includes: step 101, importing an initial image of breast cancer from a medical image database of a breast cancer patient, wherein the initial image of breast cancer includes CT or / and MRI image data of breast cancer; step 102, storing the initial image of breast cancer in a medical digital imaging and communication format; and step 103, calling a medical image processing database to process and analyze the initial image of breast cancer stored in the medical digital imaging and communication format to obtain an image of breast cancer including a standardized image matrix.

[0069] The following describes steps 101 to 103 in detail:

[0070] Import CT or / and MRI image data containing breast cancer from the medical image database of breast cancer patients, and store the initial image of breast cancer in the medical digital imaging and communication format. The medical digital imaging and communication format DICOM (Digital Imaging and Communications in Medicine) is an international standard for medical images and related information. The DICOM format defines the format and communication protocol of medical images, so that medical images and related data can be exchanged and shared between different devices and systems, such as CT scans, MRI images, etc. in this embodiment. DICOM files usually have an extension of ".dcm", which contains medical image data and related patient information, equipment information, image acquisition parameters, etc. Among them, the CT or / and MRI image data of breast cancer is stored in the DICOM format, which not only ensures the high quality of the image data, but also contains rich metadata information, such as patient information, examination date, equipment parameters, etc., which facilitates subsequent image processing and analysis.

[0071] Then, the medical image processing database is called to process and analyze the initial image of breast cancer stored in the medical digital imaging and communication format to obtain an image of breast cancer containing a standardized image matrix. The medical image processing database can be pydicom or SimpleITK, and pydicom is a Python library for processing DICOM files. It provides functions for reading, writing, and manipulating DICOM files, can parse the contents of DICOM files, and extract image data, patient information, device information, etc. SimpleITK is an open source library for medical image processing, which provides rich functions for processing and analyzing DICOM image data.

[0072] Finally, after importing and parsing the DICOM image data, it needs to be converted into a standardized image matrix for subsequent processing and analysis. The standardization of the image matrix refers to converting the DICOM image data into a unified format and resolution for comparison and analysis across devices and systems. The standardization process can include operations such as image scaling, cropping, and grayscale value adjustment.

[0073] Step 200: preprocess the breast cancer image to obtain a preprocessed breast cancer image.

[0074] In this step, a series of preprocessing operations are performed on the breast cancer image, and the preprocessing operations may be histogram equalization, grayscale, normalization, wavelet transform and edge enhancement, etc., so as to obtain the preprocessed breast cancer image. These steps can improve the efficiency and accuracy of the subsequent segmentation algorithm.

[0075] In some embodiments, preprocessing the breast cancer image to obtain the preprocessed breast cancer image specifically includes:

[0076] Step 201, performing a normalization operation on the breast cancer image of the standardized image matrix to obtain a breast cancer image with standardized pixel values;

[0077] Specifically, since images captured by different devices may have different grayscale ranges or dynamic ranges, normalization can standardize the pixel values ​​of the image to a uniform range (such as 0 to 1 or -1 to 1) to eliminate the impact of device differences and improve the robustness and training efficiency of the model.

[0078] Normalization can be achieved through a variety of algorithms, such as minimum-maximum normalization, Z-Score normalization, etc. Minimum-maximum normalization is to linearly map the pixel values ​​of an image to a new range (such as 0 to 1), so that the pixel value ranges of different images are consistent. Z-Score normalization is to subtract the mean of the pixel values ​​and divide by the standard deviation, so that the pixel values ​​have zero mean and unit variance.

[0079] Exemplarily, a minimum-maximum normalization operation is performed:

[0080]

[0081] Perform a linear transformation on the pixel values ​​of each image and adjust the pixel values ​​to the range of [0, 1] or -[1, 1]. I is the original pixel value, and Imax and Imin are the minimum and maximum values ​​of the image.

[0082] Then perform the standardization operation:

[0083]

[0084] The mean and standard deviation of each image are used for normalization, where μ is the mean of the image pixel values ​​and σ is the standard deviation of the image pixel values.

[0085] Step 202: performing a noise filtering operation on the breast cancer image with standardized pixel values ​​to obtain a pre-processed breast cancer image.

[0086] Specifically, medical images often contain various noises (such as Gaussian noise, artifacts, etc.), which can affect the accuracy of lesion detection. The purpose of noise filtering is to remove these noises while retaining the key features of the image.

[0087] Noise filtering can be achieved through a variety of filters, such as Gaussian filters, median filters, bilateral filters, etc. Gaussian filters can smooth images through convolution operations and reduce high-frequency noise; median filters can effectively remove impulse noise by sorting pixel values ​​in local areas and taking the median; bilateral filters can consider both spatial distance and pixel value differences, thereby smoothing images while retaining edge details.

[0088] For example, using a Gaussian filter to smooth an image can effectively remove high-frequency noise:

[0089]

[0090] G(x, y) is the value of the Gaussian kernel, σ is the standard deviation, which controls the filter strength. The larger σ is, the stronger the smoothing effect is.

[0091] Therefore, these steps provide a more effective image preprocessing method, which can significantly improve the quality and clarity of the image, thereby providing a more reliable basis for subsequent lesion detection and surgical boundary simulation.

[0092] Step 300: Use a pre-trained nnU-Net network to perform lesion detection on the pre-processed breast cancer image to obtain a breast cancer image after lesion detection.

[0093] Figure 2 is a rendering of an image of breast cancer after lesion detection according to an embodiment of the present invention. Figure 2 As shown, the left side shows the original MRI image, and the middle image is the breast cancer image after lesion detection.

[0094] In this step, nnU-Net is an adaptive medical image segmentation framework based on the U-Net architecture, which is formed by 2D U-Net and 3D U-Net architectures. The pre-processed breast cancer image is subjected to lesion detection using a deep learning model (nnU-Net network), which can accurately segment the lesion area and output the boundary of the lesion, thereby obtaining an image of breast cancer after lesion detection.

[0095] In some embodiments, using a pre-trained nnU-Net network to perform lesion detection on the pre-processed breast cancer image to obtain the breast cancer image after lesion detection specifically includes: step 301, using a pre-trained nnU-Net network to perform lesion detection on the pre-processed breast cancer image to obtain a breast cancer lesion boundary; step 302, determining the breast cancer lesion area according to the breast cancer lesion boundary.

[0096] The following describes steps 301 to 302 in detail:

[0097] By using the pre-trained nnU-Net network to perform lesion detection on the pre-processed breast cancer images, the lesion area can be accurately segmented and the boundary of the lesion can be output, which improves the automation and accuracy of lesion detection and reduces the influence of human subjective judgment. The determination of the lesion boundary can be post-processed through the probability map output by the network, such as extracting the boundary through threshold segmentation or morphological operations. The pre-trained nnU-Net network can be pre-trained on a large public dataset to improve its generalization ability and robustness. Then, the network can be fine-tuned using a small amount of labeled data through the transfer learning method to adapt to a specific breast cancer image dataset. After confirming the breast cancer lesion boundary, the area within the breast cancer lesion boundary is determined as the breast cancer lesion area.

[0098] As a result, these steps reduce the reliance on manual labeling by imaging experts, thereby reducing the impact of subjective judgment on breast cancer diagnosis and significantly improving the accuracy and automation level of breast cancer lesion detection.

[0099] Step 400: Perform an expansion process on the breast cancer image after lesion detection using a preset expansion algorithm to obtain an expanded breast cancer image.

[0100] Figure 3 is a rendering of an image of breast cancer after expansion processing according to an embodiment of the present invention. Figure 3 As shown, the left side shows the original MRI image, the middle image is the breast cancer image after lesion detection, and the right image is the breast cancer image after expansion processing.

[0101] In this step, based on the image of breast cancer after lesion detection, the ROI of the lesion is expanded by a preset expansion algorithm, and the expansion distance can be set by the doctor according to clinical practice (for example, 3mm, 5mm). The expansion algorithm can be implemented using a morphological dilation operation. Through this step, the lesion boundary can be expanded outward to ensure that the range of surgical resection is sufficient.

[0102] In certain embodiments, the image of the breast cancer after lesion detection is expanded using a preset expansion algorithm to obtain the image of the breast cancer after expansion, specifically comprising: step 401, expanding the lesion boundary of the breast cancer using a preset expansion algorithm and a preset expansion distance to obtain the lesion boundary of the breast cancer after expansion; step 402, determining the lesion area of ​​the breast cancer after expansion according to the lesion boundary of the breast cancer after expansion.

[0103] The following describes steps 401 to 402 in detail:

[0104] The preset expansion algorithm can adopt a method based on geometric morphology, such as a morphological dilation operation, to achieve the expansion of the boundary by dilating the structural elements on the boundary of the lesion. The preset expansion distance can be set according to clinical experience and the size of the specific lesion, usually set to a certain number of millimeters (e.g., 3mm, 5mm) to ensure that the potential lesion area is fully covered. Then, an adaptive expansion algorithm can be used in combination with the morphological characteristics of the lesion, that is, the expansion distance is dynamically adjusted according to the local characteristics of the lesion boundary. On the basis of obtaining the lesion boundary of the breast cancer after the expansion process, the area within the lesion boundary of the breast cancer after the expansion process is determined as the lesion area of ​​the breast cancer after the expansion process.

[0105] Therefore, these steps can appropriately expand the lesion boundaries to more comprehensively cover the possible lesion areas, ensure that the scope of surgical resection is sufficient, and provide more reliable data support for subsequent treatment planning.

[0106] In some embodiments, the breast cancer lesion boundary is expanded using a preset expansion algorithm and a preset expansion distance to obtain the breast cancer lesion boundary after the expansion, specifically comprising: step 411, determining the size and shape of a structure element and a preset expansion distance according to the breast cancer lesion area; step 412, sliding the structure element on the breast cancer lesion area, and checking whether the structure element overlaps with the breast cancer lesion area each time it slides, and if so, setting the corresponding position as the breast cancer lesion area; step 413, repeating the above process until the entire breast cancer image after lesion detection is traversed until the preset expansion distance is reached.

[0107] The following describes steps 411 to 413 in detail:

[0108] Based on clinical experience and the specific size of the lesion, the size, shape and necessary expansion distance of the structural element are determined. Then, the structural element is moved over the lesion area of ​​breast cancer. Each time it moves, it is checked whether the structural element overlaps with the lesion area. If there is an overlap, the location is marked as the lesion area (i.e., the foreground object). This process is repeated until the entire image of breast cancer after lesion detection is covered. Through the morphological dilation operation, the boundary of the lesion area will expand outward by a specific distance (i.e., the expansion distance). In this way, the range of surgical resection will include more surrounding tissue to ensure that the lesion is completely removed.

[0109] Step 500, calculating the volume of breast cancer lesions according to the breast cancer image after the expansion process, to obtain breast cancer lesion volume data;

[0110] In this step, the volume of the breast cancer lesion area in three-dimensional space is calculated based on the breast cancer image after expansion. The volume information of the breast cancer lesion area is of great significance for clinical diagnosis, treatment planning and efficacy evaluation. For example, doctors can use the lesion volume to evaluate the growth rate of breast cancer or determine the degree of lesion reduction after treatment.

[0111] In certain embodiments, the breast cancer lesion volume is calculated based on the breast cancer image after the expansion process to obtain the breast cancer lesion volume data, specifically including: step 501, calculating the actual area of ​​the breast cancer lesion region after the expansion process of each layer of slice image; step 502, accumulating the actual area of ​​the breast cancer lesion region after the expansion process of each layer of slice image in three-dimensional space to obtain the breast cancer lesion volume data.

[0112] Specifically, calculating the area of ​​the breast cancer lesion region after the expansion processing of each slice image specifically includes: step 511, counting the number of pixels in the breast cancer lesion region after the expansion processing, and multiplying the number of pixels by the actual physical size of a single pixel to obtain the actual area of ​​the lesion region;

[0113] Specifically, the actual area of ​​the breast cancer lesion region after the expansion processing of each layer of slice image is accumulated in the three-dimensional space to obtain the breast cancer lesion volume data, which specifically includes: step 512, accumulating the actual area of ​​the lesion region of all slices layer by layer, and multiplying it by the thickness of each slice layer to obtain the breast cancer lesion volume data.

[0114] The following describes steps 501-502 and steps 511-512 in detail:

[0115] In medical images, the 3D lesion volume is composed of multiple continuous 2D slices. The actual area of ​​the lesion region in each slice is calculated and accumulated in 3D space to obtain the lesion volume data of breast cancer:

[0116] Specifically, through the expansion process in the previous stage, an image of the breast cancer lesion area is obtained. Next, the number of pixels in the breast cancer lesion area after the expansion process is counted, and this number of pixels is multiplied by the actual physical size of each pixel (i.e., the voxel size) to calculate the actual area of ​​the lesion area in the two-dimensional slice:

[0117] A i =P i ·S x ·S y

[0118] Among them, A i Represents the lesion area of ​​the i-th slice (unit: mm 2 );Pi is the total number of pixels in the lesion area in the i-th slice, and S x and S y Represents the physical size of the pixel in the horizontal and vertical directions respectively (unit: mm / pixel).

[0119] The actual areas of the lesion regions of all two-dimensional slices are accumulated layer by layer and multiplied by the thickness d of the slice to calculate the total volume data V of the breast cancer lesion.

[0120]

[0121] V represents the total volume of the lesion, expressed in cubic millimeters (mm 3 ) is the unit; N represents the total number of slices; A i is the area of ​​the lesion in the ith slice, also in cubic millimeters (mm 3 ) is the unit; d refers to the layer thickness, which is in millimeters (mm).

[0122] Therefore, these steps provide accurate volume information of breast cancer lesion areas, which is of great significance for clinical diagnosis, treatment planning and efficacy evaluation, and helps doctors formulate more accurate treatment plans.

[0123] Step 600: Based on the breast cancer lesion volume data and the breast cancer image after the expansion process, the breast cancer lesion region is displayed through a visualization interface.

[0124] Figure 4 The embodiment of the present invention displays the breast cancer lesion area through a visual interface, such as Figure 4 As shown in the figure, the grid lines are used to locate and measure the position of objects in three-dimensional space. The four letters R, S, P, and L above the grid lines represent Right, Superior, Posterior, and Left, respectively. The letters represent reference points in different directions in the image or form a three-dimensional coordinate system. Through this coordinate system, the direction and position of the breast cancer lesion area in three-dimensional space can be more accurately displayed.

[0125] In this step, through the visual interface, the doctor can easily view the breast cancer lesion area and its volume information, and can set the expansion distance and view the expansion results in real time. By displaying the lesion area through the visual interface, the doctor can easily view the detected lesion area and its volume information, which is convenient for the doctor to make diagnosis and surgical planning.

[0126] It can be seen that the embodiments of the present invention have the following beneficial effects:

[0127] Breast cancer images contain multi-layer slice images, which can provide detailed internal structure information and help with subsequent image analysis, three-dimensional reconstruction, and visualization operations. Image preprocessing includes normalization and noise filtering of images, which can improve image quality and consistency, thereby providing more reliable image input for subsequent lesion detection. Lesion detection is performed using a pre-trained nnU-Net network, which can automatically identify lesion areas and improve detection accuracy and efficiency. External expansion processing ensures that the scope of surgical resection is sufficient through a preset external expansion algorithm and external expansion distance. Lesion volume calculation provides accurate three-dimensional volume information. The volume information of the lesion area of ​​breast cancer is of great significance for clinical diagnosis, treatment planning, and efficacy evaluation, and helps to formulate a more accurate treatment plan. Finally, the lesion area is displayed through a visual interface, so that doctors can easily view the detected lesion area and its volume information, which is convenient for doctors to make diagnosis and surgical planning.

[0128] In some embodiments, the breast cancer lesion area is displayed through a visual interface, and then the following steps are further included:

[0129] In each layer of slice image: step 701, based on the breast cancer lesion boundary after the expansion process, determine the breast cancer lesion boundary point set; step 702, map the breast cancer lesion boundary point set to a pre-established coordinate system to obtain the coordinates of each boundary point in the breast cancer lesion boundary point set; step 703, based on the coordinates of each boundary point in the breast cancer lesion boundary point set, determine the coordinates of the pixels of the breast cancer lesion area; step 704, according to the coordinates of the pixels of the breast cancer lesion area of ​​each layer of slice image and the coordinates of each boundary point in the breast cancer lesion boundary point set, generate multi-layer coordinate data of the breast cancer lesion; step 705, according to the multi-layer coordinate data of the breast cancer lesion and preset printing parameters, generate a 3D printing file in a preset format.

[0130] Steps 701 to 705 are described in detail below:

[0131] On the basis of the breast cancer lesion boundary obtained after the expansion process, sampling is performed at a certain interval to form a breast cancer lesion boundary point set; after the breast cancer lesion boundary point set is determined, a three-dimensional coordinate system is pre-defined, and then the lesion boundary point set is mapped to the pre-established coordinate system by a coordinate transformation method; based on the coordinates of these boundary points, the pixel coordinates of the lesion area can be determined by using a pixel point extraction or a region growing algorithm based on boundary tracking; after processing each layer of slice image, data containing depth information can be obtained, based on the obtained pixel coordinates of the breast cancer lesion area of ​​each layer of slice image and the coordinates of each boundary point in the breast cancer lesion boundary point set, the results are integrated into a unified data structure to obtain multi-layer coordinate data; point cloud data is generated using the multi-layer coordinate data, and then the point cloud is converted into triangular mesh data using algorithms such as Poisson reconstruction and surface reconstruction. These triangular mesh data are converted into a preset 3D printing file (such as STL format, OBJ format, etc.) through file format conversion. Finally, 3D printing is completed using the existing 3D modeling software and printer driver through the preset 3D printing file and preset printing parameters (such as accuracy, material properties (such as color, hardness, etc.)).

[0132] Thus, the present invention generates a file that can be used for 3D printing through a series of image processing and data conversion steps. Compared with the prior art, these steps can provide a more accurate 3D printing model, and doctors can easily view the physical model and use the model to prepare the prosthesis or breast tissue patch in breast reconstruction. Compared with the traditional breast prosthesis or breast tissue patch manufacturing method, 3D printing technology has higher manufacturing efficiency and precision. In the traditional manufacturing process, it is usually necessary to go through multiple complex processes such as mold design, manufacturing, injection molding and later processing, and the manufacturing cycle is long and prone to errors. However, 3D printing technology adopts a digital manufacturing method, without the need to make a mold, and directly prints layer by layer according to a 3D printing file in a preset format. A variety of different types of materials can be used to manufacture the prosthesis or breast tissue patch in breast reconstruction. The manufacturing process is simple and fast, which can greatly shorten the manufacturing cycle of the breast prosthesis or breast tissue patch.

[0133] Based on the same inventive concept, an embodiment of the present invention further provides a deep learning-based breast cancer lesion detection and surgical boundary simulation device, comprising:

[0134] An image data acquisition unit, which is used to acquire an image of breast cancer, wherein the image of breast cancer includes a multi-layer slice image of a breast cancer lesion area;

[0135] An image preprocessing unit, which is used to preprocess the breast cancer image to obtain a preprocessed breast cancer image;

[0136] A lesion detection unit, which is used to perform lesion detection on the preprocessed breast cancer image using a pretrained nnU-Net network to obtain the breast cancer image after lesion detection;

[0137] An ROI expansion unit, which is used to perform expansion processing on the breast cancer image after the lesion detection by using a preset expansion algorithm to obtain the breast cancer image after the expansion processing;

[0138] a lesion volume calculation unit, which is used to calculate the lesion volume of breast cancer according to the breast cancer image after the expansion process to obtain the lesion volume data of breast cancer; and

[0139] A user interaction unit is used to display the breast cancer lesion area through a visual interface based on the breast cancer lesion volume data and the breast cancer image after the expansion process.

[0140] Since the device is a device corresponding to the deep learning-based breast cancer lesion detection and surgical boundary simulation method of the embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0141] Based on the same inventive concept, an embodiment of the present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the deep learning-based breast cancer lesion detection and surgical boundary simulation method as described above.

[0142] It is understood that the memory may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data created according to the use of the server, etc.

[0143] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect the various parts of the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor can integrate one or a combination of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes the operating system and application programs; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but implemented separately through a chip.

[0144] Since the electronic device is an electronic device corresponding to the deep learning-based breast cancer lesion detection and surgical boundary simulation method of an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0145] Based on the same inventive concept, an embodiment of the present invention also provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the deep learning-based breast cancer lesion detection and surgical boundary simulation method as described above.

[0146] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0147] Since the storage medium is the storage medium corresponding to the deep learning-based breast cancer lesion detection and surgical boundary simulation method of the embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0148] In some possible implementations, various aspects of the method of the embodiment of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the method for breast cancer lesion detection and surgical boundary simulation based on deep learning according to various exemplary embodiments of the present application described above in this specification. Among them, the executable computer program code or "code" for executing each embodiment can be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, structured query language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0149] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0150] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0151] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for breast cancer lesion detection and surgical boundary simulation based on deep learning, characterized in that: Includes steps: Acquire a breast cancer image, wherein the breast cancer image includes a multi-layer slice image of a breast cancer lesion area; Preprocessing the breast cancer image to obtain a preprocessed breast cancer image; Using a pre-trained nnU-Net network to perform lesion detection on the pre-processed breast cancer image to obtain a breast cancer image after lesion detection; Performing an expansion process on the breast cancer image after lesion detection using a preset expansion algorithm to obtain an expanded breast cancer image; Calculating the volume of breast cancer lesions according to the breast cancer image after the expansion process to obtain breast cancer lesion volume data; Based on the breast cancer lesion volume data and the breast cancer image after the expansion process, the breast cancer lesion area is displayed through a visualization interface.

2. The method for breast cancer lesion detection and surgical boundary simulation based on deep learning according to claim 1, characterized in that: The step of acquiring the breast cancer image specifically includes: Importing an initial image of breast cancer from a medical image database of a breast cancer patient, wherein the initial image of breast cancer includes CT or / and MRI image data of breast cancer; storing the initial image of the breast cancer in a digital imaging and communications in medicine format; The medical image processing database is called to process and analyze the initial image of breast cancer stored in the medical digital imaging and communication format to obtain an image of breast cancer containing a standardized image matrix.

3. The method for breast cancer lesion detection and surgical boundary simulation based on deep learning according to claim 2, characterized in that: Preprocessing the breast cancer image to obtain a preprocessed breast cancer image specifically includes: performing a normalization operation on the breast cancer image of the standardized image matrix to obtain a breast cancer image with standardized pixel values; A noise filtering operation is performed on the breast cancer image with standardized pixel values ​​to obtain a preprocessed breast cancer image.

4. The method for breast cancer lesion detection and surgical boundary simulation based on deep learning according to claim 1, characterized in that: Performing lesion detection on the preprocessed breast cancer image using a pretrained nnU-Net network to obtain the breast cancer image after lesion detection specifically includes: Using a pre-trained nnU-Net network to perform lesion detection on the pre-processed breast cancer image to obtain the lesion boundary of the breast cancer; The breast cancer lesion area is determined according to the breast cancer lesion boundary.

5. The method for breast cancer lesion detection and surgical boundary simulation based on deep learning according to claim 4, characterized in that: Performing an expansion process on the breast cancer image after lesion detection by using a preset expansion algorithm to obtain the breast cancer image after expansion processing specifically includes: Performing an expansion process on the breast cancer lesion boundary using a preset expansion algorithm and a preset expansion distance to obtain the breast cancer lesion boundary after the expansion process; The breast cancer lesion area after the expansion process is determined according to the breast cancer lesion boundary after the expansion process.

6. The method for breast cancer lesion detection and surgical boundary simulation based on deep learning according to claim 5, characterized in that: Performing an expansion process on the breast cancer lesion boundary by using a preset expansion algorithm and a preset expansion distance to obtain the breast cancer lesion boundary after the expansion process, specifically includes: Determining the size and shape of the structural element and the preset expansion distance according to the lesion boundary of the breast cancer; Sliding the structural element on the breast cancer lesion area, and checking whether the structural element overlaps with the breast cancer lesion area each time the structural element slides, and if there is overlap, setting the corresponding position as the breast cancer lesion area; The above process is repeated until the entire breast cancer image after lesion detection is traversed until the preset expansion distance is reached.

7. The method for breast cancer lesion detection and surgical boundary simulation based on deep learning according to claim 5, characterized in that: Calculating the volume of breast cancer lesions according to the breast cancer image after the expansion process to obtain breast cancer lesion volume data specifically includes: Calculate the actual area of ​​the breast cancer lesion after the expansion process of each slice image; The actual area of ​​the breast cancer lesion region after the expansion processing of each slice image is accumulated in the three-dimensional space to obtain the breast cancer lesion volume data.

8. The method for breast cancer lesion detection and surgical boundary simulation based on deep learning according to claim 7, characterized in that: Calculating the area of ​​the breast cancer lesion region after the expansion processing of each slice image, specifically comprising: counting the number of pixels in the breast cancer lesion region after the expansion processing, and multiplying the number of pixels by the actual physical size of a single pixel to obtain the actual area of ​​the lesion region; Accumulating the actual area of ​​the breast cancer lesion region after the expansion processing of each layer of slice image in three-dimensional space to obtain the breast cancer lesion volume data specifically includes: accumulating the actual area of ​​the lesion region of all slices layer by layer, and multiplying it by the thickness of each slice layer to obtain the breast cancer lesion volume data.

9. The method for breast cancer lesion detection and surgical boundary simulation based on deep learning according to claim 1, characterized in that: The breast cancer lesion area is displayed through a visual interface, followed by: In each slice image: Based on the breast cancer lesion boundary after the expansion process, a breast cancer lesion boundary point set is determined; Mapping the breast cancer lesion boundary point set to a pre-established coordinate system to obtain the coordinates of each boundary point in the breast cancer lesion boundary point set; Determine the coordinates of pixels in the breast cancer lesion area based on the coordinates of each boundary point in the breast cancer lesion boundary point set; Generate multi-layer coordinate data of the breast cancer lesion according to the coordinates of the pixels of the breast cancer lesion area in each layer of the slice image and the coordinates of each boundary point in the breast cancer lesion boundary point set; A 3D printing file in a preset format is generated according to the multi-layer coordinate data of the breast cancer lesion and preset printing parameters.

10. A deep learning-based breast cancer lesion detection and surgical boundary simulation device, characterized in that: include An image data acquisition unit, which is used to acquire an image of breast cancer, wherein the image of breast cancer includes a multi-layer slice image of a breast cancer lesion area; An image preprocessing unit, which is used to preprocess the breast cancer image to obtain a preprocessed breast cancer image; A lesion detection unit, which is used to perform lesion detection on the preprocessed breast cancer image using a pretrained nnU-Net network to obtain the breast cancer image after lesion detection; An ROI expansion unit, which is used to perform expansion processing on the breast cancer image after the lesion detection by using a preset expansion algorithm to obtain the breast cancer image after the expansion processing; a lesion volume calculation unit, which is used to calculate the lesion volume of breast cancer according to the breast cancer image after the expansion process to obtain the lesion volume data of breast cancer; and A user interaction unit is used to display the breast cancer lesion area through a visual interface based on the breast cancer lesion volume data and the breast cancer image after the expansion process.

Images