Breast symmetry evaluation method based on deep learning and program product

Abstract

The invention relates to the field of intelligent medical treatment, in particular to a breast symmetry evaluation method based on deep learning and a program product. Comprising the steps of S1, acquiring a trunk image; s2, performing breast region segmentation on the trunk image to obtain segmented images; s3, calculating the area of a left breast region and a right breast region based on the segmented image; s4, performing depth information extraction on the trunk image to obtain a depth information image; s5, calculating the volumes of the left and right breasts based on the left and right breast area and depth information images; and S6, comparing the absolute value difference of the areas of the left and right breasts with the absolute value difference of the volumes to obtain an evaluation result of the symmetry of the breasts. According to the method, the breast features can be automatically and quickly identified, the symmetry of the breasts can be evaluated, and the method has a good clinical value.

Description

Technical Field

[0001] The present application relates to the field of intelligent medical care, and specifically to a breast symmetry assessment method, device, program product, and computer-readable storage medium based on deep learning. Background Art

[0002] Breast symmetry assessment is a very important part in clinical practice, especially in the screening and diagnosis of breast diseases. Significant asymmetry of breast volume may be associated with an increased risk of breast cancer. Doctors can use breast symmetry as one of the indicators to monitor high-risk women. Symmetry is not only related to the health of the breast, but also an important aesthetic standard for the appearance of female breasts. In breast reconstruction surgery (such as breast reconstruction after breast cancer) and cosmetic plastic surgery, symmetry assessment is crucial for surgical planning. In the field of plastic surgery, there are a variety of methods to achieve breast symmetry assessment, but there is currently no recognized standard method. Breast symmetry is usually assessed through clinical observation of patients and surgeons. However, compared with subjective evaluation, objective evaluation methods are more stable and meet the requirements for breast quantification.

[0003] Currently, there are a variety of computer-based objective symmetry assessment methods, such as breast aesthetic scoring, kOBCS, VIBA-Calc, etc., which usually rely on clinicians to manually enter data or perform calculations, which is time-consuming, labor-intensive and has poor repeatability. Summary of the invention

[0004] In view of the above problems, the present invention provides a breast symmetry assessment method based on deep learning, which specifically includes: S1: acquiring a torso image; S2: performing breast region segmentation on the torso image to obtain a segmented image; S3: calculating the areas of the left and right breast regions based on the segmented image; S4: performing depth information extraction on the torso image to obtain a depth information image; S5: calculating the volumes of the left and right breasts based on the areas of the left and right breast regions and the depth information image; S6: comparing the absolute value difference between the areas of the left and right breasts and the absolute value difference between the volumes to obtain a breast symmetry assessment result.

[0005] Furthermore, the specific steps of replacing S5 with S51 are: firstly fusing the depth information image with the segmented image to obtain a depth fusion segmentation map, and then calculating the volumes of the left and right breasts based on the depth fusion segmentation map.

[0006] The method also includes symmetry map evaluation, performing target detection on the torso image to obtain a target image, wherein the targets in the target image include: nipple-areola complexes on both sides and supraclavicular fossa; a vertical line is drawn downward from the midpoints of the nipple-areola complexes on both sides and the supraclavicular fossa as a symmetry axis, a symmetry map of the left and right breasts is obtained based on the symmetry axis, and an evaluation result of breast symmetry is obtained based on the symmetry map.

[0007] Optionally, the symmetry map includes a 2D symmetry map and a 3D symmetry map.

[0008] Furthermore, the symmetry map evaluation also includes relative difference comparison: based on the symmetry map, the overlapping part of the left and right breast areas and the overlapping part of the left and right breast volumes are obtained, based on the overlapping part of the areas and the overlapping part of the volumes, the relative difference of the areas and volumes is calculated, and the symmetry of the breasts is evaluated based on the relative difference.

[0009] The target detection is performed by using a trained target detection model to obtain a target image;

[0010] Optionally, the target detection model adopts one or more of the following: YOLOv5 / v4 / v3, R-CNN, FasterR-CNN, SSD, Mask R-CNN, DETR, CenterNet, RetinaNet;

[0011] Optionally, the training process of the trained target detection model is: acquiring torso image data; annotating the torso image data to obtain annotated data; the annotated positions include the nipple-areola complex and the supraclavicular fossa; and inputting the annotated data into the target detection model for training to obtain a trained target detection model.

[0012] Optionally, the target detection model is trained based on the YOLOv5 framework.

[0013] The breast area segmentation is performed by segmenting using a trained segmentation model to obtain a segmented image;

[0014] Optionally, the segmentation model adopts one or more of the following: mobilenetv2, FCN, U-Net, DeepLab, PSPNet, RefineNet, EncNet, HRNet.

[0015] Optionally, the training process of the trained segmentation model is: obtaining torso image data; annotating the torso image data to obtain annotated data; the annotated location is the bilateral breast area; and inputting the annotated data into the segmentation model for training to obtain the trained segmentation model. Optionally, the data annotation uses polygon annotation to outline the breast boundary.

[0016] Optionally, the segmentation model is trained based on a mobilenetv2 framework.

[0017] Optionally, the depth information extraction obtains a depth information image by calculating a depth information estimation model.

[0018] Optionally, the depth information estimation model adopts one or more of the following: transformer, CNN, SwinTransformer.

[0019] Optionally, the weight of the depth information estimation model is the weight of a pre-trained model, and the pre-trained model is DINOv2.

[0020] The depth information image also includes background removal, removing the depth information in the depth information image whose depth range is less than a preset threshold to obtain a human body depth information image, and calculating the volumes of the left and right breasts based on the areas of the left and right breasts and the human body depth information image.

[0021] Optionally, depth information in the depth information image whose depth range is less than a preset threshold is removed to obtain a human body depth information image, the human body depth information image is first fused with the segmented image to obtain a depth fusion segmentation map, and then the volumes of the left and right breasts are calculated based on the depth fusion segmentation map.

[0022] An object of the present invention is to provide a computer program product comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the above-mentioned breast symmetry assessment method based on deep learning.

[0023] An object of the present invention is to provide a computer device, which includes a memory, a processor, and a computer program or instructions stored in the memory, wherein the computer program or instructions are executed by the processor to implement the above-mentioned deep learning-based breast symmetry assessment method.

[0024] An object of the present invention is to provide a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the above-mentioned deep learning-based breast symmetry assessment method.

[0025] Advantages of the present invention:

[0026] 1. Breast symmetry is evaluated by calculating the area, volume, and symmetry diagram of the breast, and processing is performed from multiple dimensions, so that the surgeon can formulate a surgical plan before the operation based on the symmetry image. The position of the prosthesis can also be adjusted in real time based on the symmetry image during the operation, which has important clinical significance.

[0027] 2. The present invention proposes absolute symmetry and relative symmetry. Absolute symmetry compares the absolute difference in breast area and volume. Relative symmetry is based on a symmetrical image to obtain the overlapping part of the left and right breast areas and volumes, and then calculates the relative difference in area and volume. It can effectively present the difference between the two breasts, increase the evaluation indicators of breast symmetry, and further improve the objectivity of breast symmetry evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 A schematic diagram of a process flow of a breast symmetry assessment method based on deep learning provided by an embodiment of the present invention;

[0030] Figure 2 A schematic diagram of a breast symmetry assessment system based on deep learning provided by an embodiment of the present invention;

[0031] Figure 3 A schematic diagram of a deep learning-based breast symmetry assessment device provided by an embodiment of the present invention;

[0032] Figure 4 A schematic diagram of a breast symmetry assessment structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0034] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The sequence numbers of the operations, such as S101, S102, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0035] Figure 1 A schematic diagram of a breast symmetry assessment method based on deep learning provided by an embodiment of the present invention specifically includes:

[0036] S1: Acquire torso image;

[0037] In one embodiment, the torso image includes a torso RGB image or a torso depth image. When depth information is extracted from the torso image, the torso image is a torso depth image.

[0038] In a specific embodiment, the present invention needs to collect preoperative photos of the patient, and each photo needs to be taken with the patient in an upright position, exposing the area between the navel and the supraclavicular fossa, and with both hands behind the back. The chest skin should be free of large scar hyperplasia and other diseases.

[0039] In a specific embodiment, the data acquisition device used is an RGBD camera, which can simultaneously output the RGB color and depth information of the photo. The RGB color and depth information will be used for subsequent supervised training and learning.

[0040] S2: Perform breast region segmentation on the torso image to obtain a segmented image;

[0041] In one embodiment, MobileNetV2 is a lightweight convolutional neural network (CNN) designed for mobile and embedded devices. It is developed on the basis of MobileNetV1 and aims to improve the accuracy and computational efficiency of the model while maintaining a small model size. MobileNetV2 mainly achieves this by introducing inverted residual blocks and linear bottlenecks.

[0042] In traditional residual networks (ResNets), residual blocks usually contain a series of convolutional layers where the number of input and output channels is roughly the same. However, in MobileNetV2, this structure is inverted and a "narrow-wide-narrow" design is used. First, the number of input channels is increased (called dilation) through a point-by-point convolution (1x1 convolutions), then the spatial dimension is reduced through a 3x3 depth-separable convolution, and finally the number of channels is reduced back to the original width through a point-by-point convolution again. This design allows the model to increase the nonlinear expression power of the intermediate layers while maintaining computational efficiency. At the end of the inverted residual block, there is a linear bottleneck layer that uses point-by-point convolutions but without nonlinear activation functions. This design allows information to flow directly between layers, thereby reducing information loss, and since there is no activation function, it also reduces computational cost.

[0043] In one embodiment, DINOv2 is mainly based on self-supervised learning methods, which means that the model can learn the feature representation of images by observing a large amount of unlabeled image data without manually annotating the data. DINOv2 is trained on a large image dataset, including 142 million carefully selected images, which helps the model learn rich visual patterns. Because DINOv2 is trained on diverse data, it is generally more robust and generalizable, and can perform well under different image conditions and scenarios.

[0044] In one embodiment, the breast region segmentation is performed by using a trained segmentation model to obtain a segmented image;

[0045] Optionally, the segmentation model adopts one or more of the following: mobilenetv2, FCN, U-Net, DeepLab, PSPNet, RefineNet, EncNet, HRNet;

[0046] In one embodiment, the training process of the trained segmentation model is:

[0047] Acquiring torso image data;

[0048] Annotating the torso image data to obtain annotated data; the annotated location is the bilateral breast area;

[0049] The labeled data is input into the segmentation model for training to obtain a trained segmentation model.

[0050] In one embodiment, the data annotation uses polygon annotation to outline the breast boundary.

[0051] In one embodiment, the segmentation model is trained based on the mobilenetv2 framework.

[0052] In one embodiment, the depth information extraction is performed by calculating a depth information image using a depth information estimation model;

[0053] Optionally, the depth information estimation model adopts one or more of the following: transformer, CNN, SwinTransformer.

[0054] In one embodiment, the weight of the depth information estimation model is the weight of a pre-trained model, and the pre-trained model is DINOv2.

[0055] In a specific embodiment, data annotation: labelme software is used to annotate the bilateral breast areas of the patient, and a polygon annotation method is used to outline the breast boundaries. A total of 100 images are annotated.

[0056] Model construction: In order to build a lightweight segmentation network and achieve fast automatic segmentation of the breast area, the mobilenetv2 framework was used for automatic segmentation training. A total of 50 rounds of training were performed on the NVIDIA3090 GPU.

[0057] S3: Calculating the areas of the left and right breasts based on the segmented image;

[0058] In a specific embodiment, based on the automatic segmentation result of mobilenetv2, the areas of the left and right breast regions are calculated respectively for symmetry comparison.

[0059] S4: extracting depth information from the torso image to obtain a depth information image;

[0060] In one embodiment, the depth information image also includes background removal, and the depth information with a depth range less than a preset threshold in the depth information image is removed to obtain a human body depth information image, and the volumes of the left and right breasts are calculated based on the areas of the left and right breasts and the human body depth information image.

[0061] In one embodiment, the preset threshold is a value for distinguishing a human body from a background. Optionally, the preset threshold is set to 100.

[0062] In a specific embodiment, a depth information estimation model is constructed by adopting a transformer architecture, using the DINOv2 pre-training method, pre-loading the model weights for depth estimation, and fine-tuning the model on 100 patients. The model inputs an RGB image and outputs depth information.

[0063] Considering that the required image is only the human body layer, the output depth image is adjusted. The depth range is adjusted to 0-255, and the background layer and foreground layer less than 100 are scaled to 0-255. The result at this time is the depth information containing only the human body.

[0064] S5: Calculate the volume of the left and right breasts based on the area and depth information images of the left and right breasts;

[0065] In one embodiment, the specific steps of replacing S5 with S51 are: firstly fusing the depth information image with the segmented image to obtain a depth fusion segmentation map, and then calculating the left and right breast volumes based on the depth fusion segmentation map.

[0066] In one embodiment, the depth information in the depth information image whose depth range is less than a preset threshold is removed to obtain a human body depth information image, the human body depth information image is first fused with the segmented image to obtain a depth fusion segmentation map, and then the left and right breast volumes are calculated based on the depth fusion segmentation map.

[0067] S6: Compare the absolute value difference between the left and right breast areas and the absolute value difference between the volumes to obtain a breast symmetry evaluation result.

[0068] In one embodiment, YOLOv5 (You Only Look Once version 5) is an open source object detection framework. It is part of the YOLO family and is designed to provide fast and accurate object detection capabilities, especially for real-time applications. YOLOv5 is optimized on the basis of YOLOv4, not only maintaining the detection speed, but also improving the detection accuracy and ease of use of the model. Specifically, YOLOv5 adopts the structure of CSPNet (Cross Stage Partial Networks), which is a network design that can reduce the number of parameters and the amount of calculation, which helps to improve the efficiency of the model. The model uses the SPP (SpatialPyramid Pooling) module to capture features of different scales, which is particularly important for detecting objects of different sizes. The detection head part of YOLOv5 adopts a multi-scale feature fusion strategy to combine feature maps at different levels to improve the detection ability of small objects. The loss function usually includes coordinate regression loss, object existence loss, and category prediction loss, which together guide the model to learn how to accurately locate and classify objects.

[0069] In one embodiment, the method further includes a symmetry map evaluation, performing target detection on the torso image to obtain a target image, wherein the targets in the target image include: nipple-areola complexes on both sides and supraclavicular fossa; a vertical line is drawn downward from the midpoints of the nipple-areola complexes and supraclavicular fossa on both sides as an axis of symmetry, a symmetry map of the left and right breasts is obtained based on the axis of symmetry, and an evaluation result of breast symmetry is obtained based on the symmetry map.

[0070] In one embodiment, the symmetry map comprises a 2D symmetry map and a 3D symmetry map.

[0071] In a specific embodiment, the 2D and 3D symmetric images are as follows: a vertical line is drawn downward from the midpoint of the supraclavicular fossa as the symmetry axis, and the breast area is flipped left and right, such as Figure 4 As shown in the figure, the redder the color, the more prominent the part is.

[0072] In one embodiment, there is a big difference between the 2D symmetry map and the 3D symmetry map. Some people have different sizes of breasts on both sides, and breasts of the same area have different volume symmetry. Therefore, it is of great significance to evaluate the symmetry of the 2D symmetry map and the 3D symmetry map from different dimensions to ensure the accuracy of the breast symmetry evaluation of the subject.

[0073] In one embodiment, the symmetry map evaluation also includes relative difference comparison: obtaining the overlapping part of the left and right breast areas and the overlapping part of the left and right breast volumes based on the symmetry map, calculating the relative difference of the area and volume based on the overlapping part of the area and the overlapping part of the volume, and evaluating the symmetry of the breast based on the relative difference.

[0074] In one embodiment, the target detection is performed by using a trained target detection model to obtain a target image;

[0075] Optionally, the target detection model adopts one or more of the following: YOLOv5 / v4 / v3, R-CNN, FasterR-CNN, SSD, Mask R-CNN, DETR, CenterNet, RetinaNet;

[0076] In one embodiment, the training process of the trained object detection model is:

[0077] Acquiring torso image data;

[0078] Annotating the torso image data to obtain annotated data; the annotated locations include the nipple-areola complex and the supraclavicular fossa;

[0079] The labeled data is input into the target detection model for training to obtain a trained target detection model.

[0080] In one embodiment, the target detection model is trained based on the YOLOv5 framework.

[0081] In a specific embodiment, the data annotation of target detection is: labelme software is used to annotate the patient's bilateral nipples and areolas, and the supraclavicular fossa by rectangular selection, and a total of 100 images are annotated.

[0082] Model construction for target detection:

[0083] The YOLOv5 target detection framework is used to train the labeled nipple-areola complex and supraclavicular fossa. A total of 300 rounds of training are performed on the NVIDIA 3090 GPU.

[0084] Based on the target detection method of YOLOv5, the midpoints of the three rectangular boxes of the nipple-areola complex and the supraclavicular fossa on both sides are calculated respectively to obtain the specific positions of the three targets in the image. A vertical line is drawn downward along the supraclavicular fossa as the central axis of the body.

[0085] In one embodiment, after the three targets are detected, the center position of each target is calculated, the suprasternal fossa is used to determine the symmetry axis of the body, and the breast areola can determine the distance of each breast and the asymmetry in distance.

[0086] In a specific embodiment, a vertical line is drawn downward from the supraclavicular fossa, and horizontal lines are drawn from the nipples on both sides to the vertical line to obtain the body surface marking lines. The difference in distance between the two breasts and the midline and the distance from the supraclavicular fossa are quantitatively analyzed.

[0087] In a specific embodiment, based on the depth estimation model: based on the matplotlib library in python, a surface moiré map is constructed.

[0088] In a specific embodiment, the symmetry is quantitatively analyzed:

[0089] Absolute symmetry: Based on the breast segmentation image, the areas of the left and right breasts are obtained, and the degree of breast protrusion is obtained based on the depth image. The volumes of the left and right sides are further calculated, and the difference between the absolute values ​​of the breast areas and the absolute values ​​of the volumes are compared.

[0090] Relative symmetry: Based on the symmetrical image of the breast, the overlapping parts of the left and right breast areas and volumes are obtained to calculate the relative difference in area and volume.

[0091] In a specific embodiment, the absolute value difference does not consider the symmetry, but only the area and volume differences, while the relative symmetry needs to consider the relationship of the central axis, the overlap of the breasts after folding and flipping along the central axis, and the higher the relative symmetry, the more valuable it is. When evaluating symmetry, the higher the relative symmetry of the subject, the higher the degree of symmetry of the subject.

[0092] In one embodiment, the present invention obtains a target image through target detection, divides the target image into a 2D symmetry map and a 3D symmetry map by body surface marking lines, calculates the relative area difference and relative volume difference of the left and right breasts based on the 2D symmetry map and the 3D symmetry map, and then performs a breast symmetry assessment based on the relative area difference and the relative volume difference to obtain an assessment result.

[0093] In one embodiment, the present invention performs breast symmetry assessment by calculating the absolute symmetry and relative symmetry of the breast to obtain an assessment result.

[0094] In one embodiment, the process of breast symmetry assessment of the present invention is as follows Figure 4As shown, by acquiring a torso image of a subject to be tested, the torso image is a torso RGB image and / or a torso depth image, segmenting, detecting and estimating depth information of the torso image in parallel, obtaining a breast segmentation image through segmentation, performing moiré visualization conversion on the breast segmentation image to obtain a moiré map, performing depth information estimation and extraction on the torso image to obtain a depth information image, performing torso symmetry evaluation through depth information imaging to obtain a torso symmetry result, and fusion of the depth information image with the breast segmentation image to obtain a depth fusion image, and calculating the breast area and volume based on the moiré map and the depth fusion image to obtain a symmetry result of the left and right breasts; in addition, the present invention also obtains a target image through detection, visualizes the axis of symmetry of the target image to obtain a body surface marker line drawing, obtains a 2D symmetry map and a 3D symmetry map based on the body surface marker line drawing, calculates the absolute difference area and absolute difference volume of the left and right breasts based on the above-mentioned moiré map and the depth fusion image, and calculates the relative difference area and relative difference volume of the left and right breasts based on the 2D symmetry map and the 3D symmetry map; the relative difference evaluation result, the absolute difference evaluation result and the torso symmetry result are used to generate a final evaluation result.

[0095] The disclosed embodiments of the present invention further provide a computer program product or system, including a computer program, which, when executed by a processor, implements the above-mentioned deep learning-based breast symmetry assessment method steps.

[0096] Figure 2 A schematic diagram of a deep learning-based breast symmetry assessment system provided by an embodiment of the present invention specifically includes: an acquisition unit: acquiring a torso image;

[0097] Segmentation unit: segmenting the breast area of ​​the torso image to obtain a segmented image;

[0098] Area unit: calculating the area of ​​the left and right breast regions based on the segmented image;

[0099] Depth information unit: extracting depth information from the torso image to obtain a depth information image;

[0100] Volume unit: Calculate the volume of the left and right breasts based on the area and depth information images of the left and right breasts;

[0101] Evaluation unit: compares the absolute value difference between the left and right breast areas and the absolute value difference between the volumes to obtain a breast symmetry evaluation result.

[0102] Figure 3 A schematic diagram of a deep learning-based breast symmetry assessment device provided by an embodiment of the present invention specifically includes:

[0103] A memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions, and when the program instructions are executed, any one of the above-mentioned deep learning-based breast symmetry assessment methods is implemented.

[0104] The disclosed embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it is any one of the above-mentioned deep learning-based breast symmetry assessment methods.

[0105] The verification results of this verification embodiment show that assigning inherent weights to indications can improve the performance of the method relative to the default setting. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units. A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above-mentioned embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0106] A person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be implemented by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. The above-mentioned medium storage can be a read-only memory, a disk or an optical disk, etc.

[0107] The above is a detailed introduction to a computer device provided by the present invention. For a person skilled in the art, according to the concept of the embodiments of the present invention, there may be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A breast symmetry assessment method based on deep learning, characterized in that: include: S1: Acquire torso image; S2: Perform breast region segmentation on the torso image to obtain a segmented image; S3: Calculating the areas of the left and right breasts based on the segmented image; S4: extracting depth information from the torso image to obtain a depth information image; S5: calculating the volumes of the left and right breasts based on the areas of the left and right breasts and the depth information image; S6: comparing the absolute value difference between the areas of the left and right breasts and the absolute value difference between the volumes to obtain a breast symmetry evaluation result.

2. The breast symmetry assessment method based on deep learning according to claim 1, characterized in that: The specific steps of replacing S5 with S51 are: firstly fusing the depth information image with the segmented image to obtain a depth fusion segmentation map, and then calculating the left and right breast volumes based on the depth fusion segmentation map.

3. The breast symmetry assessment method based on deep learning according to claim 1, characterized in that: The method also includes symmetry map evaluation, performing target detection on the torso image to obtain a target image, wherein the targets in the target image include: nipple-areola complexes on both sides and supraclavicular fossa; a vertical line is drawn downward from the midpoints of the nipple-areola complexes on both sides and the supraclavicular fossa as a symmetry axis, a symmetry map of the left and right breasts is obtained based on the symmetry axis, and an evaluation result of breast symmetry is obtained based on the symmetry map; optionally, the symmetry map includes a 2D symmetry map and a 3D symmetry map.

4. The breast symmetry assessment method based on deep learning according to claim 3, characterized in that: The symmetry map evaluation also includes relative difference comparison: based on the symmetry map, the overlapping part of the left and right breast areas and the overlapping part of the left and right breast volumes are obtained, based on the overlapping part of the area and the overlapping part of the volume, the relative difference of the area and the volume is calculated, and the symmetry of the breast is evaluated based on the relative difference.

5. The breast symmetry assessment method based on deep learning according to claim 3, characterized in that: The target detection is performed by a trained target detection model to obtain a target image; optionally, the target detection model adopts one or more of the following: YOLOv5 / v4 / v3, R-CNN, Faster R-CNN, SSD, Mask R-CNN, DETR, CenterNet, RetinaNet; optionally, the training process of the trained target detection model is: acquiring torso image data; annotating the torso image data to obtain annotated data; the annotated positions include the nipple-areola complex and the supraclavicular fossa; inputting the annotated data into the target detection model for training to obtain a trained target detection model; optionally, the target detection model is trained based on the YOLOv5 framework.

6. The breast symmetry assessment method based on deep learning according to claim 1, characterized in that: The breast area segmentation is performed by segmenting using a trained segmentation model to obtain a segmented image; Optionally, the segmentation model adopts one or more of the following: mobilenetv2, FCN, U-Net, DeepLab, PSPNet, RefineNet, EncNet, HRNet; Optionally, the training process of the trained segmentation model is: acquiring torso image data; annotating the torso image data to obtain annotated data; the annotated location is the bilateral breast area; inputting the annotated data into the segmentation model for training to obtain a trained segmentation model; Optionally, the data annotation adopts polygon annotation to outline the breast boundary; Optionally, the segmentation model is trained based on the mobilenetv2 framework; Optionally, the depth information extraction obtains a depth information image by calculating a depth information estimation model; optionally, the depth information estimation model adopts one or more of the following: transformer, CNN, Swin Transformer; optionally, the weight of the depth information estimation model is the weight of a pre-trained model, and the pre-trained model is DINOv2.

7. The breast symmetry assessment method based on deep learning according to claim 1 or 2, characterized in that: The depth information image also includes background removal, and the depth information in the depth information image with a depth range less than a preset threshold is removed to obtain a human body depth information image, and the volumes of the left and right breasts are calculated based on the areas of the left and right breasts and the human body depth information image; optionally, the depth information in the depth information image with a depth range less than a preset threshold is removed to obtain a human body depth information image, the human body depth information image is first fused with the segmented image to obtain a depth fusion segmentation map, and then the volumes of the left and right breasts are calculated based on the depth fusion segmentation map.

8. A computer program product comprising a computer program or instructions, characterized in that: The computer program or instructions are executed by a processor to implement the deep learning-based breast symmetry assessment method according to any one of claims 1 to 7.

9. A computer device comprising a memory, a processor and a computer program or instruction stored in the memory, characterized in that: The computer program or instructions are executed by a processor to implement the deep learning-based breast symmetry assessment method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: The computer program or instructions are executed by a processor to implement the deep learning-based breast symmetry assessment method according to any one of claims 1 to 7.

Images