A deep learning-based breast symmetry assessment method and program product

Abstract

This application relates to the field of intelligent healthcare, specifically to a method and program product for breast symmetry assessment based on deep learning. It includes: S1: acquiring a torso image; S2: segmenting the torso image into breast regions to obtain segmented images; S3: calculating the areas of the left and right breast regions based on the segmented images; S4: extracting depth information from the torso image to obtain depth information images; S5: calculating the volumes of the left and right breasts based on the areas and depth information images; S6: comparing the absolute differences in the areas and volumes of the left and right breasts to obtain the assessment result of breast symmetry. This application can automatically and quickly identify breast features and assess breast symmetry, possessing significant clinical value.

Description

Technical Field

[0001] This application relates to the field of intelligent healthcare, specifically to a method, device, program product, and computer-readable storage medium for breast symmetry assessment based on deep learning. Background Technology

[0002] Breast symmetry assessment is a crucial aspect of clinical practice, particularly in breast disease screening and diagnosis. Significant breast volume asymmetry may be associated with an increased risk of breast cancer, and physicians can use breast symmetry as one indicator for monitoring high-risk women. Furthermore, symmetry not only relates to breast health but is also an important aesthetic standard for female breast appearance. In breast reconstruction surgery (e.g., breast reconstruction after breast cancer) and cosmetic surgery, symmetry assessment is essential for surgical planning. In the field of plastic surgery, various methods exist for breast symmetry assessment, but there is currently no universally accepted standard method. Breast symmetry is typically assessed through clinical observation by both the patient and the surgeon. However, objective assessment methods are more stable and quantify breast symmetry compared to subjective assessment.

[0003] Currently, there are various computer-based objective methods for assessing symmetry, such as breast aesthetic scoring, kOBCS, and VIBA-Calc. These methods typically rely on clinicians manually inputting data or performing calculations, which is time-consuming, labor-intensive, and has poor repeatability. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a deep learning-based method for assessing breast symmetry, specifically comprising: S1: acquiring a torso image; S2: segmenting the torso image into breast regions to obtain a segmented image; S3: calculating the area of ​​the left and right breast regions based on the segmented image; S4: extracting depth information from the torso image to obtain a depth information image; S5: calculating the volume of the left and right breasts based on the area of ​​the left and right breast regions and the depth information image; S6: comparing the absolute difference in the area and the absolute difference in the volume of the left and right breasts to obtain the assessment result of breast symmetry.

[0005] Furthermore, the specific steps for replacing S5 with S51 are as follows: first, the depth information image and the segmentation image are fused to obtain a depth fusion segmentation map, and then the volumes of the left and right breasts are calculated based on the depth fusion segmentation map.

[0006] The method further includes symmetry image evaluation, which involves performing target detection on the torso image to obtain a target image, wherein the targets in the target image include: the nipple-areola complex and supraclavicular fossa on both sides; a perpendicular line is drawn downward from the midpoint of the nipple-areola complex and supraclavicular fossa on both sides as the axis of symmetry, and a symmetry image of the left and right breasts is obtained based on the axis of symmetry, and an evaluation result of breast symmetry is obtained based on the symmetry image.

[0007] Optionally, the symmetry graph includes 2D symmetry graphs and 3D symmetry graphs.

[0008] Furthermore, the symmetry diagram evaluation also includes relative difference comparison: based on the symmetry diagram, the overlapping parts of the areas and volumes of the left and right breasts are obtained, the relative difference between the areas and volumes is calculated based on the overlapping parts of the areas and volumes, and the symmetry of the breasts is evaluated based on the relative difference.

[0009] The target detection is performed by a trained target detection model to obtain the target image;

[0010] Optionally, the target detection model may employ one or more of the following: YOLOv5 / v4 / v3, R-CNN, Faster R-CNN, SSD, Mask R-CNN, DETR, CenterNet, RetinaNet;

[0011] Optionally, the training process of the trained target detection model is as follows: acquiring trunk image data; annotating the trunk image data to obtain labeled data; the labeled locations include the nipple-areola complex and the supraclavicular fossa; and inputting the labeled data into the target detection model for training to obtain the trained target detection model.

[0012] Optionally, the object detection model is trained based on the YOLOv5 framework.

[0013] The breast region segmentation is performed using a trained segmentation model to obtain a segmented image.

[0014] Optionally, the segmentation model may be one or more of the following: mobilenetv2, FCN, U-Net, DeepLab, PSPNet, RefineNet, EncNet, HRNet.

[0015] Optionally, the training process of the trained segmentation model is as follows: acquiring torso image data; annotating the torso image data to obtain labeled data; the labeled location is the bilateral breast region; and inputting the labeled data into the segmentation model for training to obtain the trained segmentation model. Optionally, the data annotation uses polygonal annotations to delineate the breast boundaries.

[0016] Optionally, the segmentation model is trained based on the Mobilenetv2 framework.

[0017] Optionally, the depth information extraction is performed by calculating the depth information image using a depth information estimation model.

[0018] Optionally, the depth information estimation model may employ one or more of the following: transformer, CNN, or SwinTransformer.

[0019] Optionally, the weights of the depth information estimation model are the weights of a pre-trained model, and the pre-trained model is DINOv2.

[0020] The depth information image also includes background removal, in which depth information with a depth range less than a preset threshold is removed to obtain a human body depth information image, and the volume of the left and right breasts is calculated based on the area of ​​the left and right breast regions and the human body depth information image.

[0021] Optionally, depth information with a depth range less than a preset threshold is removed from the depth information image to obtain a human body depth information image. The human body depth information image is then fused with the segmented image to obtain a depth fusion segmentation map. The volumes of the left and right breasts are then calculated based on the depth fusion segmentation map.

[0022] The purpose of this invention is to provide a computer program product that includes a computer program or instructions, which are executed by a processor to implement the above-described deep learning-based breast symmetry assessment method.

[0023] The purpose of this invention is to provide a computer device comprising a memory, a processor, and a computer program or instructions stored in the memory, wherein the computer program or instructions are executed by the processor to implement the aforementioned deep learning-based breast symmetry assessment method.

[0024] The purpose of this invention is to provide a computer-readable storage medium having a computer program or instructions stored thereon, which are executed by a processor to implement the above-described deep learning-based breast symmetry assessment method.

[0025] Advantages of this invention:

[0026] 1. Breast symmetry is assessed by calculating the area, volume, and symmetry diagram of the breast. This multi-dimensional approach allows surgeons to develop a surgical plan preoperatively based on the symmetry image. Intraoperatively, the implant position can be adjusted in real-time based on the symmetry image, which has significant clinical implications.

[0027] 2. This invention proposes absolute symmetry and relative symmetry. Absolute symmetry compares the absolute differences in breast area and volume. Relative symmetry is based on symmetrical images to obtain the overlapping parts of the left and right breast areas and volumes, and then calculates the relative difference in area and volume. This can effectively present the differences between the two breasts, increase the evaluation index of breast symmetry, and further improve the objectivity of breast symmetry assessment. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A schematic diagram of the breast symmetry assessment method based on deep learning provided in an embodiment of the present invention;

[0030] Figure 2 A schematic diagram of a breast symmetry assessment system based on deep learning provided in an embodiment of the present invention;

[0031] Figure 3 A schematic diagram of a breast symmetry assessment device based on deep learning provided in an embodiment of the present invention;

[0032] Figure 4 This is a schematic diagram of a breast symmetry assessment structure provided in an embodiment of the present invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0034] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as S101, S102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0035] Figure 1 A schematic diagram of a deep learning-based breast symmetry assessment method provided in this embodiment of the invention specifically includes:

[0036] S1: Obtain the torso image;

[0037] In one embodiment, the torso image includes a torso RGB image or a torso depth image. When performing depth information extraction on the torso image, the torso image is a torso depth image.

[0038] In one specific embodiment, the present invention requires the collection of preoperative photographs of the patient. Each photograph must be taken with the patient standing upright, exposing the area between the navel and the supraclavicular fossa, with both hands behind their back. The chest skin should be free of significant scarring or other diseases.

[0039] In one specific embodiment, the data acquisition device used is an RGBD camera, which can simultaneously output the RGB color and depth information of the photo. The RGB color and depth information will be used for subsequent supervised training and learning.

[0040] S2: Perform breast region segmentation on the torso image to obtain a segmented image;

[0041] In one embodiment, MobileNetV2 is a lightweight convolutional neural network (CNN) designed specifically for mobile and embedded devices. It builds upon MobileNetV1, aiming to improve model accuracy and computational efficiency while maintaining a small model size. MobileNetV2 achieves this primarily by introducing inverted residuals and linear bottlenecks.

[0042] In traditional ResNets, residual blocks typically contain a series of convolutional layers with roughly the same number of input and output channels. However, in MobileNetV2, this structure is reversed, using a "narrow-wide-narrow" design. First, the number of input channels is increased (called dilation) through a pointwise convolution (1x1 convolutions), then spatial dimensionality is reduced through a 3x3 depthwise separable convolution, and finally, the number of channels is reduced back to the original width through another pointwise convolution. This design allows the model to increase the non-linear expressiveness of intermediate layers while maintaining computational efficiency. At the end of the inverted residual block, there is a linear bottleneck layer that uses pointwise convolutions but without a non-linear activation function. This design allows information to flow directly between layers, reducing information loss and also reducing computational cost due to the absence of activation functions.

[0043] In one embodiment, DINOv2 is primarily based on a self-supervised learning approach, meaning the model can learn feature representations of images by observing a large amount of unlabeled image data without requiring manual annotation. DINOv2 is trained on a massive image dataset containing 142 million carefully selected images, which helps the model learn rich visual patterns. Because DINOv2 is trained on diverse data, it typically exhibits strong robustness and generalization ability, performing well under different image conditions and scenes.

[0044] In one embodiment, the breast region segmentation is performed using a trained segmentation model to obtain a segmented image;

[0045] Optionally, the segmentation model may be one or more of the following: mobilenetv2, FCN, U-Net, DeepLab, PSPNet, RefineNet, EncNet, HRNet;

[0046] In one embodiment, the training process of the trained segmentation model is as follows:

[0047] Acquire torso image data;

[0048] The torso image data is annotated to obtain labeled data; the labeled location is the bilateral breast region;

[0049] The labeled data is input into the segmentation model for training to obtain a trained segmentation model.

[0050] In one embodiment, the data annotation uses polygonal annotations to delineate the breast boundaries.

[0051] In one embodiment, the segmentation model is trained based on the Mobilenetv2 framework.

[0052] In one embodiment, the depth information extraction is achieved by calculating a depth information image using a depth information estimation model;

[0053] Optionally, the depth information estimation model may employ one or more of the following: transformer, CNN, or SwinTransformer.

[0054] In one embodiment, the weights of the depth information estimation model are the weights of a pre-trained model, and the pre-trained model is DINOv2.

[0055] In one specific embodiment, data annotation was performed using LabelMe software to annotate the bilateral breast regions of the patients, employing a polygon annotation method to delineate the breast boundaries. A total of 100 images were annotated.

[0056] Model Construction: To build a lightweight segmentation network and achieve fast automatic breast region segmentation, the MobileNetv2 framework was used for automatic segmentation training. A total of 50 training epochs were performed on an NVIDIA 3090 GPU.

[0057] S3: Calculate the area of ​​the left and right breast regions based on the segmented image;

[0058] In one specific embodiment, based on the automatic segmentation results of Mobilenetv2, the areas of the left and right breast regions are calculated separately for symmetry comparison.

[0059] S4: Extract depth information from the torso image to obtain a depth information image;

[0060] In one embodiment, the depth information image further includes background removal, in which depth information in the depth information image with a depth range less than a preset threshold is removed to obtain a human body depth information image, and the volume of the left and right breasts is calculated based on the area of ​​the left and right breast regions and the human body depth information image.

[0061] In one embodiment, the preset threshold is a value that distinguishes the human body from the background; optionally, the preset threshold is set to 100.

[0062] In one specific embodiment, the depth information estimation model is constructed using a transformer architecture and DINOv2 pre-training. The model weights for depth estimation are pre-loaded and fine-tuned in 100 patients. The model takes RGB images as input and outputs depth information.

[0063] Since the required image is only of the human body, the output depth image was adjusted. The depth range was adjusted to 0-255, and the values ​​less than 100 were classified as background. The foreground was also scaled down to 0-255, resulting in a depth image containing only the human body.

[0064] S5: Calculate the volume of the left and right breasts based on the area and depth information of the left and right breast regions in the image;

[0065] In one embodiment, the specific steps for replacing S5 with S51 are as follows: first, the depth information image and the segmentation image are fused to obtain a depth fusion segmentation map, and then the volumes of the left and right breasts are calculated based on the depth fusion segmentation map.

[0066] In one embodiment, depth information with a depth range less than a preset threshold is removed from the depth information image to obtain a human body depth information image. The human body depth information image is then fused with the segmented image to obtain a depth fusion segmentation map. The volumes of the left and right breasts are then calculated based on the depth fusion segmentation map.

[0067] S6: Compare the absolute differences in area and volume between the left and right breasts to obtain the assessment results of breast symmetry.

[0068] In one embodiment, YOLOv5 (You Only Look Once version 5) is an open-source object detection framework. Part of the YOLO family, it aims to provide fast and accurate object detection capabilities, particularly suitable for real-time applications. YOLOv5 optimizes upon YOLOv4, maintaining detection speed while improving detection accuracy and ease of use. Specifically, YOLOv5 employs the CSPNet (Cross Stage Partial Networks) architecture, a network design that reduces the number of parameters and computational cost, contributing to improved model efficiency. The model utilizes the SPP (Spatial Pyramid Pooling) module, capable of capturing features at different scales, which is particularly important for detecting objects of varying sizes. YOLOv5's detection head employs a multi-scale feature fusion strategy, combining feature maps from different levels to enhance the detection capability of small objects. Loss functions typically include coordinate regression loss, object presence loss, and class prediction loss, which collectively guide the model in learning how to accurately locate and classify objects.

[0069] In one embodiment, the method further includes symmetry image evaluation, performing target detection on the torso image to obtain a target image, the target image including: the nipple-areola complex and supraclavicular fossa on both sides; drawing a perpendicular line downward from the midpoint of the nipple-areola complex and supraclavicular fossa on both sides as the axis of symmetry, obtaining a symmetry image of the left and right breasts based on the axis of symmetry, and evaluating the symmetry image to obtain an evaluation result of breast symmetry.

[0070] In one embodiment, the symmetry graph includes a 2D symmetry graph and a 3D symmetry graph.

[0071] In one specific embodiment, the 2D and 3D symmetry diagrams are as follows: A perpendicular line is drawn downwards from the midpoint of the supraclavicular fossa as the axis of symmetry, and the breast region is flipped left and right, as shown below. Figure 4 As shown in the image, the redder the color, the more prominent the part.

[0072] In one embodiment, there is a significant difference between 2D and 3D symmetry diagrams. Some people have breasts that are not the same size, and breasts of the same area may have different degrees of volume symmetry. Therefore, it is important to assess symmetry from different dimensions using 2D and 3D symmetry diagrams to ensure the accuracy of the assessment of the subject's breast symmetry.

[0073] In one embodiment, the symmetry diagram evaluation further includes a relative difference comparison: obtaining the overlapping portion of the left and right breast areas and the overlapping portion of the left and right breast volumes based on the symmetry diagram, calculating the relative difference between the area and volume based on the overlapping portion of the area and the overlapping portion of the volume, and evaluating the symmetry of the breasts based on the relative difference.

[0074] In one embodiment, the target detection is performed by a trained target detection model to obtain a target image;

[0075] Optionally, the target detection model may employ one or more of the following: YOLOv5 / v4 / v3, R-CNN, Faster R-CNN, SSD, Mask R-CNN, DETR, CenterNet, RetinaNet;

[0076] In one embodiment, the training process of the trained object detection model is as follows:

[0077] Acquire torso image data;

[0078] The torso image data is annotated to obtain labeled data; the labeled locations include the nipple-areola complex and the supraclavicular fossa.

[0079] The labeled data is input into the object detection model for training to obtain a trained object detection model.

[0080] In one embodiment, the object detection model is trained based on the YOLOv5 framework.

[0081] In one specific embodiment, the target detection data annotation was performed using LabelMe software, with rectangular bounding boxes used to annotate the bilateral nipples and areolas, and supraclavicular fossa of the patients. A total of 100 images were annotated.

[0082] Model building for object detection:

[0083] The YOLOv5 object detection framework was used to train the labeled nipple-areola complex and supraclavicular fossa. A total of 300 training epochs were performed on an NVIDIA 3090 GPU.

[0084] Based on the YOLOv5-based target detection method, the midpoints of three rectangles—the nipple-areola complex on both sides and the supraclavicular fossa—were calculated to obtain the specific locations of the three targets in the image. A perpendicular line was then drawn downwards from the supraclavicular fossa as the central axis of the body.

[0085] In one embodiment, after three targets are detected, the center position of each target is calculated, the suprasternal notch is used to determine the axis of symmetry of the body, and the areola of the breast can determine the distance of each breast and the degree of asymmetry in distance.

[0086] In one specific embodiment, a vertical line is drawn downwards from the supraclavicular fossa, and horizontal lines are drawn from both nipples towards this vertical line to obtain surface landmarks. The differences in the distance between the two breasts from the midline and the difference in their distance from the supraclavicular fossa are quantitatively analyzed.

[0087] In one specific embodiment, a depth estimation model is used: a surface moiré pattern is constructed based on the matplotlib library in Python.

[0088] In one specific embodiment, symmetry is quantitatively analyzed:

[0089] Absolute symmetry: Based on the breast segmentation image, the area of ​​the left and right breasts is obtained, and the degree of breast protrusion is obtained based on the depth image. The volume of the left and right sides is further calculated, and the difference between the absolute values ​​of the breast area and the difference between the absolute values ​​of the volume are compared.

[0090] Relative symmetry: Based on the symmetrical image of the breast, the overlapping part of the area and volume of the left and right breasts is obtained, and the relative difference in area and volume is calculated.

[0091] In one specific embodiment, absolute differences are considered only in terms of area and volume, without regard to symmetry. Relative symmetry, however, requires consideration of the relationship to the central axis, specifically the degree of overlap of the breasts after folding and flipping along the central axis. Higher relative symmetry is considered more valuable. When assessing symmetry, a higher relative symmetry indicates a greater degree of symmetry in the subject.

[0092] In one embodiment, the present invention obtains a target image through target detection, divides the target image into a 2D symmetry image and a 3D symmetry image by body surface marking lines, calculates the relative area difference and relative volume difference of the left and right breasts based on the 2D symmetry image and the 3D symmetry image, and then evaluates the breast symmetry based on the relative area difference and relative volume difference to obtain the evaluation result.

[0093] In one embodiment, the present invention obtains the evaluation result by calculating the absolute symmetry and relative symmetry of the breast.

[0094] In one embodiment, the breast symmetry assessment process of the present invention is as follows: Figure 4As shown, by acquiring the torso image of the subject, which is a torso RGB image and / or a torso depth image, the torso image is segmented, detected, and depth information estimated in parallel. A breast segmentation image is obtained through segmentation, and a moiré pattern visualization conversion is performed on the breast segmentation image to obtain a moiré pattern. Depth information is estimated and extracted from the torso image to obtain a depth information image. Torso symmetry is evaluated through depth information imaging to obtain a torso symmetry result. Furthermore, the depth information image is fused with the breast segmentation image to obtain a depth fusion image. Based on the moiré pattern and the depth fusion image, the breast area and volume are calculated to obtain the symmetry result of the left and right breasts. In addition, the present invention also obtains a target image by detection, visualizes the symmetry axis of the target image to obtain surface marker lines, and obtains 2D and 3D symmetry images based on the surface marker lines. The absolute difference area and absolute difference volume of the left and right breasts are calculated by combining the above moiré pattern and depth fusion image. The relative difference area and relative difference volume of the left and right breasts are calculated based on the 2D and 3D symmetry images. The relative difference evaluation results, absolute difference evaluation results, and torso symmetry results are used to generate the final evaluation result.

[0095] The present invention also discloses a computer program product or system, including a computer program that, when executed by a processor, implements the above-described steps of the deep learning-based breast symmetry assessment method.

[0096] Figure 2 A schematic diagram of a breast symmetry assessment system based on deep learning provided in this embodiment of the invention specifically includes: an acquisition unit for acquiring a torso image;

[0097] Segmentation unit: The breast region is segmented from the torso image to obtain a segmented image;

[0098] Area unit: The area of ​​the left and right breast regions is calculated based on the segmented image;

[0099] Depth information unit: Extracts depth information from the torso image to obtain a depth information image;

[0100] Volume unit: Calculate the volume of the left and right breasts based on the area and depth information of the left and right breast regions in the image;

[0101] Assessment Unit: Compare the absolute differences in area and volume between the left and right breasts to obtain the assessment results of breast symmetry.

[0102] Figure 3 A schematic diagram of a breast symmetry assessment device based on deep learning provided in this embodiment of the invention specifically includes:

[0103] A memory and a processor; the memory is used to store program instructions; the processor is used to invoke the program instructions, when any of the above-described deep learning-based breast symmetry assessment methods are executed.

[0104] The present invention also discloses a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, is any of the above-described deep learning-based breast symmetry assessment methods.

[0105] The verification results of this verification embodiment show that assigning inherent weights to indications can improve the performance of this method compared to the default settings. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separated; the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of this embodiment. Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0106] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0107] The computer device provided by the present invention has been described in detail above. For those skilled in the art, there will be changes in the specific implementation and application scope based on the ideas of the embodiments of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A deep learning-based method for assessing breast symmetry, characterized in that, include: S1: Acquire a torso image; the torso image includes a torso RGB image; S2: Perform breast region segmentation on the RGB image of the torso to obtain a segmented image; Moody images are obtained by performing moody visualization transformation on breast segmentation images; S3: Extract depth information from the torso RGB image to obtain a depth information image; the depth information image also includes background removal, removing depth information in the depth information image whose depth range is less than a preset threshold to obtain a human body depth information image; Trunk symmetry was assessed using depth information imaging to obtain trunk symmetry results; S4: The depth information image is fused with the segmentation image to obtain a depth fusion segmentation map, and the breast area and volume are calculated based on the moiré pattern map and the depth fusion segmentation image; S5: Compare the absolute differences in area and volume between the left and right breasts. The results of the assessment of breast symmetry are obtained through the absolute differences and trunk symmetry results.

2. The breast symmetry assessment method based on deep learning according to claim 1, characterized in that, The method further includes symmetry image evaluation, which involves performing target detection on the torso image to obtain a target image, wherein the targets in the target image include: the nipple-areola complex and supraclavicular fossa on both sides; a perpendicular line is drawn downward from the midpoint of the nipple-areola complex and supraclavicular fossa on both sides as the axis of symmetry, and a symmetry image of the left and right breasts is obtained based on the axis of symmetry, and an evaluation result of breast symmetry is obtained based on the symmetry image.

3. The breast symmetry assessment method based on deep learning according to claim 2, characterized in that, The symmetry diagram includes 2D symmetry diagrams and 3D symmetry diagrams.

4. The breast symmetry assessment method based on deep learning according to claim 2, characterized in that, The symmetry diagram evaluation also includes relative difference comparison: based on the symmetry diagram, the overlapping parts of the left and right breast areas and the overlapping parts of the left and right breast volumes are obtained, the relative difference degree of area and volume is calculated based on the overlapping parts of area and volume, and the symmetry of the breasts is evaluated based on the relative difference degree.

5. The breast symmetry assessment method based on deep learning according to claim 2, characterized in that, The target detection is performed by a trained target detection model to obtain the target image.

6. The breast symmetry assessment method based on deep learning according to claim 5, characterized in that, The target detection model adopts one or more of the following: YOLOv5 / v4 / v3, R-CNN, Faster R-CNN, SSD, Mask R-CNN, DETR, CenterNet, RetinaNet.

7. The breast symmetry assessment method based on deep learning according to claim 5, characterized in that, The training process of the trained target detection model is as follows: Acquire torso image data; The trunk image data is annotated to obtain labeled data; the labeled locations include the nipple-areola complex and the supraclavicular fossa. The labeled data is input into the object detection model for training to obtain a trained object detection model.

8. The breast symmetry assessment method based on deep learning according to claim 5, characterized in that, The object detection model is trained based on the YOLOv5 framework.

9. The breast symmetry assessment method based on deep learning according to claim 1, characterized in that, The breast region segmentation is performed using a trained segmentation model to obtain segmented images.

10. The breast symmetry assessment method based on deep learning according to claim 9, characterized in that, The segmentation model adopts one or more of the following: mobilenetv2, FCN, U-Net, DeepLab, PSPNet, RefineNet, EncNet, HRNet.

11. The breast symmetry assessment method based on deep learning according to claim 9, characterized in that, The training process for the trained segmentation model is as follows: Acquire torso image data; The torso image data is annotated to obtain labeled data; the labeled location is the bilateral breast region; The labeled data is input into the segmentation model for training to obtain a trained segmentation model.

12. The deep learning-based breast symmetry assessment method according to claim 11, characterized in that, The data annotation uses polygonal annotations to delineate the boundaries of the breast.

13. The breast symmetry assessment method based on deep learning according to claim 9, characterized in that, The segmentation model is trained based on the Mobilenetv2 framework.

14. The breast symmetry assessment method based on deep learning according to claim 1, characterized in that, The depth information extraction is achieved by calculating the depth information image using a depth information estimation model.

15. The deep learning-based breast symmetry assessment method according to claim 14, characterized in that, The depth information estimation model adopts one or more of the following: transformer, CNN, and Swing Transformer.

16. The deep learning-based breast symmetry assessment method according to claim 14, characterized in that, The weights of the depth information estimation model are the weights of the pre-trained model, which is DINOv2.

17. A computer program product comprising a computer program or instructions, characterized in that, The computer program or instructions are executed by a processor to implement the deep learning-based breast symmetry assessment method according to any one of claims 1-16.

18. A computer device comprising a memory, a processor, and a computer program or instructions stored in the memory, characterized in that, The computer program or instructions are executed by a processor to implement the deep learning-based breast symmetry assessment method according to any one of claims 1-16.

19. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, The computer program or instructions are executed by a processor to implement the deep learning-based breast symmetry assessment method according to any one of claims 1-16.

Images