Breast modeling method

Abstract

The present application relates to the technical field of breast modeling, and particularly relates to a breast modeling method. A plurality of target optical images of different perspectives corresponding to a breast to be modeled and target image data and target ultrasound data corresponding to the breast to be modeled are acquired; surface geometry prior data, internal structure prior data and dynamic mechanics prior data corresponding to the breast to be modeled are constructed based on the target optical images, the target image data and the target ultrasound data; an enhanced feature vector is generated based on the surface geometry prior data, the internal structure prior data and the dynamic mechanics prior data; and a target breast model corresponding to the breast to be modeled is generated based on the enhanced feature vector. The accuracy requirement of surgical planning and prosthesis design is met, and soft tissue deformation characteristics such as breast droop and asymmetry are accurately restored; dependence on special high-precision scanning equipment is eliminated, and high-quality reconstruction can be realized based on conventional multi-source clinical data, which is convenient for popularization and application in a conventional clinical environment.

Description

Technical Field

[0001] This invention relates to the field of breast modeling technology, and more specifically to breast modeling methods. Background Technology

[0002] Three-dimensional reconstruction technology is one of the core supporting technologies in the medical field, demonstrating irreplaceable value, especially in the entire process of breast disease diagnosis and treatment. In the disease diagnosis stage, accurate three-dimensional breast models can assist doctors in quantitatively analyzing glandular distribution, tumor location, and size. In the preoperative planning stage, whether it's the morphological design of breast augmentation surgery, the boundary delineation of tumor resection surgery, or the customized development of personalized implants, all rely on high-precision three-dimensional breast structural data. Furthermore, in the postoperative evaluation stage, three-dimensional reconstruction technology is a key means to objectively quantify indicators such as breast shape, symmetry, and ptosis. Therefore, the accuracy, efficiency, and cost of three-dimensional breast reconstruction directly affect the scientific nature of clinical diagnosis and treatment decisions and the accessibility of medical services.

[0003] For a long time, three-dimensional breast reconstruction in clinical practice has mainly relied on structured light scanning technology. While commercially available high-precision scanners like ArtecEva can output high-precision geometric data, their high procurement costs, complex operating procedures, and high professional requirements for operators make them difficult to popularize in routine clinical settings. Therefore, three-dimensional breast reconstruction has been difficult to promote in primary healthcare institutions, and has also hindered the large-scale implementation of personalized medicine.

[0004] Therefore, how to effectively perform breast modeling has become an urgent problem to be solved. Summary of the Invention

[0005] This invention provides a breast modeling method to solve the problem of how to effectively perform breast modeling.

[0006] In a first aspect, the present invention provides a method for modeling a breast to be modeled, the method comprising: acquiring multiple target optical images of the breast to be modeled from different perspectives, as well as target image data and target ultrasound data corresponding to the breast to be modeled; constructing surface geometric prior data, internal structure prior data, and dynamic mechanical prior data corresponding to the breast to be modeled based on each target optical image, target image data, and target ultrasound data; generating enhanced feature vectors based on the surface geometric prior data, internal structure prior data, and dynamic mechanical prior data; and generating a target breast model corresponding to the breast to be modeled based on the enhanced feature vectors.

[0007] In one optional implementation, surface geometric prior data corresponding to the breast to be modeled is constructed based on each target optical image, target image data, and target ultrasound data. This includes: identifying each target optical image to determine an initial target camera parameter set corresponding to each target optical image; extracting the sternal midline contour from the target image data; correcting the initial target camera parameter set based on the sternal midline contour to obtain an optimized target camera parameter set; performing dense matching on each target optical image using the SfM+MVS algorithm to output an initial depth image corresponding to each target optical image, generating an initial depth image set; adjusting the initial depth image set based on the target ultrasound data to generate physiological constraint depth images corresponding to each initial depth image; and processing the physiological constraint depth images corresponding to each initial depth image based on the optimized target camera parameter set to obtain the surface geometric prior data corresponding to the breast to be modeled.

[0008] In one optional implementation, based on the optimized target camera parameter set, the physiological constraint depth images corresponding to each initial depth image are processed to obtain the surface geometric prior data corresponding to the breast to be modeled. This includes: projecting each pixel in the physiological constraint depth image to a unified world coordinate system based on the optimized target camera parameter set to generate initial dense point cloud data; optimizing the initial dense point cloud data using a region growing algorithm to generate target dense point cloud data; transforming the target dense point cloud data into an initial triangular mesh model using a Poisson reconstruction algorithm; smoothing the initial triangular mesh model using a Laplacian smoothing algorithm to generate a breast-specific triangular mesh model corresponding to the breast to be modeled; mapping the depth value of each pixel in the physiological constraint depth image to the corresponding vertex of the breast-specific triangular mesh model based on the optimized target camera parameter set, so that each vertex of the mesh simultaneously possesses three-dimensional coordinates and physiological depth constraints, thereby obtaining the target breast triangular mesh model; and determining the target breast triangular mesh model as the surface geometric prior data corresponding to the breast to be modeled.

[0009] In one optional implementation, prior data of the internal structure of the breast to be modeled is constructed based on the target optical images, target image data, and target ultrasound data. This includes: using the target breast triangular mesh model as the registration reference, and using the target region contour of the target image data and the hardness grayscale matrix of the target ultrasound data as the data to be registered; the target region includes at least one internal structure among glands, fat, and chest wall bones; a multimodal registration algorithm based on mutual information is used to register the data to be registered to obtain a final registration transformation matrix; based on the final registration transformation matrix, the target region contour and hardness grayscale matrix are mapped to the spatial position of the target breast triangular mesh model; and the registered target region is transformed into a first three-dimensional volume. The process involves several steps: First, the registered hardness grayscale matrix is ​​transformed into a second 3D voxel model. Then, each vertex of the target breast triangular mesh model is traversed, and the shortest Euclidean distance from each vertex to the corresponding internal structure in the target region is calculated in 3D space. The internal structure corresponding to the shortest Euclidean distance is used as the structural attribute label for the vertex. For each vertex of the target breast triangular mesh model, the hardness level of the ultrasonic hardness voxel corresponding to the vertex's spatial location is extracted and used as the vertex's hardness attribute label. Finally, the first and second 3D voxel models are fused to generate the internal structure voxel model corresponding to the breast to be modeled. Based on the internal structure voxel model, the prior data of the internal structure corresponding to the breast to be modeled are determined.

[0010] In one optional implementation, the prior data of the internal structure includes the proportion of glandular volume, the distribution of fat layer thickness, and the curvature deviation between the chest wall centerline and the breast base contour. Based on the internal structure voxel model, the prior data of the internal structure corresponding to the breast to be modeled is determined, including: based on the internal structure voxel model, obtaining the three-dimensional volume of the glandular region in the target area and the total breast volume corresponding to the breast to be modeled; based on the three-dimensional volume of the glandular region in the target area and the total breast volume corresponding to the breast to be modeled, obtaining the proportion of glandular volume; based on the triangular mesh model of the target breast, measuring the average thickness of the fat layer corresponding to the breast to be modeled in multiple anatomical orientations; based on the internal structure voxel model, obtaining the spatial positional relationship between the chest wall centerline and the breast base contour, and calculating the curvature deviation between the chest wall centerline and the breast base contour.

[0011] In one optional implementation, each target optical image is a temporal optical image covering three body positions: standing, lying flat, and leaning forward at preset degrees. Based on each target optical image, target image data, and target ultrasound data, dynamic biomechanical prior data corresponding to the breast to be modeled is constructed, including: using the first frame of the target optical image in the standing position as the reference image frame, extracting other SIFT feature points from other frames of optical images other than the reference image frame based on a feature point registration algorithm; for each other frame of optical images, matching the other SIFT feature points corresponding to the other frames of optical images with the reference SIFT feature points corresponding to the reference image frame to determine the registration transformation matrix corresponding to the other frames of optical images; and registering each other frame of optical images with the reference image frame based on each registration transformation matrix to obtain the registered other frames of optical images. The process involves: 1. Marking a reference image frame with the pixel coordinates of a predetermined number of reference key markers. 2. Back-projecting the pixel coordinates of each reference key marker to the world coordinate system based on the camera parameters of the reference image frame, obtaining the first three-dimensional initial coordinates of each reference key marker. 3. Using the reference key markers of the reference image frame as seed points, tracking their pixel coordinates in other registered images frame by frame, obtaining the pixel coordinates of other key markers. 4. For each other registered image, back-projecting the pixel coordinates of other key markers to the world coordinate system based on the camera parameters of the other registered images, obtaining the second three-dimensional initial coordinates of each other key marker. 5. Calculating the three-dimensional displacement of each sub-point in other registered images at frame t. 6. Determining the three-dimensional displacement as the dynamic mechanical prior data corresponding to the breast to be modeled.

[0012] In one optional implementation, an enhanced feature vector is generated based on prior surface geometry data, prior internal structure data, and prior dynamic mechanics data. This includes: extracting features from the prior surface geometry data to obtain a surface geometry feature vector corresponding to the breast to be modeled; extracting features from the prior internal structure data to obtain a physiological constraint feature vector corresponding to the breast to be modeled; extracting features from the prior dynamic mechanics data to obtain a dynamic mechanics feature vector corresponding to the breast to be modeled; weighted fusing the surface geometry feature vector, physiological constraint feature vector, and dynamic mechanics feature vector to generate a fused feature vector; and processing the fused feature vector based on a preset autoencoder to generate an enhanced feature vector.

[0013] In one optional implementation, a target breast model is generated based on the enhanced feature vector, including: obtaining a preset 3D latent generative model corresponding to the breast to be modeled; splitting the enhanced feature vector into a key vector and a value vector; using the initial latent features generated during the preset 3D latent generative model generation process as a query vector; calculating the association weight between the latent features and the enhanced feature vector in each injection layer through a cross-attention mechanism; performing a weighted calculation on the key vector and the value vector based on the association weight to obtain a weighted feature vector; updating the initial latent features based on the weighted feature vector to obtain updated latent features; outputting a virtual breast model based on the updated latent features from the preset 3D latent generative model; calculating the target loss value between the virtual breast model and the surface geometry prior data, the internal structure prior data, and the dynamic mechanical prior data based on a preset loss function; correcting the parameters in the preset 3D latent generative model based on the target loss value until the target loss value is less than the preset loss function value to obtain the target breast model; the target breast model includes a static breast 3D model and a dynamic breast 3D model.

[0014] In one optional implementation, the preset loss function includes geometric consistency loss, internal structural constraint loss, and biomechanical loss. Based on the preset loss function, a target loss value is calculated between the virtual breast model and the surface geometric prior data, internal structural prior data, and dynamic mechanical prior data, including: calculating the geometric consistency loss between the virtual breast model and the surface geometric prior data; calculating the internal structural constraint loss between the virtual breast model and the internal structural prior data; calculating the biomechanical loss between the virtual breast model and the dynamic mechanical prior data; and fusing the geometric consistency loss, internal structural constraint loss, and biomechanical loss to generate the target loss value.

[0015] The breast modeling method provided in this application integrates three types of heterogeneous data—target optical images, target imaging data, and target ultrasound data—from different perspectives, breaking the information limitations of a single data source. This provides comprehensive and multi-dimensional original support for subsequent prior data construction, ensuring the integrity and clinical relevance of the prior data. Surface geometry prior data is constructed based on the target optical images, providing precise surface morphology constraints for the model and preventing surface contour distortion in the generated model. Internal structure prior data is constructed by fusing target imaging data and target ultrasound data, addressing the problem of traditional 3D reconstruction's emphasis on surface over internal structure, ensuring the clinical accuracy of internal tissue distribution. Dynamic mechanical prior data is constructed, giving the model the ability to constrain the deformation patterns of breast soft tissue, ensuring that the generated model conforms to the clinical mechanical properties of "higher hardness, smaller deformation." Based on surface geometry prior data, internal structure prior data, and dynamic mechanical prior data, enhanced feature vectors are generated, establishing a spatial relationship between surface geometry and internal structure, balancing the contributions of geometric, physiological, and mechanical features; then, an autoencoder removes redundant dimensions, extracting the core enhanced feature vectors. This vector retains key clinical information while exhibiting low redundancy and high information density. As a strong constraint input to the generative model, it significantly improves the accuracy and efficiency of subsequent model generation. Using the enhanced feature vector as a constraint, the 3D generative model outputs a target breast model, achieving three core advantages: the model's surface morphology and internal structure closely match prior clinical data, meeting the accuracy requirements for surgical planning and implant design; it ensures global consistency of the model across multiple perspectives while accurately reproducing soft tissue deformation features such as breast ptosis and asymmetry; and it eliminates reliance on dedicated high-precision scanning equipment, enabling high-quality reconstruction based on conventional multi-source clinical data, facilitating widespread application in routine clinical settings. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic flowchart of a breast modeling method according to an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0020] According to an embodiment of the present invention, a breast modeling method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0021] This embodiment provides a breast modeling method that can be used in electronic devices. Figure 1 This is a flowchart of a breast modeling method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Acquire multiple target optical images of the breast to be modeled from different perspectives, as well as target image data and target ultrasound data of the breast to be modeled.

[0022] Specifically, the electronic device can receive multiple target optical images of the breast to be modeled from different perspectives, as well as target image data and target ultrasound data of the breast to be modeled, input by the user. It can also receive multiple target optical images of the breast to be modeled from different perspectives, as well as target image data and target ultrasound data of the breast to be modeled, sent by other devices.

[0023] Step S102: Based on the optical images, image data and ultrasound data of each target, construct the surface geometry prior data, internal structure prior data and dynamic mechanical prior data corresponding to the breast to be modeled.

[0024] Specifically, electronic devices can utilize the acquired multi-view optical images, video data, and ultrasonic data to construct surface geometric prior data for constraining the surface morphology of the model, internal structural prior data for constraining the distribution of internal tissues, and dynamic mechanical prior data for constraining the deformation law of the body position.

[0025] This step will be explained in detail below.

[0026] Step S103: Based on the prior data of surface geometry, prior data of internal structure, and prior data of dynamic mechanics, an enhanced feature vector is generated.

[0027] Specifically, electronic devices can generate enhanced feature vectors with low redundancy and high information density by using three types of prior data: surface geometry prior data, internal structure prior data, and dynamic mechanical prior data, through feature extraction, fusion, and enhancement processing.

[0028] This step will be explained in detail below.

[0029] Step S104: Based on the enhanced feature vector, generate the target breast model corresponding to the breast to be modeled.

[0030] Specifically, the electronic device can input the enhanced feature vector into a preset 3D latent generative model, driving the model to generate a target breast 3D model that matches the breast to be modeled.

[0031] This step will be explained in detail below.

[0032] The breast modeling method provided in this application integrates three types of heterogeneous data: target optical images from different perspectives (providing intuitive information on breast surface texture and contour), target imaging data (such as MRI / CT, ​​providing internal tissue anatomy), and target ultrasound data (reflecting the hardness and distribution characteristics of glands and fat). This breaks the information limitations of a single data source and provides comprehensive, multi-dimensional original support for subsequent prior data construction, ensuring the integrity and clinical relevance of the prior data. Surface geometric prior data is constructed based on target optical images, providing precise surface morphological constraints for the model and avoiding surface contour distortion in the generated model. Internal structure prior data is constructed by fusing target imaging data and target ultrasound data, solving the problem of traditional 3D reconstruction's "emphasis on surface, neglect of interior," ensuring the clinical accuracy of internal tissue distribution. Dynamic mechanical prior data is constructed, giving the model the ability to constrain the deformation laws of breast soft tissue, ensuring that the generated model conforms to the clinical mechanical characteristics of "higher hardness, smaller deformation." Based on prior data of surface geometry, internal structure, and dynamic mechanics, an enhanced feature vector is generated to establish a spatial relationship between surface geometry and internal structure, balancing the contributions of geometric, physiological, and mechanical features. An autoencoder then removes redundant dimensions to extract the core enhanced feature vector. This vector retains key clinical information while exhibiting low redundancy and high information density, serving as a strong constraint input for the generative model and significantly improving the accuracy and efficiency of subsequent model generation. Using the enhanced feature vector as a constraint, the 3D generative model outputs a target breast model, achieving three core advantages: the model's surface morphology and internal structure closely match the prior clinical data, meeting the accuracy requirements for surgical planning and prosthesis design; it ensures global coordination of the model from multiple perspectives while accurately reproducing soft tissue deformation features such as breast ptosis and asymmetry; and it eliminates the reliance on dedicated high-precision scanning equipment, enabling high-quality reconstruction based on conventional multi-source clinical data, facilitating widespread application in routine clinical environments.

[0033] This embodiment provides a breast modeling method that can be used in electronic devices. The process includes the following steps: Step S201: Acquire multiple target optical images of the breast to be modeled from different perspectives, as well as target image data and target ultrasound data of the breast to be modeled.

[0034] Please refer to the above description of step S101 for details on this step, which will not be repeated here.

[0035] Step S202: Based on the optical images, image data and ultrasound data of each target, construct the surface geometry prior data, internal structure prior data and dynamic mechanical prior data corresponding to the breast to be modeled.

[0036] Specifically, step S202 above may include the following steps: Step S2021: Identify each target optical image and determine the initial target camera parameter set corresponding to each target optical image.

[0037] Specifically, the electronic device can extract feature points from each target optical image using the SIFT (Scale Invariant Feature Transform) algorithm. The specific process is as follows: construct a Gaussian difference pyramid (6 layers by default), detect local extrema, and complete sub-pixel localization without additional filtering (retaining all feature points). A 128-dimensional SIFT descriptor is generated for each feature point and normalized to eliminate the influence of illumination. This specific process is existing technology, and this application embodiment will not describe this step in detail.

[0038] Then, one target optical image is selected as the reference image, and the rest are images to be matched. The electronic device calculates the Euclidean distance between each descriptor of the image to be matched and all descriptors of the reference image, and selects the one with the smallest distance as the initial matching pair. A distance ratio threshold of 0.7 is set, and matching pairs with a "nearest neighbor distance / second nearest neighbor distance" ≤ 0.7 are retained, while obvious mismatches are eliminated to obtain candidate matching pairs.

[0039] Next, the RANSAC iteration count is set to 1000, and 8 candidate matching pairs are randomly selected in each iteration. The fundamental matrix F (satisfying) is then solved. (where x1 and x2 are the homogeneous coordinates of the matching points). The electronic device calculates the reprojection error of all candidate matching pairs and retains matching pairs with an error ≤ 2 pixels as valid matching pairs. The fundamental matrix F with the largest number of valid matching pairs is selected to complete the matching pair purification. Then, the electronic device transforms the fundamental matrix F into the essential matrix E: E = K T FK (where K is the initial intrinsic parameter matrix of the camera). Then, the electronic device performs SVD decomposition on the intrinsic matrix E, obtaining 4 possible pose solutions. Based on "depth consistency", the unique valid rotation matrix R and translation vector t are selected, which are the camera extrinsic parameters. Finally, with the goal of "minimizing the reprojection error of all valid matching points", the electronic device uses the LM algorithm to fine-tune the intrinsic parameters (focal length, principal point coordinates) and extrinsic parameters (R, t). Optimization stops when the average reprojection error is <0.5 pixels, resulting in the final initial target camera parameter set (intrinsic parameter matrix K + extrinsic parameter R / t).

[0040] Step S2022: Extract the midline contour of the sternum from the target image data.

[0041] Specifically, the electronic device can convert target image data (such as an MRI T1-weighted sequence) into a three-dimensional grayscale matrix I(x,y,z), where x and y are pixel coordinates and z is the slice number (corresponding to the scan slice). Then, the three-dimensional grayscale matrix I(x,y,z) is normalized, with grayscale values ​​for bone tissue concentrated in the range of 220-255 and soft tissue concentrated in the range of 0-200, facilitating subsequent thresholding. Next, the electronic device performs thresholding on the normalized three-dimensional grayscale matrix I(x,y,z), setting a grayscale threshold ≥220 (corresponding to bone tissue) to extract the chest wall bone region. Then, the electronic device uses morphological opening operations (erosion followed by dilation) to eliminate minor noise in the bone region, obtaining a clear chest wall bone mask.

[0042] The electronic device employs a skeleton extraction algorithm (3D thinning algorithm) to extract the centerline of the bones from the chest wall skeleton mask. Then, along the left-right symmetrical direction of the breast, the sternal midline contour (a straight line segment perpendicular to the ground) is selected to eliminate interference from other chest wall bones. Next, the electronic device transforms the extracted sternal midline contour into a straight line equation in the world coordinate system using a registration transformation matrix (the registration result between image data and optical image): y=kx+b, where k is the slope and b is the intercept.

[0043] Step S2023: Based on the sternal midline contour, the initial target camera parameter set is corrected to obtain the optimized target camera parameter set.

[0044] Specifically, the electronic device can project the breast region in each target optical image onto the world coordinate system based on the initial target camera parameter set, using a camera projection model (intrinsic parameters + extrinsic parameters) to obtain a 3D point cloud. Then, with the sternal midline (straight line equation y=kx+b) as the axis of symmetry, the volume difference between the left and right breasts is calculated: ΔV=|V left -V right ∣ / (V left +V right ()×100%. The electronic device calculates the Euclidean distance between the left and right breast contour curves. For each point on the left and right breast contour curves, it calculates the distance to the axis of symmetry and calculates the standard deviation σ of the Euclidean distance.

[0045] Then, the electronic device determines whether the symmetry deviation exceeds the standard based on the calculated volume difference between the left and right breasts and the standard deviation σ of the Euclidean distance. For example, if ΔV > 5% or σ > 2mm, it is determined that the symmetry deviation exceeds the standard.

[0046] If the symmetry deviation exceeds the limit, the electronic device can adjust only the camera extrinsic parameters (rotation matrix R, translation vector t) in the initial target camera parameter set, while the intrinsic parameters are determined by the camera hardware characteristics and remain unchanged.

[0047] The specific correction process includes: The electronic device can calculate the offset Δt of the breast center in the world coordinate system and adjust the camera translation vector t to align the breast center with the midline of the sternum. The adjustment range of the translation vector t is ≤0.05m to avoid 3D reconstruction distortion caused by large adjustments to the camera position. The electronic device can calculate the tilt angle Δθ of the camera pose and adjust the rotation matrix R to make the camera optical axis perpendicular to the midline of the sternum. The adjustment angle of the rotation matrix R is ≤3° to ensure that the corrected camera pose conforms to the actual shooting scene. The electronic device repeats step 1, recalculating the projection consistency of the multi-view images based on the corrected camera parameters; if the difference in volume between the left and right breasts is ≤5% and the standard deviation of the Euclidean distance between the left and right breast contours is ≤2mm, the symmetry constraint is satisfied, thus obtaining the optimized target camera parameter set (intrinsic parameters remain unchanged, extrinsic parameters satisfy the symmetry constraint).

[0048] Step S2024: The SfM+MVS algorithm is used to perform dense matching on each target optical image, and the initial depth image corresponding to each target optical image is output to generate an initial depth image set.

[0049] Specifically, the electronic device can call the SfM+MVS fusion algorithm to perform dense matching of the optical images of each target, output the initial depth image corresponding to each target optical image, and generate an initial depth image set. (Recommended tool: COLMAP+PMVS2, this step is existing technology and will not be described in detail here).

[0050] The MVS algorithm, based on an optimized target camera parameter set, performs pixel-level dense matching of breast regions from adjacent viewpoints, calculating the disparity (the difference in pixel position under different viewpoints) of each pixel, with a matching accuracy ≤1 pixel. Then, based on the relationship between disparity and camera focal length, the disparity is converted into depth values, with the depth value range set to 0.5~2.0m (consistent with the actual spatial depth of the human breast). The electronic device can focus on optimizing the depth calculation of key areas such as the nipple and breast edge, ensuring that the depth error in these areas is ≤0.3mm. Finally, a set of initial depth images from multiple viewpoints is obtained (each optical image corresponds to one depth map, with depth values ​​corresponding one-to-one with pixels).

[0051] Step S2025: Based on the target ultrasound data, adjust the initial depth image set to generate physiological constraint depth images corresponding to each initial depth image.

[0052] Specifically, the electronic device can use an affine transformation matrix (the registration result of the target ultrasound data and the target optical image) to map the hardness grayscale matrix corresponding to the target ultrasound data to the pixel space of the initial depth image, so that each pixel in the initial depth image corresponds to a hardness level (1~5 points), forming a pixel-level association of "depth value - hardness level". Then, based on the anatomical rules of the breast, the correspondence between hardness and surface morphology is as follows: Hardness 3~5 points (glandular region): the glandular tissue is dense, corresponding to the small protrusions on the surface of the breast (the depth value should be slightly higher than the surrounding fat area, with a difference of 0.3~0.5mm); Hardness 1~2 points (fat area): the fat tissue is loose, corresponding to the smooth area on the surface of the breast (the depth value fluctuates less, with a standard deviation ≤0.2mm).

[0053] The electronic device iterates through each pixel in the depth map, checking whether its depth value conforms to the above association rules, and filters out pixels with deviations. For example, the depth value of the glandular region is lower than that of the surrounding fat region, or the depth value of the fat region changes abruptly (fluctuation > 0.5 mm).

[0054] Then, while keeping the overall shape unchanged, the electronic device makes local adjustments to the deviation pixels, taking into account both visual matching results and physiological characteristics, to generate physiologically constrained depth images corresponding to each initial depth image.

[0055] Step S2026: Based on the optimized target camera parameter set, process the physiological constraint depth images corresponding to each initial depth image to obtain the surface geometric prior data corresponding to the breast to be modeled.

[0056] Specifically, step S2026 above may include the following steps: Step a1: Based on the optimized target camera parameter set, project each pixel in the physiologically constrained depth image onto a unified world coordinate system to generate initial dense point cloud data.

[0057] Specifically, for each pixel (u,v) in each physiologically constrained depth image, the electronic device can read the corresponding depth value Z (in meters) from the physiologically constrained depth image. Then, based on the target camera intrinsic parameters in the optimized target camera parameter set, the pixel coordinates (u,v) are converted into three-dimensional coordinates (x,y,Z) in the camera coordinate system, calculated using the following formula: ,in,( , ) is the coordinate of the principal point, and f is the focal length.

[0058] Based on the camera extrinsic parameters (rotation matrix R and translation vector t) in the optimized target camera parameter set, the formula for converting the point (x, y, Z) in the camera coordinate system to the point (X, Y, Z) in the unified world coordinate system is as follows: ,in, It is the inverse of the rotation matrix.

[0059] The electronic device traverses all pixels of all physiologically constrained depth images, integrates all calculated 3D points (X,Y,Z), and generates initial dense point cloud data.

[0060] Step a2: Optimize the initial dense point cloud data using a region growing algorithm to generate the target dense point cloud data.

[0061] Specifically, the electronic device can call the region growing algorithm in the PCL point cloud library and set the following core screening conditions: Normal vector deviation threshold ≤ 15°. This condition ensures that the normal vector directions of adjacent points within the growing region are consistent, thus guaranteeing good smoothness of the generated point cloud surface. Curvature threshold ≤ 0.02mm -1 This condition is used to remove isolated points with excessive curvature, which are often noise or artifacts.

[0062] The electronic device can select significant feature points on the surface of the breast to be modeled as seed points, such as the center point of the nipple. Then, starting from the seed point, the region growing algorithm iteratively adds neighboring points that meet the above two selection criteria to the current region until no further growth is possible. After growth is complete, points outside the region are considered outliers and are removed. The outlier percentage is required to be ≤1%. Then, the grown point cloud undergoes local smoothing, for example, using moving least squares (MLS), to eliminate jagged noise while preserving physiological protrusion details at the 0.5mm level, finally generating the target dense point cloud data.

[0063] Step a3: The Poisson reconstruction algorithm is used to transform the dense point cloud data of the target into an initial triangular mesh model.

[0064] Specifically, the electronic device can call the Poisson reconstruction algorithm in the PCL point cloud library and adjust the following core parameters according to the characteristics of breast soft tissue: Smooth Factor: Set to 0.8. This value is higher than the default value for general object reconstruction (usually 0.5), aiming to improve the overall smoothness of the mesh to better adapt to the soft morphological features of the breast without sharp edges. Reconstruction Depth: Set to ≥10 levels. A higher reconstruction depth ensures that the algorithm can capture and restore the fine details in the target point cloud. Sampling Density: Set to correspond to the input point cloud density (5 points / mm²) to ensure that the generated initial mesh has sufficient resolution.

[0065] Specifically, the electronic device can fit a tangent plane based on neighborhood points, calculate the normal vector of each point in the dense point cloud data of the target, and unify the direction of the normal vector (towards the outside of the object). The electronic device can treat the dense point cloud data of the target as sampling points in three-dimensional space, and fit a continuous implicit surface (describing the object surface) using the Poisson equation. The fitted implicit surface is then subjected to isosurface extraction (MarchingCubes algorithm) to generate a triangular mesh model.

[0066] Step a4: Smooth the initial triangular mesh model using the Laplace smoothing algorithm to generate a breast-specific triangular mesh model corresponding to the breast to be modeled.

[0067] Specifically, the electronic device can invoke the Laplacian smoothing algorithm, setting the number of iterations to 5 (this 5 is an empirical value that can eliminate noise while preserving important geometric details to the greatest extent. Too few iterations will not have a noticeable effect, while too many will lead to over-smoothing and loss of details). The Laplacian smoothing algorithm achieves the smoothing effect of the mesh by adjusting the coordinates of each mesh vertex, moving it towards the average position of all its neighboring vertices.

[0068] The electronic device can identify areas with typically poor mesh quality, such as the lower part of the breast (the area connected to the chest wall) and the axillary side (an irregularly shaped area). For these areas, the electronic device can add 1-2 smoothing iterations to eliminate degenerate triangles (such as triangles that are too small or too elongated), optimize the mesh topology, and generate a breast-specific triangular mesh model. This ensures that there are no degenerate triangles or holes in the breast-specific triangular mesh model, that the average side length deviation of the mesh is ≤0.1mm, and guarantees the uniformity of the mesh.

[0069] Step a5: Based on the optimized target camera parameter set, map the depth value of each pixel in the physiological constraint depth image to the corresponding vertex of the breast-specific triangular mesh model, so that each vertex of the mesh has both three-dimensional coordinates and physiological depth constraints, thus obtaining the target breast triangular mesh model.

[0070] Specifically, for each vertex P(X,Y,Z) in the breast-specific triangular mesh model, the electronic device back-projects it onto the corresponding physiological constraint depth image based on the optimized target camera parameter set, obtaining the vertex's pixel coordinates (u',v') on the depth map. Since the vertex projection position may not completely coincide with the pixel center of the depth map, a bilinear interpolation algorithm is used to calculate the precise depth value Z' corresponding to vertex P based on the depth values ​​of the four pixels surrounding (u',v'). Then, the interpolated depth value Z' is assigned to vertex P, updating its Z coordinate (or stored as an additional attribute), thus obtaining the target breast triangular mesh model.

[0071] Step a6: Determine the target breast triangular mesh model as the prior surface geometry data corresponding to the breast to be modeled.

[0072] Specifically, the electronic device can save the physiological constraint depth map set as a 16-bit TIFF format to retain high-precision depth information, and save the target breast triangular mesh model as a PLY or OBJ format. This file contains the 3D coordinates of all vertices, the connection information of the triangle faces, and the depth value attribute corresponding to each vertex.

[0073] The electronic device generates the final prior data of breast surface geometry. This data includes: a set of physiologically constrained depth maps and a triangular mesh model of the target breast.

[0074] In an optional implementation, step S202 may further include the following steps: Step S2027: Using the target breast triangular mesh model as the registration reference, the target region contour of the target image data and the hardness grayscale matrix of the target ultrasound data are used as the registration data.

[0075] The target area includes at least one internal structure among glands, fat, and chest wall bones.

[0076] Specifically, the electronic device can select a target breast triangular mesh model as the registration reference, i.e., a fixed template. This breast-specific triangular mesh model has undergone geometric optimization and serves as the reference for the spatial position of the breast surface. All data to be registered must be mapped to the world coordinate system of this model.

[0077] Electronic devices can extract the core geometric features of a breast-specific triangular mesh model: vertex 3D coordinates, local curvature, and depth value distribution, as reference features for registration.

[0078] Then, at least one internal structural contour (gland, fat, chest wall bone) is extracted from the target image data and converted into a three-dimensional vector contour (.json / .ply format), ensuring that the contour coordinates are in the original image coordinate system and have not undergone spatial transformation. The electronic device can convert the hardness grayscale matrix of the target ultrasound data (which has been initially aligned with the optical image), with pixel values ​​corresponding to hardness of 1~5, into a three-dimensional matrix (if it is a two-dimensional ultrasound slice, it needs to be stacked into three-dimensional volume data), preserving the spatial resolution (0.1mm / voxel recommended).

[0079] Step S2028: The multimodal registration algorithm based on mutual information is used to register the data to be registered, and the final registration transformation matrix is ​​obtained.

[0080] Mutual Information (MI) measures the "shared information" between two random variables (surface mesh features and internal structure data), and is expressed as: MI(A,B) = H(A) + H(B) - H(A,B), where H(A) is the entropy of the breast-specific triangular mesh model features, H(B) is the entropy of the data to be registered, and H(A,B) is the joint entropy. A higher MI value indicates a better spatial match between the two sets of data.

[0081] Specifically, the electronic device can extract the coordinates of a predetermined number of key marker points from the breast-specific triangular mesh model, the target region contour of the target image data, and the hardness grayscale matrix of the target ultrasound data. These key marker points serve as initial values ​​for registration, reducing the number of iterations and avoiding local optima. For example, four key marker points are extracted: the nipple center point, the midpoint of the inframammary fold, the apex of the lateral edge of the breast, and the connection point between the chest wall and the breast.

[0082] The electronic device can select affine transformation (including six degrees of freedom: translation, rotation, and scaling) as the registration transformation type to adapt to the spatial deformation of the internal breast structure. Then, based on the coordinate correspondence of key marker points, the electronic device can calculate the initial affine transformation matrix T0, initially mapping the data to be registered to the reference coordinate system. Next, it extracts the geometric features (vertex depth, local curvature) of the target breast triangular mesh model to construct feature set A; and extracts features from the data to be registered (grayscale distribution of image contours, ultrasound hardness distribution) to construct feature set B. Finally, the electronic device calculates the current transformation T. k The mutual information value MI between A and B k .

[0083] Then, the electronic device can use gradient descent (or Powell's optimization algorithm) to iteratively adjust the affine transformation parameters (translation Δx / Δy / Δz, rotation angle Δα / Δβ / Δγ, scaling factor s), with the optimization objective being to maximize the mutual information value MI. The transformation matrix T is updated in each iteration. k+1 .

[0084] If the mutual information value fluctuates by ≤0.001 for three consecutive iterations, the convergence condition is confirmed, the iteration is stopped, and the final registration transformation matrix T is output. final。 If convergence is not achieved after more than 50 iterations, backtrack the accuracy of the marker points and re-initialize them.

[0085] Step S2029: Based on the final registration transformation matrix, map the target region contour and hardness grayscale matrix to the spatial position of the target breast triangular mesh model.

[0086] Specifically, the electronic device can determine the coordinates (X, X, Y) of each vertex of the target region contour. img ,Y img Zimg ), perform the transformation: (X grid ,Y grid Z grid )=T final ·(X img ,Y img Z img ,1) T (Note: Added homogeneous coordinate 1 to adapt to affine transformation matrix operations). Electronic devices provide coordinates (X, Y, Z) for each voxel in the hardness grayscale matrix. us ,Y us Z us Perform the same transformation to obtain the voxel coordinates in the coordinate system of the standard breast triangular mesh model.

[0087] Step S20210: The registered target region is converted into a first three-dimensional voxel model, and the registered hardness grayscale matrix is ​​converted into a second three-dimensional voxel model.

[0088] Specifically, the electronic device can create a three-dimensional voxel space (voxel resolution 0.1mm) based on the mapped target region contour (gland / fat / chest wall skeleton), and use a voxel filling algorithm (such as flood fill) to assign voxel values ​​to the region enclosed by the contour: gland region voxel value = 1, fat region = 2, chest wall skeleton = 3, background = 0; output the first three-dimensional voxel model (.nii / .mhd format), with voxel values ​​corresponding to the internal structure type.

[0089] The electronic device can construct a three-dimensional voxel space with the same resolution and spatial range as the first voxel model based on the mapped hardness grayscale matrix. The hardness level (1~5 points) of each voxel is assigned to the corresponding spatial position, and missing areas (not covered by ultrasound) are filled by neighborhood interpolation. The second three-dimensional voxel model is then output, with the voxel values ​​corresponding to the hardness levels.

[0090] Step S20211: Traverse each vertex of the target breast triangular mesh model and calculate the shortest Euclidean distance from each vertex to the first three-dimensional voxel model corresponding to each internal structure in the target region in three-dimensional space.

[0091] Specifically, the electronic device traverses each vertex (Xv, Yv, Zv) of the target breast triangular mesh model and calculates the shortest Euclidean distance from each vertex to different internal structures (glandules, fat, chest wall bones) in the first voxel model: ,in( , , Let be the voxel coordinates of the target structure. The electronic device can use a voxel neighborhood search (searching only voxels within a 10mm radius of the vertex) to avoid global traversal. The electronic device can store the shortest distance value (e.g., d) from each vertex to each internal structure. 腺体d 脂肪 d 骨骼 ).

[0092] Step S20212: Use the internal structure corresponding to the shortest Euclidean distance as the structural attribute label of the vertex.

[0093] Specifically, for each vertex of the target breast triangular mesh model, the electronic device can compare its shortest distance to each internal structure and select the structure with the smallest distance as the structural attribute label for that vertex. If the distance difference between multiple structures is ≤0.1mm (such as the junction of gland and fat), it is marked as "transition region" (label=4). For example, gland, fat 2, chest wall bone 3, transition region 4.

[0094] Step S20213: For each vertex of the target breast triangular mesh model, extract the hardness level of the ultrasonic hardness voxel corresponding to the spatial position of the vertex, and use it as the hardness attribute label of the vertex.

[0095] Specifically, for each vertex (Xv, Yv, Zv) of the target breast triangular mesh model, the corresponding voxel position in the second 3D voxel model is located. If the vertex coordinates do not perfectly match the voxel center, trilinear interpolation is used to calculate the hardness value at that position (retaining one decimal place). The electronic device uses the interpolated hardness level (1~5 points) as the hardness attribute label for the vertex; for vertices in areas not covered by ultrasound, the hardness value is marked as 0, and the note "No ultrasound data" is added.

[0096] Step S20214: The first three-dimensional voxel model and the second three-dimensional voxel model are fused to generate the internal structure voxel model corresponding to the breast to be modeled.

[0097] Specifically, the electronic device can construct a fused voxel space with the same spatial range and resolution as the first and second 3D voxel models. Then, for each voxel, the structure type and hardness level information are fused. The voxel value encoding rule is: Fusion value = Structure type × 10 + Hardness level (e.g., gland + hardness 3 points = 13, fat + hardness 2 points = 22). Example: Glandular hardness level is 3, fusion value is 13; fat hardness level is 2, fusion value is 22.

[0098] Next, the electronic device can perform morphological closing operations (eliminating tiny holes) on the fused voxel model to ensure the connectivity of the voxels, remove background voxels (fusion value = 0), and retain the effective internal structural regions, thereby generating an internal structural voxel model (.nii format) corresponding to the breast to be modeled. This model contains both internal anatomical structure and physiological rigidity information.

[0099] Step S20215: Based on the internal structure voxel model, determine the prior data of the internal structure of the breast to be modeled.

[0100] Specifically, electronic devices can determine the internal structure voxel model as the prior data of the internal structure of the breast to be modeled.

[0101] In an optional implementation, the prior data on internal structure may further include glandular volume percentage, fat layer thickness distribution, and curvature deviation of the chest wall centerline and breast base contour line; step S20215 above may include the following steps: Step b1: Based on the internal structural voxel model, obtain the three-dimensional volume of the glandular region in the target area and the total volume of the breast corresponding to the breast to be modeled.

[0102] Specifically, the electronic device can traverse all voxels of the internal structural voxel model and count them according to their attribute labels. Glandular voxel count (N) gland ): Filter out voxels with the structural attribute label "gland" (the tens digit of the fusion value code is 1, such as 10 / 11 / 12…15), and count the total number; total breast voxel count (N total ): Select voxels with the structural attribute label "gland + fat + chest wall" (tens digits are 1 / 2 / 3, such as 10 / 20 / 30…35), and count the total number (excluding background voxels, i.e., voxels with a fusion value of 0). The resolution of the internal structure voxel model is d×d×d (d=0.1mm recommended). The volume calculation formula for a single voxel is: V voxel =d×d×d, substituting d=0.1mm. Electronic equipment can calculate gland volume: V gland =N gland ×V voxel Total breast volume: V total =N total ×V voxel .

[0103] Step b2: Based on the three-dimensional volume of the glandular region in the target area and the total volume of the breast corresponding to the breast to be modeled, the proportion of glandular volume is obtained.

[0104] Specifically, the electronic device calculates the gland volume percentage using the formula: R gland =V gland / V total ×100%.

[0105] Among them, the physiological standard range is: 20% ≤ R gland ≤60% (adapted to the physiological characteristics of adult female breasts). If Rgland <20%: indicates that the proportion of glandular tissue is too low (fatty breast), and it is necessary to check whether glandular tissue is misclassified as fat; if Rgland >60%: indicates that the proportion of glandular tissue is too high (dense breast), and it is necessary to check whether fat is misclassified as glandular tissue.

[0106] Step b3: Based on the target breast triangular mesh model, measure the average thickness of the fat layer corresponding to multiple anatomical orientations of the breast to be modeled.

[0107] Specifically, the electronic device can use the maximum outline of the target breast triangular mesh model as a reference, combined with the midline of the sternum and the inframammary fold, to divide the area into four symmetrical anatomical regions. For example, these are: an upper breast region, encompassing the area from the apex of the upper breast edge to the midpoint of the breast, with a width ≥ 2 cm; a lower breast region, encompassing the area from the midpoint of the breast to the inframammary fold, with a width ≥ 2 cm; a medial breast region, encompassing the area from the midline of the sternum to the medial edge of the breast, with a width ≥ 2 cm; and a lateral breast region, encompassing the area from the apex of the lateral edge of the breast to the midline of the breast, with a width ≥ 2 cm. It is ensured that the width of all four regions is consistent, covering the main body of the breast and avoiding non-fatty areas such as the nipple and chest wall.

[0108] Then, the electronic device can use a uniform sampling method (with grid vertex spacing ≤ 0.5cm) to collect N≥50 sampling points for each anatomical orientation, ensuring stable statistical results. Each sampling point P... k,i The three-dimensional coordinates (i=1 / 2 / 3 / 4 correspond to 4 anatomical orientations, k=1, N is the sampling point number) must exist simultaneously in the surface mesh and the internal voxel model, with no missing data.

[0109] Next, the electronic device calculates the surface normal direction (perpendicular to the breast surface and pointing inwards) of the mesh vertices corresponding to the sampling points. It then traverses the voxel model inwards along the normal direction from the surface vertex, recording two key locations. The starting point of these two key locations is the breast surface mesh vertex (corresponding to the skin voxel, without structural labels); the ending point is the boundary voxel center between the fat voxel (label=2) and the glandular / chest wall voxel (label=1 / 3). The electronic device calculates the straight-line distance between the starting and ending points, which is the fat layer thickness dk,i at that sampling point, converted to cm (1mm=0.1cm).

[0110] Formula for calculating the average thickness of the fat layer in multiple anatomical orientations of the breast to be modeled by electronic devices: Where: i=1 (upper part), i=2 (lower part), i=3 (inner side), i=4 (outer side); T fat,i The average fat layer thickness (in cm) is the thickness of the fat layer at the i-th anatomical orientation.

[0111] Then, the electronic device can calculate the overall average fat layer thickness of the breast to be modeled based on the average fat layer thickness at various anatomical locations, using the formula: .

[0112] Electronic devices can calculate the standard deviation of thickness using the following formula: .

[0113] The uniformity of electronic devices can be determined according to the following rule: σ fat <0.3cm: The fat layer is evenly distributed; 0.3cm≤σ fat ≤0.8cm: The distribution is basically uniform; σ fat >0.8cm: Large distribution differences (e.g., the outer fat layer is much thicker than the inner one).

[0114] The electronic device outputs a fat layer thickness analysis report, including: the average thickness T at four anatomical locations. fat,i Overall average thickness T fat Standard deviation σ fat .

[0115] Step b4: Based on the internal structural voxel model, obtain the spatial positional relationship between the chest wall centerline and the breast base contour line, and calculate the curvature deviation between the chest wall centerline and the breast base contour line.

[0116] Specifically, the electronic device can select chest wall skeletal voxels (label=3) from the internal structural voxel model, extract their central axis (along the midline of the sternum), and fit it as a three-dimensional curve L. chest (s) (s is the curve arc length parameter).

[0117] Specifically, the electronic device extracts the contour line of the region connecting the breast and the chest wall (i.e., the breast base) from the target breast triangular mesh model and fits it as a three-dimensional curve L. base (s), ensuring that the arc length parameter s is consistent with the range of the centerline of the chest wall.

[0118] Electronic devices for L chest (s) and L base (s) Perform circular arc fitting separately and calculate their respective radian parameters: Curvature of the centerline of the chest wall: θ chest (Unit: °); Curve of the breast base contour: θ bas e (unit: °); Electronic devices can use the least squares method to fit circular arcs, ensuring that the fitting error is ≤0.1°.

[0119] Then, the electronic device can calculate the radian deviation using the following formula: Unit: ° (angle), rounded to one decimal place.

[0120] In one optional implementation, each target optical image is a temporal optical image covering three body positions: standing, lying flat, and leaning forward at a preset degree; step S202 may further include the following steps: Step S20216: Using the first frame of the target optical image in a standing position as the reference image frame, extract other SIFT feature points from the other frames of optical images other than the reference image frame using a registration algorithm based on feature points.

[0121] Specifically, the electronic device can select the first frame of the target optical image in the standing position as the reference image frame (denoted as I0). This frame must meet the requirements of natural breast shape, no motion blur, and uniform illumination.

[0122] Then, the electronic device can call the OpenCV function `cv2.SIFT_create()`, setting the parameters: `nFeatures=2000` (number of feature points) and `nOctaveLayers=3` (number of pyramid layers). For each frame of the image except I0 (denoted as It, t=1,2,...,T), SIFT feature points are extracted frame by frame (denoted as Ft, i.e., "other SIFT feature points"). Feature points with a response value ≥0.01 are retained, and edge noise points are removed to ensure that there are ≥1000 effective feature points per frame.

[0123] Step S20217: For each other optical image frame, match the other SIFT feature points corresponding to the other optical images frame with the reference SIFT feature points corresponding to the reference image frame to determine the registration transformation matrix corresponding to the other optical images frame.

[0124] Specifically, the electronic device can use the reference SIFT feature point (F0) of the baseline image frame I0 as a reference and employ a FLANN matcher (fast nearest neighbor search) to match feature points Ft of other frames It. Then, the electronic device can use "K-nearest neighbor matching (K=2)" with a distance threshold ≤0.75 (i.e., the distance ratio between the best and second-best matches ≤0.75) to eliminate false matches. Each frame has ≥500 valid matching pairs, with a matching accuracy ≥95%.

[0125] K-Nearest Neighbor (KNN) matching is a classic feature matching algorithm. Its core principle is to find the K nearest feature points (K=2, representing the best and second-best matches) in the SIFT feature point library of the reference image frame for each SIFT feature point in the frame to be matched. For each feature point to be matched, the two matching results (denoted as match1 (best) and match2 (second-best)) are calculated as follows: distance ratio = match1.distance / match2.distance. If the distance ratio is ≤0.75, it is considered a valid match (the best-matched feature point is far superior to the second-best, indicating a true breast feature match, thus the best-matched feature point is matched with the reference SIFT feature point to obtain a matching pair). If the distance ratio is >0.75, it is considered a false match (the best and second-best similarities are close, making it impossible to determine the true correspondence, and they are directly discarded).

[0126] Then, the electronic device can use the RANSAC algorithm to solve the affine transformation matrix (denoted as Mt) based on the matched feature point pairs. The transformation type is 6 degrees of freedom (translation + rotation + scaling).

[0127] Specifically, the electronic device can randomly select 6 points from N matching pairs (at least 6 points are needed to solve a 6-DOF affine matrix) as "initial interior points", and then fit the initial affine transformation matrix using these 6 points. The electronic device can perform calculations on all matching pairs. The transformed reprojection error is used to select points with an error ≤ 1 pixel as "inliers". This process is repeated 2000 times, and the matrix with the most "inliers" is selected as the optimal matrix M. t This allows us to obtain the registration transformation matrix corresponding to other frames of optical images. The mask is a 0 / 1 array, where 1 represents an interior point (a matching pair that conforms to the transformation rule) and 0 represents an exterior point (a residual mismatch).

[0128] Step S20218: Based on each registration transformation matrix, register each other frame optical image with the reference image frame to obtain other registered images.

[0129] Specifically, the electronic device can call the OpenCV cv2.warpAffine() function, input the optical images It of each other frame and the corresponding registration transformation matrix Mt of the other frame optical images, and output the other registered images It′ after registration.

[0130] Step S20219: Mark the pixel coordinates of a preset number of reference key marker points in the reference image frame.

[0131] Specifically, the electronic device can annotate the pixel coordinates of a preset number of reference key marker points in the reference image frame. The preset number can be 5, 6, or other numbers. For example, the reference key marker points are: P1: nipple center point, core motion reference point; P2: midpoint of the inframammary fold, basal motion reference point; P3: apex of the lateral edge of the breast, lateral morphological reference point; P4: apex of the medial edge of the breast, medial morphological reference point; P5: apex of the upper edge of the breast, upper morphological reference point.

[0132] Step S20220: Based on the camera parameters of the reference image frame, back-project the pixel coordinates of each reference key marker point to the world coordinate system to obtain the first three-dimensional initial coordinates of each reference key marker point.

[0133] Specifically, the electronic device can read the optimized camera parameters (intrinsic parameter K, extrinsic parameter R0,t0) of the reference image frame I0: .

[0134] The electronic device for each reference key marker point (u 0,i ,v 0,i Assuming an initial depth value Z0 (which can be obtained from a previous depth map), calculate the coordinates (x, y) in the camera coordinate system. 0,i ,y 0,i ,Z0): .

[0135] Electronic devices can use inverse extrinsic parameter transformation to convert the camera coordinate system to the world coordinate system, obtaining the first three-dimensional initial coordinates (X). 0,i ,Y 0,i Z 0,i ): .

[0136] Step S20221: Using the reference key marker point of the reference image frame as the seed point, track its pixel coordinates in each of the other registered images frame by frame to obtain the pixel coordinates of other key marker points.

[0137] Specifically, electronic devices can employ the KLT optical flow method (Kanade-Lucas-Tomasi) to adapt for sub-pixel-level tracking of dynamic markers. This is based on a reference key marker (u... 0,i ,v 0,i () is the seed point, and the first frame of the image is used as the input for other registered images. Electronic devices (t=1T), iteratively calculate the pixel coordinates (u) of the marker point. t,i ,v t,i(Pixel coordinates of other key markers). Set the tracking window size to 15×15 pixels and the pyramid layer number to 3, ensuring tracking accuracy ≤ 0.1 pixels. If the optical flow vector magnitude of a tracking point is > 5 pixels (determined as a sudden motion change), use the coordinate interpolation of the previous frame for correction; if tracking fails (e.g., a marker is occluded), pause tracking and mark it as "missing," and reinitialize in subsequent frames. The electronic device obtains other registered images for each frame. Pixel coordinate table of each marker point (u t,i ,v t,i ), t=1T, i=15.

[0138] Step S20222: For each of the other registered images, based on the camera parameters of the other registered images, back-project the pixel coordinates of each of the other key marker points to the world coordinate system to obtain the second three-dimensional initial coordinates of each of the other key marker points.

[0139] Specifically, for other key markers (ut,i,vt,i), the electronic device acquires the corresponding depth value Zt (matched from the depth map sequence), transforms the position information of the other key markers from pixel coordinates to the camera coordinate system, and obtains... The formula is: Then, the position information corresponding to other key marker points is transformed from the camera coordinate system to the world coordinate system: This yields the second set of initial 3D coordinates. The formula is: Unify the 3D coordinates of all frames to the world coordinate system of the reference frame to eliminate the influence of camera pose changes.

[0140] Step S20223: For each type of sub-point, calculate the three-dimensional displacement of each sub-point in other registered images in frame t.

[0141] Specifically, for each seed point (marker point i), the electronic device calculates the three-dimensional displacement ΔP of the seed point (marker point i) in other registered images in frame t. t,i The formula is:

[0142] For example, Table 1 below is a schematic table of the three-dimensional displacement of the marked points.

[0143] Table 1 is a schematic table of the three-dimensional displacement of the marked points.

[0144] Step S20224: The three-dimensional displacement is determined as the prior dynamic mechanical data corresponding to the breast to be modeled.

[0145] Specifically, the electronic device determines the three-dimensional displacement as the dynamic mechanical prior data corresponding to the breast to be modeled.

[0146] Step S203: Based on the prior data of surface geometry, prior data of internal structure, and prior data of dynamic mechanics, an enhanced feature vector is generated.

[0147] Specifically, step S203 above may include the following steps: Step S2031: Extract features from the prior surface geometry data to obtain the surface geometry feature vector corresponding to the breast to be modeled.

[0148] Specifically, the electronic device can use PointNet++ as the core encoder, and make two customized modifications to the surface geometry prior data, namely the target breast triangular mesh model. First, the input layer is expanded to support triangular mesh data input with 500,000 vertices, adapting to the high-detail geometric model of the breast; second, a new clinically relevant area enhancement module is added: the mesh vertices in the 3cm area around the nipple and the glandular concentrated area are assigned a weight of 1.5 times, improving the accuracy of feature extraction in key areas.

[0149] The core encoder output layer is initialized, the feature dimension mapping path is set, and finally a 352-dimensional surface geometric feature vector is output.

[0150] As shown in the table below, Table 2 is a schematic diagram of the calculation and coding of three types of core morphological features.

[0151] Table 2. Schematic diagram of calculation and coding of three types of core morphological features

[0152] Electronic devices combine 256-dimensional curvature features, 64-dimensional symmetry features, and 32-dimensional sag features into a 352-dimensional surface geometric feature vector.

[0153] Step S2032: Extract features from the prior data of the internal structure to obtain the physiological constraint feature vector corresponding to the breast to be modeled.

[0154] Specifically, electronic devices can use lightweight CNN models to extract elastic feature parameters from prior data of internal structure, adapt to the high-resolution characteristics of target ultrasound images, and ensure computational efficiency, as shown in Table 3 below, which is the network architecture table of the CNN model.

[0155] Table 3 Network Architecture of CNN Models

[0156] The electronic device can extract features from elastic characteristic parameters to obtain clinical physiological parameters (64 dimensions). These clinical physiological parameters include breast volume, base width, nipple height, areola diameter, glandular volume ratio, and average fat layer thickness (a total of 6 items, all core assessment parameters for clinical breast augmentation / ptosis correction surgery). Then, each core parameter is normalized to [0,1] (eliminating the influence of different units / numerical ranges, such as volume units cm³ and diameter units mm). These are mapped to 64-dimensional clinical physiological parameters through a fully connected layer, preserving nonlinear correlations between parameters through high-dimensional mapping (such as the negative correlation between glandular volume ratio and fat layer thickness). Finally, the electronic device combines 128-dimensional elastic features with 64-dimensional physiological parameter features to form a 192-dimensional physiological constraint feature vector, taking into account both "tissue mechanical properties" and "anatomical parameter properties."

[0157] Step S2033: Extract features from the prior data of dynamic mechanics to obtain the dynamic mechanics feature vector corresponding to the breast to be modeled.

[0158] Specifically, electronic devices can be adapted to the mechanical properties of soft tissue based on Hooke's Law. For example, For elastic modulus E: 1.5~2.5 kPa in glandular areas, 0.5~1.0 kPa in adipose areas (measured values ​​in soft tissue, consistent with clinical biomechanical studies); For shear modulus G: according to the formula... Calculate (Poisson's ratio ν = 0.45, a common value for soft tissue).

[0159] Electronic devices can compress the global distribution characteristics of E and G (such as the mean / variance of E in the glandular region and the mean / variance of E in the adipose region) into 64-dimensional mechanical parameter characteristics through a fully connected layer, while preserving the spatial distribution characteristics of the mechanical parameters.

[0160] Next, the electronic device can use an LSTM network to extract features from the dynamic mechanical prior data, obtaining deformation trend parameters and 32 deformation trend features. The temporal correlation of the time-series displacement data is adapted, as shown in Table 4, which is the network architecture table of the LSTM network. Table 4 Network Architecture of LSTM Networks

[0161] The electronic device will stitch together 64-dimensional mechanical parameter features and 32-dimensional deformation trend features to obtain a 96-dimensional dynamic mechanical feature vector.

[0162] Step S2034: Weighted fusion of surface geometric feature vector, physiological constraint feature vector and dynamic mechanical feature vector to generate fused feature vector.

[0163] Specifically, electronic devices can determine the surface geometric feature vector as the query vector and the physiological constraint feature vector as the key vector and value vector, respectively.

[0164] Electronic devices can calculate the similarity between the query and the key (using the scaled dot product formula): Where Q = 352-dimensional surface geometry vector, K = 192-dimensional physiological constraint vector, and d k =The dimension of Key (192) is used for scaling to avoid excessively large values; QK T Generate a 352×192 similarity matrix (the similarity between each surface feature dimension and each internal feature dimension).

[0165] Electronic devices can set thresholds to filter strongly correlated pairs: only feature pairs with a similarity of ≥0.6 are retained (e.g., the similarity between "nipple high curvature dimension" and "gland high hardness dimension" is 0.8, so they are retained; the similarity between "fat area curvature dimension" and "gland hardness dimension" is 0.3, so they are removed).

[0166] Then, the electronic device can multiply the weight of the filtered feature pairs by 1.2 times (configurable) to further strengthen clinically important associations (such as nipple-gland, inframammary fold-fat layer).

[0167] Next, the electronic device can perform a weighted summation of the enhanced weight matrix and the Value vector: V′ = weight matrix × V (outputting 128 dimensions; dimension compression is used to adapt the dimensions to the concatenated Query vector). The weighted internal features (V′) are then concatenated with the original Query vector to obtain the weighted concatenated features F. cross :F cross =concat(Q,V′)=352+128=480 dimensions. The 480-dimensional vector not only preserves the complete surface geometric features, but also incorporates internal structural features that are strongly correlated with the surface (such as nipple features binding gland features).

[0168] Then, the electronic device can acquire the weighted splicing feature F. cross The weight information corresponding to the physiological constraint feature vector and the dynamic mechanical feature vector is obtained, and then a weighted calculation is performed based on the obtained weight information to obtain a 768-dimensional fused feature vector. The specific formula is: F fusion =0.55×F cross +0.25×F physio +0.2×F dynamic Among them, F physio F is the physiological constraint feature vector. dynamic This represents the dynamic mechanical characteristic vector.

[0169] Step S2035: Process the fused feature vector based on the preset autoencoder to generate an enhanced feature vector.

[0170] Specifically, the electronic device can employ a symmetrical autoencoder with a structure of 3 encoder layers and 3 decoder layers. The encoder part (dimensionality reduction path): 768 dimensional → 512 dimensional → 256 dimensional → 2048 dimensional (bottleneck layer); activation function: ReLU, ensuring non-linear feature representation; the decoder part (reconstruction path): 2048 dimensional → 256 dimensional → 512 dimensional → 768 dimensional; loss function: mean squared error (MSE), requiring reconstruction error ≤ 0.01 to ensure no loss of feature information. The electronic device inputs the 768-dimensional fused feature vector into the trained pre-defined autoencoder, extracts the 2048-dimensional feature vector from the bottleneck layer (the core feature with the lowest redundancy), and generates a 2048-dimensional enhanced feature vector, which serves as a strong constraint input to the pre-defined 3D latent generative model.

[0171] Step S204: Based on the enhanced feature vector, generate the target breast model corresponding to the breast to be modeled.

[0172] Specifically, step S204 above may include the following steps: Step S2041: Obtain the preset 3D potential generative model corresponding to the breast to be modeled.

[0173] Specifically, the electronic device can receive a preset 3D latent generative model corresponding to the breast to be modeled, input by the user, or it can receive a preset 3D latent generative model corresponding to the breast to be modeled, sent by the device. The preset 3D latent generative model can be a 3D latent diffusion model or a 3DGAN (adapted to the generation of complex breast geometry), pre-trained on a breast 3D mesh dataset.

[0174] The preset 3D latent generation model includes a "latent feature generation layer + multiple feature injection layers + mesh generation layer", where the "injection layer" is the core of feature constraints (each layer can inject external constraint features).

[0175] Step S2042: The enhanced feature vector is split into a key vector and a value vector.

[0176] Specifically, electronic devices can divide enhanced feature vectors according to preset dimensions to obtain key vectors and value vectors (e.g., the first 1024 dimensions are key vectors, and the last 1024 dimensions are value vectors). Among them, the key vector retains the core geometric and physiological features; the value vector retains the core mechanical and deformation features.

[0177] Step S2043: Use the initial latent features in the generation process of the preset 3D latent generation model as the query vector.

[0178] Specifically, the preset 3D latent generative model will randomly generate "initial latent features" (e.g., 1024 dimensions) in the early stages of generation, representing the basic latent representation of the breast model. The electronic device can use these initial latent features as a query vector (Q) to query the association with the enhancement features (i.e., the "basic features generated by the model" must match the "clinically prior enhancement features").

[0179] In step S2044, in each injection layer, the association weights between latent features and enhanced feature vectors are calculated through a cross-attention mechanism.

[0180] Specifically, this is performed at each injection layer of the pre-defined 3D latent generative model (e.g., a total of 6 injection layers, covering the generation stages from coarse to fine). The electronic device again uses a cross-attention mechanism to calculate the association weights between the "model latent features (Q)" and the "enhanced features (K)": Align the latent features of the pre-defined 3D latent generative model with "clinical prior features" (e.g., the nipple curvature dimension generated by the model will align with the nipple curvature dimension in the enhanced features).

[0181] Step S2045: The key vector and value vector are weighted based on the association weight to obtain the weighted feature vector.

[0182] Specifically, the electronic device multiplies the key vector and value vector by the associated weights respectively and then adds them together to calculate the weighted feature vector.

[0183] Step S2046: Update the initial latent features based on the weighted feature vector to obtain the updated latent features.

[0184] Specifically, electronic devices can fuse weighted feature vectors with initial latent features to obtain updated latent features.

[0185] For example, the fusion formula is: Qupdate = Q initial +α×V weighted Q initial For initial latent feature fusion, V weighted α is the weighted feature vector, and α is the fusion coefficient.

[0186] Step S2047: Based on the updated latent features, the preset 3D latent generative model outputs a virtual breast model; Specifically, the preset 3D latent generative model is based on the updated latent features and outputs a virtual breast 3D model (in PLY / STL format, containing 500,000+ vertices, suitable for high detail requirements) through the generation path of "latest features → 3D voxels → triangular mesh".

[0187] Step S2048: Based on the preset loss function, calculate the target loss value between the virtual breast model and the prior data of surface geometry, internal structure, and dynamic mechanics.

[0188] Specifically, the preset loss function includes geometric consistency loss, internal structural constraint loss, and biomechanical loss; step S2048 above may include the following steps: Step c1: Calculate the geometric consistency loss between the virtual breast model and the prior surface geometry data.

[0189] Specifically, the surface triangular mesh corresponding to the virtual breast model is denoted as Mgen, and the surface triangular mesh corresponding to the surface geometric prior data is denoted as Mgt.

[0190] The electronic device uses "Poisson disk sampling" (to avoid point set aggregation) to perform uniform spatial sampling on Mgen and Mgt, ensuring that the number of points in the two point sets is exactly the same (denoted as N), resulting in the generated point set Pgen={p1,p2,...,pN} and the prior point set Pgt={q1,q2,...,qN}.

[0191] Then, the electronic device calculates the mean unidirectional minimum distance d1 from the generated point set to the prior point set: ,in: The Euclidean distance (the straight-line distance between two points), min j∈[1,N] This means that for each generated point pi, we find the point q in the prior point set that is closest to it. j Then calculate the distance.

[0192] The electronic device calculates the mean unidirectional minimum distance d2 from the prior point set to the generated point set. The logic is the opposite of d1: for each prior point q j Find the point p that is closest to it in the set of generated points. i Then calculate the distance.

[0193] The final geometric consistency loss formula is: Among them, key areas (nipple, breast edge): L geo ≤0.3mm (This area is the core of clinical assessment and requires higher precision); Non-critical area (middle of the breast): L geo ≤0.5mm (slight deviation is permissible in fatty areas). L geo The smaller the value, the closer the surface morphology of the generated model is to the prior data (the ideal value is close to 0).

[0194] Step c2: Calculate the internal structure constraint loss between the virtual breast model and the prior data of the internal structure.

[0195] Specifically, the electronic device denotes the internal structure voxels of the virtual breast model as Vgen and the internal structure voxel model corresponding to the prior data of the internal structure as Vgt. The electronic device performs tissue segmentation on Vgen and Vgt respectively, generating binary masks for three types of tissues (the mask is a voxel-level "region label"). Among them, gland mask: Agen (virtual breast model) / Agt (internal structure voxel model), mask value = 1 represents glandular region, mask value = 0 represents non-glandular region; fat mask: Fgen / Fgt, mask value = 1 represents fat region; chest wall mask: Cgen / Cgt, mask value = 1 represents chest wall region.

[0196] For each type of tissue, the Dice coefficient is calculated. The Dice coefficient is a classic metric for medical image segmentation, used to quantify the overlap between two masks (values ​​range from 0 to 1, where 1 = complete overlap and 0 = no overlap). The formula for the Dice coefficient of a single tissue type is as follows: Where: X represents A (gland) / F (fat) / C (chest wall); |Xgen| is the number of voxels with a value of 1 in the generated mask (tissue volume); |Xgen∩Xgt| is the number of voxels that overlap between the generated mask and the prior mask.

[0197] The total internal structural loss calculation (weighted average) is: Lstruct = 0.5 × (1 - Dice(Agen,Agt)) + 0.3 × (1 - Dice(Fgen,Fgt)) + 0.2 × (1 - Dice(Cgen,Cgt)). Here, glandular tissue has a weight of 0.5: glandular tissue distribution is central to breast modeling in clinical practice (e.g., precise matching of glandular tissue location is required for breast augmentation / reconstruction surgery); fat has a weight of 0.3: it is secondary but its distribution must be reasonable; chest wall has a weight of 0.2: as a basal structure, its accuracy requirement is slightly lower.

[0198] Among them, the average Dice coefficient of the three types of tissues is ≥0.85 (the clinical qualification standard for medical image segmentation). The corresponding loss threshold is: Lstruct≤0.1 (derived: if the average Dice=0.85, then 1-0.85=0.15, and after weighting, ≤0.1). The smaller the Lstruct value, the more consistent the internal tissue distribution is with the prior data (the ideal value approaches 0).

[0199] Step c3: Calculate the physiological and mechanical loss between the virtual breast model and the dynamic mechanical prior data.

[0200] Specifically, the electronic device can select marker points (e.g., select 4 clinical core marker points, namely the nipple center point (the area with the most significant deformation), the midpoint of the inframammary fold (the core of ptosis assessment), the lateral edge apex (the key to contour deformation), and the medial edge apex (the key to symmetry assessment)).

[0201] Electronic devices can calculate the virtual displacement corresponding to each marker point based on a virtual breast model: (i=1-4 are the marker number, t is the target body position such as lying flat / leaning forward at 45°, 0 is the standing position reference). The electronic device can extract the displacement ΔPgt,i (clinically measured body displacement data) of the corresponding marker point from the prior dynamic biomechanical data. Then, the biomechanical loss formula is calculated: , where N=4 (number of markers); The average of the displacement errors of each marker point is summed using the square of the L2 norm (the square of the Euclidean distance of the displacement vector, which amplifies the error for easier optimization), thus quantifying the deviation of the overall deformation pattern.

[0202] Step c4 involves fusing the geometric consistency loss, internal structural constraint loss, and biomechanical loss to generate the target loss value.

[0203] Specifically, the electronic device can acquire the weight information corresponding to geometric consistency loss, internal structural constraint loss, and biomechanical loss, respectively. Then, based on the weight information corresponding to the geometric consistency loss, internal structural constraint loss, and biomechanical loss, the geometric consistency loss, internal structural constraint loss, and biomechanical loss are fused to generate the target loss value.

[0204] Step S2049: Based on the target loss value, the parameters in the preset 3D latent generative model are corrected until the target loss value is less than the preset loss function value, thus obtaining the target breast model. The target breast model includes a static breast 3D model and a dynamic breast 3D model.

[0205] Specifically, electronic devices can use the backpropagation algorithm to adjust the network parameters (such as attention weights and decoding layer weights) of a preset 3D potential generative model based on the target loss value. When the target loss value is less than the preset loss function value (such as 0.05), or the number of iterations reaches the upper limit (such as 100 rounds), the optimization stops, and the final target breast model is generated.

[0206] The breast modeling method provided in this application processes the target optical image, determines and corrects camera parameters based on the sternal midline contour, generates an initial depth image using the SfM+MVS algorithm, adjusts it with ultrasound data to obtain a physiologically constrained depth image, and then projects it to generate an initial dense point cloud. The point cloud is optimized using a region growing algorithm, and after Poisson reconstruction and Laplacian smoothing, a breast-specific triangular mesh model is obtained. This model is mapped with physiological depth values ​​and used as prior data for surface geometry, providing precise surface morphology constraints. Based on this mesh model, the image region contour and ultrasound hardness grayscale matrix are aligned using a mutual information registration algorithm, converted into a voxel model, and structure and hardness attribute labels are assigned to the mesh vertices. This is then fused to generate prior data for internal structure, providing internal tissue and mechanical constraints. Based on a standing image, SIFT feature points from multiple frames are matched for registration, key marker points are tracked and back-projected to obtain three-dimensional coordinates, and three-dimensional displacements under multiple positions are calculated as prior data for dynamic mechanics, providing constraints on deformation patterns. Feature vectors from three types of prior data are extracted, weighted, and fused using an autoencoder to generate enhanced feature vectors. These enhanced feature vectors are then split and fused with latent features from a 3D latent generative model via a cross-attention mechanism to update the features and generate a virtual model. The deviation between the virtual model and the prior data is quantified based on a preset loss function, and the model parameters are iteratively corrected until the loss reaches the target value. Finally, a target breast model incorporating both static morphology and dynamic deformation is obtained, achieving complete 3D reconstruction.

[0207] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A breast modeling method, characterized in that, The method includes: Acquire multiple target optical images of the breast to be modeled from different perspectives, as well as target image data and target ultrasound data of the breast to be modeled; Based on the target optical images, target image data and target ultrasound data, the surface geometry prior data, internal structure prior data and dynamic mechanical prior data corresponding to the breast to be modeled are constructed. Based on the surface geometry prior data, the internal structure prior data, and the dynamic mechanical prior data, an enhanced feature vector is generated; Based on the enhanced feature vector, a target breast model corresponding to the breast to be modeled is generated.

2. The method according to claim 1, characterized in that, The construction of prior surface geometry data corresponding to the breast to be modeled, based on the target optical images, target image data, and target ultrasound data, includes: Each of the target optical images is identified to determine the initial target camera parameter set corresponding to each target optical image; Extract the midline contour of the sternum from the target image data; Based on the sternal midline contour, the initial target camera parameter set is corrected to obtain the optimized target camera parameter set; The SfM+MVS algorithm is used to perform dense matching on each of the target optical images, and the initial depth image corresponding to each of the target optical images is output to generate an initial depth image set. Based on the target ultrasound data, the initial depth image set is adjusted to generate physiological constraint depth images corresponding to each initial depth image; Based on the optimized target camera parameter set, the physiological constraint depth images corresponding to each initial depth image are processed to obtain the surface geometric prior data corresponding to the breast to be modeled.

3. The method according to claim 2, characterized in that, The process involves processing the physiological constraint depth images corresponding to each initial depth image based on the optimized target camera parameter set to obtain the surface geometric prior data corresponding to the breast to be modeled, including: Based on the optimized target camera parameter set, each pixel in the physiological constraint depth image is projected onto a unified world coordinate system to generate initial dense point cloud data; The initial dense point cloud data is optimized using a region growing algorithm to generate the target dense point cloud data. The Poisson reconstruction algorithm is used to transform the target dense point cloud data into an initial triangular mesh model; The initial triangular mesh model is smoothed using the Laplace smoothing algorithm to generate a breast-specific triangular mesh model corresponding to the breast to be modeled. Based on the optimized target camera parameter set, the depth value of each pixel in the physiological constraint depth image is mapped to the corresponding vertex of the breast-specific triangular mesh model, so that each vertex of the mesh has both three-dimensional coordinates and physiological depth constraints, thus obtaining the target breast triangular mesh model. The target breast triangular mesh model is determined as the surface geometry prior data corresponding to the breast to be modeled.

4. The method according to claim 3, characterized in that, Based on the target optical images, target image data, and target ultrasound data, prior data of the internal structure corresponding to the breast to be modeled is constructed, including: Using the target breast triangular mesh model as the registration reference, the target region contour of the target image data and the hardness grayscale matrix of the target ultrasound data are used as the registration data; the target region includes at least one internal structure among glands, fat, and chest wall bones. The multimodal registration algorithm based on mutual information is used to register the data to be registered, and the final registration transformation matrix is ​​obtained. Based on the final registration transformation matrix, the target region contour and hardness grayscale matrix are mapped to the spatial position of the target breast triangular mesh model; The registered target region is converted into a first three-dimensional voxel model, and the registered hardness grayscale matrix is ​​converted into a second three-dimensional voxel model. Traverse each vertex of the target breast triangular mesh model and calculate the shortest Euclidean distance in three-dimensional space from each vertex to the first three-dimensional voxel model corresponding to each internal structure in the target region. The internal structure corresponding to the shortest Euclidean distance is used as the structural attribute label of the vertex; For each vertex of the target breast triangular mesh model, the hardness level of the ultrasonic hardness voxel corresponding to the spatial position of the vertex is extracted and used as the hardness attribute label of the vertex. The first three-dimensional voxel model and the second three-dimensional voxel model are fused to generate the internal structure voxel model corresponding to the breast to be modeled; Based on the internal structure voxel model, the prior data of the internal structure corresponding to the breast to be modeled are determined.

5. The method according to claim 4, characterized in that, The prior data on the internal structure also includes the proportion of glandular volume, the distribution of fat layer thickness, and the curvature deviation of the chest wall centerline and the breast base contour line; the determination of the prior data on the internal structure corresponding to the breast to be modeled based on the internal structure voxel model includes: Based on the internal structural voxel model, the three-dimensional volume of the glandular region in the target region and the total volume of the breast corresponding to the breast to be modeled are obtained. Based on the three-dimensional volume of the glandular region in the target area and the total volume of the breast corresponding to the breast to be modeled, the proportion of glandular volume is obtained; Based on the target breast triangular mesh model, the average thickness of the fat layer corresponding to multiple anatomical orientations of the breast to be modeled was measured. Based on the internal structural voxel model, the spatial positional relationship between the chest wall centerline and the breast base contour line is obtained, and the curvature deviation between the chest wall centerline and the breast base contour line is calculated.

6. The method according to claim 1, characterized in that, Each of the target optical images is a temporal optical image covering three body positions: standing, lying flat, and tilting forward at a preset degree. Based on each of the target optical images, the target image data, and the target ultrasound data, dynamic biomechanical prior data corresponding to the breast to be modeled is constructed, including: Using the first frame of the target optical image in a standing position as the reference image frame, a registration algorithm based on feature points is used to extract other SIFT feature points from other frames of optical images besides the reference image frame. For each of the other optical images, the other SIFT feature points corresponding to the other optical images are matched with the reference SIFT feature points corresponding to the reference image frame to determine the registration transformation matrix corresponding to the other optical images; Based on each of the registration transformation matrices, the other optical images of each frame are registered with the reference image frame to obtain the other registered images; In the reference image frame, the pixel coordinates of a predetermined number of reference key marker points are marked; Based on the camera parameters of the reference image frame, the pixel coordinates of each reference key marker point are back-projected to the world coordinate system to obtain the first three-dimensional initial coordinates of each reference key marker point; Using the reference key marker point of the reference image frame as the seed point, the pixel coordinates of the key marker point are tracked frame by frame in each of the other registered images to obtain the pixel coordinates of the other key marker points; For each of the other registered images, based on the camera parameters of the other registered images, the pixel coordinates of each of the other key marker points are back-projected to the world coordinate system to obtain the second three-dimensional initial coordinates of each of the other key marker points; For each of the seed points, calculate the three-dimensional displacement of each seed point in other registered images in frame t; The three-dimensional displacement is determined as the prior dynamic mechanical data corresponding to the breast to be modeled.

7. The method according to claim 1, characterized in that, The process of generating enhanced feature vectors based on the surface geometry prior data, the internal structure prior data, and the dynamic mechanics prior data includes: Feature extraction is performed on the surface geometry prior data to obtain the surface geometry feature vector corresponding to the breast to be modeled; Feature extraction is performed on the prior data of the internal structure to obtain the physiological constraint feature vector corresponding to the breast to be modeled; Feature extraction is performed on the dynamic mechanical prior data to obtain the dynamic mechanical feature vector corresponding to the breast to be modeled; The surface geometric feature vector, the physiological constraint feature vector, and the dynamic mechanical feature vector are weighted and fused to generate a fused feature vector. The fused feature vector is processed based on a preset autoencoder to generate the enhanced feature vector.

8. The method according to claim 1, characterized in that, The step of generating a target breast model corresponding to the breast to be modeled based on the enhanced feature vector includes: Obtain the preset 3D potential generative model corresponding to the breast to be modeled; The enhanced feature vector is split into a key vector and a value vector; The initial latent features generated during the process of generating the preset 3D latent generative model are used as query vectors; In each injection layer, the association weights between the latent features and the enhanced feature vectors are calculated using a cross-attention mechanism; Based on the association weights, the key vector and the value vector are weighted to obtain a weighted feature vector; The initial latent features are updated based on the weighted feature vector to obtain the updated latent features; The preset 3D latent generative model outputs a virtual breast model based on the updated latent features; Based on a preset loss function, the target loss value between the virtual breast model and the surface geometry prior data, the internal structure prior data, and the dynamic mechanical prior data is calculated. The parameters in the preset 3D potential generative model are corrected based on the target loss value until the target loss value is less than the preset loss function value, thereby obtaining the target breast model; the target breast model includes a static breast 3D model and a dynamic breast 3D model.

9. The method according to claim 8, characterized in that, The preset loss function includes geometric consistency loss, internal structural constraint loss, and biomechanical loss; the calculation of the target loss value between the virtual breast model and the surface geometric prior data, the internal structural prior data, and the dynamic biomechanical prior data based on the preset loss function includes: Calculate the geometric consistency loss between the virtual breast model and the surface geometry prior data; Calculate the internal structure constraint loss between the virtual breast model and the prior data of the internal structure; Calculate the physiological and mechanical loss between the virtual breast model and the dynamic mechanical prior data; The geometric consistency loss, the internal structural constraint loss, and the biomechanical loss are fused together to generate the target loss value.

Images