A method and apparatus for three-dimensional reconstruction of breast tissue under unpressurized conditions
By performing deformation compensation and probe pose correction in the breast scanning area, and constructing a deformation field model of breast tissue using Gaussian radial basis functions, the problems of image distortion and spatial deviation under pressure in the three-dimensional reconstruction of breast tissue were solved, and the three-dimensional reconstruction of breast tissue under non-pressure conditions was realized.
Patent Information
- Application Number
- CN202611129781.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-28
- Publication Date
- 2026-11-17
AI Technical Summary
In the three-dimensional reconstruction of breast tissue under non-pressurized conditions, the spatial correspondence between the breast ultrasound images and probe pose under pressurized conditions and the spatial position under non-pressurized conditions is misaligned, which makes it impossible for existing technologies to effectively fuse them.
By scanning at each sampling point from zero pressure to the target pressure with a fixed step size, breast ultrasound images are acquired. A deformation field model of the breast tissue is constructed by linear solution and spatial interpolation using Gaussian radial basis functions, deformation compensation and probe pose correction are performed, and the model is mapped to a three-dimensional voxel mesh reconstruction model.
This method enables the synchronous restoration of breast ultrasound images and probe pose under pressure to the unpressurized state, overcoming image distortion and spatial positioning deviation. It establishes a three-dimensional breast model with consistent spatial relationship between the body surface and internal tissues, which has significant clinical value.
Smart Images

Figure CN123118960A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method and apparatus for three-dimensional reconstruction of breast tissue under unpressurized conditions. Background Technology
[0002] 3D reconstruction of breast tissue in the unpressurized state involves two data sources: first, surface breast point cloud data acquired by a 3D camera before scanning (reflecting the surface morphology in the unpressurized state); and second, in vivo breast tissue structure data reconstructed from ultrasound image sequences during scanning (reflecting the internal morphology in the pressurized state). During scanning, a robotic arm applies contact pressure to the body surface with the ultrasound probe to ensure image quality. Under this pressure, the breast tissue undergoes elastic deformation at the millimeter level, causing deviations in the depth and orientation of lesions in the ultrasound image compared to the unpressurized state. Simultaneously, the scanning position (probe pose) of the robotic arm probe is recorded under pressurized conditions, and this position is offset from the spatial position of the probe in the unpressurized state. Therefore, the two data sources are in different deformation states—unpressurized surface but compressed internally. Directly superimposing and fusing these two data will create spatial contradictions at the boundary between surface and internal tissues, leading to a misalignment in the spatial correspondence between the internal and external breast tissues. Summary of the Invention
[0003] In order to synchronously restore the breast ultrasound images and probe pose under pressure during scanning to the unpressurized state, eliminate ultrasound image distortion and spatial positioning deviation caused by tissue compression and anatomical structure displacement, and fuse the restored in vivo tissue structure with the in vitro point cloud, this invention provides a method and device for three-dimensional reconstruction of breast tissue under unpressurized state.
[0004] In a first aspect, the present invention provides a method for three-dimensional reconstruction of breast tissue under unpressurized conditions, the method comprising:
[0005] The locations of multiple sampling points in the breast scanning area were determined, and the scanning was performed at each sampling point starting from zero pressure and increasing to the target pressure in a fixed step size, so as to obtain the breast ultrasound image at each sampling point under each pressure.
[0006] Using the breast ultrasound image at each sampling point under zero pressure as a reference, the breast ultrasound images at corresponding sampling points under different pressures are registered, and the deformation data of each tissue layer is extracted; the deformation data includes three-dimensional spatial coordinates, pressure and three-dimensional displacement vector, and each tissue layer includes skin layer, fat layer and glandular layer.
[0007] Based on the deformation data of each tissue layer, with three-dimensional spatial coordinates and pressure as inputs and three-dimensional displacement vectors as outputs, the Gaussian radial basis function is linearly solved and spatially interpolated to obtain the tissue deformation field model in the breast.
[0008] Based on the tissue deformation field model in the breast, deformation compensation is performed on each pixel in each frame of breast ultrasound image, and the probe pose corresponding to each frame of breast ultrasound image is corrected to obtain the compensated breast ultrasound image sequence and the corrected probe pose sequence, respectively.
[0009] Based on the corrected probe pose sequence, the compensated ultrasound image sequence is mapped onto a three-dimensional voxel grid to obtain a voxel grid reconstruction model. The voxel grid reconstruction model is then superimposed onto the body surface grid generated from the three-dimensional point cloud data of the breast to obtain a three-dimensional reconstruction model of the breast under unpressurized conditions.
[0010] In one or more optional embodiments of this application, the deformation data of each tissue layer includes the deformation data of N discrete points in each tissue layer;
[0011] Based on deformation data from various tissue layers, using three-dimensional spatial coordinates and pressure as input and three-dimensional displacement vectors as output, the Gaussian radial basis function is linearly solved and spatially interpolated to obtain a tissue deformation field model within the breast, including:
[0012] Using the three-dimensional spatial coordinates and pressure of N discrete points in each tissue layer as input values and the three-dimensional displacement vector as output values, and substituting them into the following Gaussian radial basis function formula, we obtain N deformation equations concerning the weighting coefficients of the Gaussian radial basis function:
[0013] ,
[0014] In the formula, This represents the three-dimensional displacement vector of the j-th discrete point in the i-th tissue layer, i.e., the output value. This represents the three-dimensional spatial coordinates and pressure of the j-th discrete point in the i-th tissue layer, i.e., the input value. This represents the three-dimensional spatial coordinates and pressure of the k-th discrete point in the i-th tissue layer. This represents the k-th weight coefficient of the i-th organizational layer. This represents the shape parameter of the i-th tissue layer;
[0015] By linearly solving the N deformation equations, the weighting coefficients of the Gaussian radial basis function are determined, and spatial interpolation is performed to obtain the tissue deformation field model within the breast.
[0016] In one or more optional embodiments of this application, spatial interpolation is performed using the following formula to obtain a tissue deformation field model within the breast:
[0017] ,
[0018] In the formula, This represents the three-dimensional displacement vector of the point to be determined in the i-th tissue layer. This represents the k-th weight coefficient of the i-th organizational layer determined by linear solution. Let represent the three-dimensional spatial coordinates and pressure of the point to be determined in the i-th tissue layer.
[0019] In one or more optional embodiments of this application, based on the tissue deformation field model within the breast tissue, deformation compensation is performed on each pixel in each frame of breast ultrasound image according to the following formula to obtain a compensated breast ultrasound image sequence:
[0020] ,
[0021] ,
[0022] In the formula, This represents the three-dimensional spatial coordinates of any pixel in a breast ultrasound image under P pressure. This represents the three-dimensional spatial coordinates of the compensated pixel. Represents the three-dimensional spatial coordinates in the tissue deformation field model within the breast tissue. The three-dimensional displacement vector corresponding to the pressure P.
[0023] In one or more optional embodiments of this application, based on the tissue deformation field model within the breast tissue, the probe pose corresponding to each frame of breast ultrasound image is corrected according to the following formula to obtain the corrected probe pose sequence:
[0024] ,
[0025] ,
[0026] In the formula, This indicates the corrected probe pose. This indicates the probe pose corresponding to the breast ultrasound image. This represents the displacement correction vector of the probe in three-dimensional space. The rotation matrix representing the probe pose corresponding to the breast ultrasound image. This represents the displacement component in the z-direction corresponding to the spatial position of the probe in the tissue deformation field model within the breast tissue.
[0027] In one or more optional embodiments of this application, based on the corrected probe pose sequence, the compensated ultrasound image sequence is mapped onto a three-dimensional voxel mesh to obtain a voxel mesh reconstruction model, including:
[0028] Based on the spatial coverage of the corrected probe pose sequence, the spatial resolution and size of the three-dimensional voxel grid are determined, and a three-dimensional voxel grid for storing gray values and tissue category labels is established.
[0029] The pixel gray values and spatial coordinates of each frame of compensated breast ultrasound image are mapped to a three-dimensional voxel grid to obtain the pixel gray values of the corresponding voxel grid. The weighted average of multiple pixel gray values falling into the same voxel grid is calculated to obtain the final gray value of the voxel grid.
[0030] Each frame of the compensated breast ultrasound image is segmented to obtain the tissue category label of each pixel and mapped to a three-dimensional voxel grid. The tissue category label that falls into the same voxel grid the most times is determined as the final tissue category of the voxel grid.
[0031] Based on the final grayscale value and the final tissue category of the voxel mesh, a voxel mesh reconstruction model is obtained.
[0032] Secondly, the present invention provides a three-dimensional reconstruction device for breast tissue in an unpressurized state, comprising:
[0033] The breast ultrasound image acquisition module is used to determine the location of multiple sampling points in the breast scanning area, and scan at each sampling point starting from zero pressure and increasing to the target pressure in a fixed step size to obtain a breast ultrasound image at each sampling point under each pressure.
[0034] The deformation data extraction module is used to register breast ultrasound images at corresponding sampling points under different pressures, using the breast ultrasound image at each sampling point under zero pressure as a reference, and extract deformation data of each tissue layer; the deformation data includes three-dimensional spatial coordinates, pressure and three-dimensional displacement vector, and each tissue layer includes skin layer, fat layer and glandular layer.
[0035] The deformation field model construction module is used to obtain the tissue deformation field model in the breast based on the deformation data of each tissue layer, with three-dimensional spatial coordinates and pressure as inputs and three-dimensional displacement vectors as outputs, linearly solving the Gaussian radial basis function and performing spatial interpolation.
[0036] The deformation compensation module is used to perform deformation compensation on each pixel in each frame of breast ultrasound image based on the deformation field model of the breast tissue, and to correct the probe pose corresponding to each frame of breast ultrasound image, so as to obtain the compensated breast ultrasound image sequence and the corrected probe pose sequence respectively.
[0037] The breast 3D reconstruction module is used to map the compensated ultrasound image sequence onto a 3D voxel grid based on the corrected probe pose sequence to obtain a voxel grid reconstruction model, and then superimpose the voxel grid reconstruction model onto the body surface grid generated from the breast 3D point cloud data to obtain a breast 3D reconstruction model under unpressurized conditions.
[0038] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the three-dimensional reconstruction method of breast tissue in an unpressurized state as described in the first aspect above.
[0039] Fourthly, embodiments of the present invention provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the three-dimensional reconstruction method of breast tissue in an unpressurized state as described in the first aspect above.
[0040] Fifthly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the three-dimensional reconstruction method of breast tissue in an unpressurized state as described in the first aspect above.
[0041] The beneficial effects of the above-described technical solutions provided in the embodiments of the present invention include at least the following:
[0042] This invention provides a method for three-dimensional reconstruction of breast tissue under unpressurized conditions. Starting from zero pressure at each sampling point, the method scans at a fixed step size, increasing to the target pressure, to obtain breast ultrasound images at each sampling point under each pressure. Using the breast ultrasound images at each sampling point under zero pressure as a reference, the method registers the breast ultrasound images at corresponding sampling points under different pressures, extracting deformation data (three-dimensional spatial coordinates, pressure, and three-dimensional displacement vector) for each tissue layer. Using three-dimensional spatial coordinates and pressure as input and three-dimensional displacement vector as output, the method linearly solves the Gaussian radial basis function and performs spatial interpolation to obtain a tissue deformation field model within the breast. Based on this model, pixel deformation compensation is performed on each frame of breast ultrasound images, and the probe pose corresponding to each frame of breast ultrasound images is corrected, resulting in a compensated breast ultrasound image sequence and a corrected probe pose sequence. Based on the corrected probe pose sequence, the compensated ultrasound image sequence is mapped to a three-dimensional voxel mesh to obtain a voxel mesh reconstruction model. This model is then superimposed and fused with a surface mesh generated from breast three-dimensional point cloud data to obtain a three-dimensional breast reconstruction model under unpressurized conditions. This invention provides a method for obtaining a deformation field model of breast tissue by linearly solving the radial basis function and using linear interpolation. It compensates for the deformation of each frame of ultrasound image under pressure and simultaneously corrects the probe pose, ensuring that the ultrasound image and spatial positioning are consistently restored to the unpressurized state. This overcomes the problems of image distortion and spatial positioning deviation caused by only compensating for the image or only compensating for the pose. A voxel mesh reconstruction model representing breast tissue under unpressurized conditions is established and superimposed and fused with a surface mesh to construct a three-dimensional breast model with consistent spatial relationships between the surface and internal tissues under unpressurized conditions. This model reflects the true three-dimensional morphology of the breast and has significant clinical value for surgical planning, lesion localization, and follow-up comparison. Attached Figure Description
[0043] The accompanying drawings are provided to further illustrate the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application, and do not constitute a limitation on the technical solutions of this application. In the accompanying drawings:
[0044] Figure 1 This is a schematic diagram of the three-dimensional reconstruction method for breast tissue under unpressurized conditions provided in the embodiments of this application;
[0045] Figure 2 A flowchart illustrating the deformation compensation and probe pose correction of a single-frame breast ultrasound image provided in this application embodiment;
[0046] Figure 3 This is a schematic diagram of a three-dimensional reconstruction device for breast tissue in an unpressurized state, as provided in an embodiment of this application. Detailed Implementation
[0047] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. However, this description is exemplary and not restrictive, and those skilled in the art can implement this application in various forms without being limited to the embodiments set forth herein. These embodiments are provided to enable a more thorough understanding of this disclosure and to fully convey the scope of this disclosure to those skilled in the art.
[0048] The embodiments, features, and elements disclosed in this application can also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment can also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application can be implemented individually or in any suitable combination.
[0049] To facilitate understanding of the technical solutions of the embodiments of this application, some terms or concepts involved in the embodiments of this application will be briefly described first.
[0050] 1. Three-dimensional voxel mesh: It consists of several voxel units evenly distributed along the x, y, and z axes. Each voxel unit has corresponding three-dimensional physical space coordinates and attribute values (such as gray value and tissue type) to characterize the physical properties of breast tissue at the corresponding spatial location.
[0051] 2. Tissue Category Label: Used to uniquely identify the type of breast anatomical structure to which the three-dimensional voxel mesh belongs, wherein the anatomical structure type includes at least skin, fat, and glands.
[0052] 3. Majority voting mechanism: This refers to a statistical decision-making method that accumulates the frequency of the corresponding tissue category label in all ultrasound image frames for any voxel grid and determines the label with the highest frequency as the final tissue category for that voxel grid.
[0053] 4. Registration: A method that uses image processing techniques such as optical flow or cross-correlation algorithms to spatially align ultrasound images under different pressures (floating images) with a reference ultrasound image under zero pressure (reference image).
[0054] 5.12 clock directions: With the nipple as the origin, the unilateral breast is divided into 12 radial sectors according to the orientation of a clock face, with each sector at 30°.
[0055] 6. Outer edge of the areola: the junction between the areola and normal skin.
[0056] 7. Axillary accessory breast area: In a unified clock coordinate system, the area from the outer side of the mammary gland towards the armpit corresponds to the 9 o'clock position and the terminal area of the mammary gland extending towards the armpit.
[0057] In their research on 3D reconstruction of breast tissue, the inventors discovered that breast ultrasound images under pressure exhibit distortion and spatial positioning deviations. Existing technologies lack a method for synchronously restoring the pressure-generated breast ultrasound images and probe pose to the unpressured state frame by frame. This makes it impossible to correct in vivo compression deformation to a deformation state consistent with the in vitro point cloud, thus hindering the effective fusion of in vivo and in vitro breast tissue in 3D reconstruction. To address this issue, the inventors attempted to construct an in vivo breast tissue deformation field model by solving Gaussian radial basis functions, thereby improving the accuracy of 3D reconstruction of pressure-generated breast ultrasound images. For example, deformation compensation and probe pose correction are performed. Research has found that based on the deformation data of each tissue layer obtained from acquisition and image registration, a deformation field model of breast tissue can be obtained by solving the Gaussian radial basis function and performing spatial interpolation. Based on this model, deformation compensation is performed on each frame of breast ultrasound image, while simultaneously correcting the probe pose and mapping it to a three-dimensional voxel mesh, resulting in a voxel mesh reconstruction model. This voxel mesh reconstruction model is then superimposed onto the body surface mesh generated from breast three-dimensional point cloud data to obtain a three-dimensional reconstruction model of breast tissue under unpressurized conditions. Based on this, the present invention provides a method and apparatus for three-dimensional reconstruction of breast tissue under unpressurized conditions.
[0058] Example 1
[0059] Embodiment 1 of the present invention provides a method for three-dimensional reconstruction of breast tissue under unpressurized conditions, referring to... Figure 1 As shown, the method includes the following steps:
[0060] S101: Determine the location of multiple sampling points in the breast scanning area, and scan at each sampling point starting from zero pressure and increasing to the target pressure with a fixed step size to obtain a breast ultrasound image at each sampling point under each pressure.
[0061] S102: Using the breast ultrasound image at each sampling point under zero pressure as a reference, register the breast ultrasound images at the corresponding sampling points under different pressures and extract the deformation data of each tissue layer; the deformation data includes three-dimensional spatial coordinates, pressure and three-dimensional displacement vector, and each tissue layer includes skin layer, fat layer and glandular layer.
[0062] S103: Based on the deformation data of each tissue layer, with three-dimensional spatial coordinates and pressure as inputs and three-dimensional displacement vectors as outputs, the Gaussian radial basis function is linearly solved and spatially interpolated to obtain the tissue deformation field model in the breast.
[0063] S104: Based on the tissue deformation field model in the breast, deformation compensation is performed on each pixel in each frame of breast ultrasound image, and the probe pose corresponding to each frame of breast ultrasound image is corrected to obtain the compensated breast ultrasound image sequence and the corrected probe pose sequence, respectively.
[0064] S105: Based on the corrected probe pose sequence, the compensated ultrasound image sequence is mapped to a three-dimensional voxel grid to obtain a voxel grid reconstruction model. The voxel grid reconstruction model is then superimposed onto the body surface grid generated from the three-dimensional point cloud data of the breast to obtain a three-dimensional reconstruction model of the breast under unpressurized conditions.
[0065] The three-dimensional reconstruction method for breast tissue under unpressurized conditions provided by this invention obtains a deformation field model of breast tissue by linearly solving the radial basis function and linear interpolation. Deformation compensation is performed on each frame of ultrasound image under pressurized conditions, and the probe pose is corrected simultaneously, so that the ultrasound image and spatial positioning are consistently restored to the unpressurized state. This overcomes the problems of image distortion and spatial positioning deviation caused by only compensating for the image or only compensating for the pose. A voxel mesh reconstruction model representing the breast tissue under unpressurized conditions is established and superimposed and fused with the body surface mesh to construct a three-dimensional breast model with consistent spatial relationship between the body surface and the internal tissue under unpressurized conditions. It can reflect the true three-dimensional morphology of the breast and has important clinical value for surgical planning, lesion localization, and follow-up comparison.
[0066] In this embodiment, in order to fully characterize the breast region and follow human anatomical features, the multiple sampling points selected in step S101 include at least key areas such as the nipple, areola, and armpit. For example, the locations of multiple sampling points can be determined according to the following selection criteria:
[0067] For the left or right breast scanning area, with the nipple center as the origin, the locations of multiple sampling points are determined according to the following rules:
[0068] The center of the nipple was selected as one sampling point;
[0069] Select 1 to 2 sampling points along each of the 12 clock directions;
[0070] Select 2-3 sampling points on the outer edge of the areola;
[0071] Select 1-2 sampling points in the axillary accessory breast area.
[0072] Using the nipple center as the anatomical reference, with the nipple as the center, the 12 o'clock position consistently points towards the head (cephalic side), the 6 o'clock position consistently points towards the foot (foot side), the 3 o'clock position consistently points towards the midline of the body (medial / sternal direction), and the 9 o'clock position consistently points towards the lateral side of the body (lateral / axillary direction). One to two sampling points are selected in each clock direction: one in the areola and / or one in the peripheral region. The peripheral region refers to the area located lateral to the areola, closer to the edge of the breast tissue. Typically, the glandular tissue density in this area is lower than in the areola and closer to the chest wall or axilla. The 12 clock directions cover the entire anatomical orientation of the breast, and the clock-face positioning method aligns with clinicians' habits of locating breast lesions. The deformation field model established using this method can directly correspond to the clinically diagnosed area. The outer edge of the areola shows significant changes in tissue thickness and a large elastic modulus gradient, making it a key transition zone for deformation. The tissue structure of the axillary accessory breast area differs from that of the main breast, and this area has a high proportion of adipose tissue and a large deformation response. Sampling separately can improve the accuracy of deformation field construction in the lateral region.
[0073] In one embodiment, the number of sampling points for a unilateral breast scan area is preferably 18 to 22. Compared with uniform grid sampling, these anatomically oriented sampling points ensure coverage and diversity of sampling locations, guarantee deformation accuracy in key clinically relevant areas (such as the axillary accessory breast area), and effectively reduce the sampling workload.
[0074] In this embodiment, step S101 requires maintaining a fixed probe spatial orientation and applying pressure incrementally within a range tolerable to the human body for scanning. A fixed step size, such as 1N or 2N, is selected based on accuracy requirements to ensure deformation sampling density. The target pressure can be 10N, and the pressure scanning range covers all pressure intervals potentially used in clinical scanning. After each pressure change, wait 2-3 seconds for tissue deformation to stabilize before acquiring images to eliminate the influence of viscoelastic creep. Incremental pressure scanning yields breast ultrasound images at each sampling point under each pressure.
[0075] In this embodiment, step S102 involves dividing breast tissue into skin, fat, and glandular layers to extract deformation data, thereby constructing a layered deformation field and improving the accuracy of deformation characterization for different tissues. Step S102 can employ a deep learning-based medical image segmentation model (such as U-Net or DeepLab) to segment each frame of the breast ultrasound image, identifying the layer boundaries of each tissue in the breast ultrasound image. In three-dimensional space, this corresponds to the lower boundary of each tissue layer, including the lower boundary of the skin layer (interface between the skin and fat layers), the lower boundary of the fat layer (interface between the fat and glandular layers), and the lower boundary of the glandular layer. This determines the spatial distribution range of each tissue layer. Coordinate transformation converts the pixels of each tissue layer in the breast ultrasound image into three-dimensional spatial coordinates (discrete points) of each tissue layer. Using the three-dimensional spatial position of the breast tissue under zero pressure as a reference, image registration determines the three-dimensional spatial coordinates of the same discrete point in each tissue layer under different pressures. The three-dimensional displacement vector (deformation under pressure) of the discrete point in each tissue layer at each sampling point under each pressure is calculated using the following formula:
[0076] ,
[0077] ,
[0078] ,
[0079] In the formula, indices i=1,2,3 represent the skin layer, fat layer, and glandular layer, respectively. , , ), ( , , ) represent the three-dimensional spatial coordinates of any discrete point in the i-th tissue layer at any sampling point under zero pressure and P pressure, respectively. , , ) represents the three-dimensional displacement vector of the corresponding discrete point in the i-th tissue layer at the corresponding sampling point under pressure P.
[0080] Using three-dimensional spatial coordinates and pressure as inputs and three-dimensional displacement vectors as outputs, deformation data for each tissue layer is constructed.
[0081] In this embodiment, the deformation data of each tissue layer in step S103 includes the deformation data of N discrete points in each tissue layer, and the tissue deformation field model in the breast can be obtained by following the steps below:
[0082] S1031: Taking the three-dimensional spatial coordinates and pressure of N discrete points in each tissue layer as input values and the three-dimensional displacement vector as output values, and substituting them into the following Gaussian radial basis function formula, we obtain N deformation equations regarding the weighting coefficients of the Gaussian radial basis function:
[0083] ,
[0084] In the formula, This represents the three-dimensional displacement vector of the j-th discrete point in the i-th tissue layer, i.e., the output value. This represents the three-dimensional spatial coordinates and pressure of the j-th discrete point in the i-th tissue layer, i.e., the input value. This represents the three-dimensional spatial coordinates and pressure of the k-th discrete point in the i-th tissue layer. This represents the k-th weight coefficient of the i-th organizational layer. This represents the shape parameter of the i-th tissue layer.
[0085] S1032: By linearly solving the N deformation equations, the weighting coefficients of the Gaussian radial basis function are determined, and spatial interpolation is performed to obtain the tissue deformation field model within the breast.
[0086] Because the deformation in the three-dimensional direction is anisotropic, in step S1031 above, the displacement in the x, y, and z directions in each tissue layer is... The weighting coefficients are solved linearly. In the linear solution, the three-dimensional spatial coordinates and pressure are the same as the input values, and the corresponding output values are respectively... According to the Gaussian radial basis function formula, the weighting coefficients in the three-dimensional directions can be solved respectively, thereby reflecting the different mechanical responses of breast tissue in the direction of force application and the side.
[0087] In step S1032 above, when linearly solving the N deformation equations, It was determined through cross-validation, and the specific determination process is as follows:
[0088] After acquiring breast ultrasound images of an individual in step S101, step S102 is performed to extract deformation data for each tissue layer (three-dimensional spatial coordinates and pressure as input values, and three-dimensional displacement vector as output values). The deformation data of all discrete points in each tissue layer are pre-divided into N modeling points and n verification points. The deformation data of the N modeling points is used to substitute into the Gaussian radial basis function to obtain N deformation equations regarding the weighting coefficients. The deformation data of the n verification points is used for verification and determination. .set up The candidate value range is [0.1, 10]. This continuous value interval is discretized into several values (for example, with a step size of 0.1). When taking a fixed value, the weighting coefficients are obtained by linearly solving step S103 based on the deformation data of N modeling points. And the corresponding deformation field model, traversing All values, when processed in step S103, yield the corresponding deformation field model. Substituting the input values from the deformation data of each verification point into any deformation field model, the theoretical three-dimensional displacement vector of each verification point is obtained. The error between the theoretical three-dimensional displacement vector and the true three-dimensional deformation vector (the output value from the deformation data of the verification point) is calculated and summed. The shape parameter corresponding to the smallest error is... and weighting coefficients That is the optimal solution.
[0089] After determining the weighting coefficients of the Gaussian radial basis function through linear solution, spatial interpolation can be performed using the following formula to obtain the tissue deformation field model within the breast:
[0090] ,
[0091] In the formula, This represents the three-dimensional displacement vector of the point to be determined in the i-th tissue layer. This represents the k-th weight coefficient of the i-th organizational layer determined by linear solution. This represents the three-dimensional spatial coordinates and pressure of the point to be determined in the i-th tissue layer. After spatial interpolation, the deformation (three-dimensional displacement vector) of all points to be determined (unknown points) is obtained, which, together with the deformation of known discrete points, constitutes the tissue deformation field model within the breast.
[0092] In this embodiment, in step S104, the deformation field model of breast tissue is used to synchronously compensate and correct the deformation of each frame of breast ultrasound image and the corresponding probe pose, so as to restore each frame of breast ultrasound image from the pressurized state to the unpressurized state.
[0093] Using a tissue deformation field model within the breast tissue, deformation compensation was performed on each pixel in each frame of breast ultrasound images in the following manner:
[0094] Based on the aforementioned tissue deformation field model within the breast, deformation compensation is performed on each pixel in each frame of breast ultrasound images according to the following formula to obtain a compensated breast ultrasound image sequence:
[0095] ,
[0096] ,
[0097] In the formula, This represents the three-dimensional spatial coordinates (in the pressurized state) of any pixel in a breast ultrasound image under pressure P. This represents the three-dimensional spatial coordinates of the compensated pixel (in the uncompressed state). Represents the three-dimensional spatial coordinates in the tissue deformation field model within the breast tissue. The three-dimensional displacement vector corresponding to the pressure P.
[0098] Reference Figure 2 As shown, for deformation compensation of any frame of breast ultrasound image, based on the pressure P corresponding to the breast ultrasound image, the tissue deformation field model within the breast is queried to obtain the three-dimensional displacement vector (deformation) of each tissue layer under pressure P. For each pixel in the image, its tissue layer (skin layer, fat layer, or glandular layer) is determined according to the aforementioned segmentation results. The deformation field corresponding to that tissue layer is called, and the three-dimensional spatial coordinates corresponding to the pixel are reversed from the pressurized state to the unpressurized state using the aforementioned deformation compensation formula. In addition, for cases where lesions exist in the glandular layer, the lesions can also be segmented in the glandular layer during image segmentation, and the three-dimensional spatial coordinates corresponding to the pixels belonging to the lesions can also be reversed from the pressurized state to the unpressurized state.
[0099] When performing pixel-by-pixel compensation on each frame of breast ultrasound image, the probe pose needs to be corrected synchronously. Since tissue compression mainly occurs in the direction of probe force application (z-direction), a rotation matrix is constructed using the RPY orientation of the probe pose to be corrected. The z-direction displacement component output by the tissue deformation model within the breast is then used for correction to obtain the corrected probe position. The specific correction method is as follows:
[0100] Based on the aforementioned tissue deformation field model within the breast, the probe pose corresponding to each frame of breast ultrasound image is corrected according to the following formula to obtain the corrected probe pose sequence:
[0101] ,
[0102] ,
[0103] In the formula, This indicates the corrected probe pose. This indicates the probe pose corresponding to the breast ultrasound image. This represents the displacement correction vector of the probe in three-dimensional space. The rotation matrix representing the probe pose corresponding to the breast ultrasound image. This represents the displacement component in the z-direction corresponding to the spatial position of the probe in the tissue deformation field model within the breast tissue.
[0104] In this embodiment, after image deformation compensation and probe pose correction, a voxel mesh reconstruction model covering the tissue within the breast is constructed in step S105. Specifically, the voxel mesh reconstruction model can be obtained through the following steps:
[0105] S1051: Based on the spatial coverage of the corrected probe pose sequence, determine the spatial resolution and size of the three-dimensional voxel grid, and establish a three-dimensional voxel grid for storing gray values and tissue category labels.
[0106] S1052: Map the pixel grayscale values and spatial coordinates of each frame of compensated breast ultrasound image to a three-dimensional voxel grid to obtain the pixel grayscale values of the corresponding voxel grid. Perform a weighted average calculation on the multiple pixel grayscale values falling into the same voxel grid to obtain the final grayscale value of the voxel grid.
[0107] S1053: Segment each frame of the compensated breast ultrasound image, obtain the tissue category label of each pixel and map it to a three-dimensional voxel grid, and determine the tissue category label that falls into the same voxel grid the final tissue category of the voxel grid.
[0108] S1054: Based on the final grayscale value of the voxel grid and the final tissue category of the voxel grid, a voxel grid reconstruction model is obtained.
[0109] Step S1051 above is the process of initializing the voxel grid. It iterates through the corrected probe pose sequence, extracting the maximum and minimum coordinate values of all probe poses to determine the spatial range of the three-dimensional voxel grid. The resolution of the three-dimensional voxel grid can be set according to accuracy requirements (e.g., voxel side length 0.2–0.5 mm) to finely reconstruct the spatial details of the anatomical structure. Based on the spatial range and voxel side length, the total number of voxel grids required in the x, y, and z dimensions (i.e., the grid size) is calculated. Each voxel grid is initialized with two attributes: one is the pixel grayscale value of the compensated breast ultrasound image (initial value set to 0 or empty), and the other is the tissue category label (initial value set to "none" or "background"), thus completing the establishment of the three-dimensional voxel grid for storing grayscale values and tissue category labels.
[0110] Because ultrasound scans overlap, a single location in three-dimensional space (voxel grid) will be recorded by multiple frames of breast ultrasound images. Therefore, each voxel grid corresponds to multiple pixel gray values and multiple tissue category labels. In step S1052 above, a weighted average is calculated for the multiple pixel gray values of the same voxel grid to obtain the final gray value of the voxel grid. The weights in the weighting calculation can be dynamically adjusted based on factors such as image quality score or pixel-to-probe distance, assigning higher weights to high-quality images. In step S1053 above, the tissue category labels obtained from segmentation include at least skin, fat, and glands. When lesions are present in the glandular layer, the segmented tissue category labels include at least skin, fat, glands, and lesions. A majority voting mechanism is used to determine the final tissue category (skin / fat / gland / lesion) for each voxel grid. For example, if a voxel grid is covered by 40 pixels, and the category labels for these 40 pixels are statistically analyzed (3 pixels are skin, 33 pixels are glands, and 4 pixels are fat), then after voting, glands become the most frequent tissue category label and are used as the final tissue category for the voxel grid. Under conditions of overlapping coverage during scanning, the majority voting mechanism can effectively improve the robustness of tissue category labeling. In step S1054 above, the final grayscale value and the final tissue category are mapped to each voxel grid to obtain the voxel grid reconstruction model.
[0111] Reference Figure 2 As shown, the strategy of segmenting and then compensating breast ultrasound images in this embodiment can ensure that different tissue layers (skin / fat / gland) receive compensation that matches their mechanical properties, avoid compensation deviation caused by uniform deformation compensation across layers, and improve the accuracy of deformation compensation.
[0112] In one specific embodiment, the lesion and gland can be reconstructed in three dimensions separately (the lesion is located in the gland and needs to be segmented during the aforementioned image segmentation). At the boundary between the lesion and the gland, an isosurface extraction algorithm (such as Marching Cubes) is used to reconstruct a triangular surface mesh for three-dimensional visualization and spatial measurement annotation, supporting quantitative measurement functions such as accurate volume calculation, distance measurement, and multi-angle cutting analysis.
[0113] In this embodiment, the mapping relationship between the sampling points and the three-dimensional point cloud of the breast is utilized in step S101 to align the deformed internal tissue (voxel mesh reconstruction model) to the body surface coordinate system, achieving spatial fusion of the voxel mesh reconstruction model and the body surface mesh model. In step S105, the three-dimensional point cloud data of the breast can be converted into a continuous triangular mesh model (body surface mesh) using the Poisson surface reconstruction algorithm. The minimum bounding cube of the body surface mesh is calculated, and the portion of the voxel mesh reconstruction model that exceeds the cube's range is clipped to ensure that the spatial range of the internal tissue matches the body surface. Finally, the clipped voxel mesh reconstruction model is superimposed onto the body surface mesh to obtain the three-dimensional reconstruction model of the breast in the unpressurized state, achieving spatial alignment and precise fusion of the internal tissue and the external body surface.
[0114] The three-dimensional reconstruction method of breast tissue under unpressurized conditions provided by this invention uses the collected deformation data as a basis and employs Gaussian radial basis functions to perform spatial interpolation to obtain a deformation field model of breast tissue. The ultrasound image of breast tissue under pressurized conditions and the probe pose are synchronously restored to the unpressurized state and fused with the in vitro three-dimensional point cloud to obtain a complete three-dimensional reconstruction model of breast tissue with consistent spatial relationship between the body surface and internal tissue, intuitively displaying the true three-dimensional morphology of the breast.
[0115] Example 2
[0116] Based on the same inventive concept, this invention also provides a three-dimensional reconstruction device for breast tissue in an unpressurized state, referring to... Figure 3 As shown, it includes:
[0117] The breast ultrasound image acquisition module 101 is used to determine the location of multiple sampling points in the breast scanning area, and to scan at each sampling point starting from zero pressure and increasing to the target pressure in a fixed step size, so as to obtain the breast ultrasound image at each sampling point under each pressure.
[0118] The deformation data extraction module 102 is used to register breast ultrasound images at corresponding sampling points under different pressures, using the breast ultrasound image at each sampling point under zero pressure as a reference, and extract deformation data of each tissue layer; the deformation data includes three-dimensional spatial coordinates, pressure and three-dimensional displacement vector, and each tissue layer includes skin layer, fat layer and glandular layer.
[0119] The deformation field model construction module 103 is used to obtain the tissue deformation field model in the breast based on the deformation data of each tissue layer, with three-dimensional spatial coordinates and pressure as input and three-dimensional displacement vector as output, by linearly solving the Gaussian radial basis function and performing spatial interpolation.
[0120] The deformation compensation module 104 is used to perform deformation compensation on each pixel in each frame of breast ultrasound image based on the deformation field model of breast tissue, and to correct the probe pose corresponding to each frame of breast ultrasound image, so as to obtain the compensated breast ultrasound image sequence and the corrected probe pose sequence respectively.
[0121] The breast three-dimensional reconstruction module 105 is used to map the compensated ultrasound image sequence to a three-dimensional voxel grid based on the corrected probe pose sequence to obtain a voxel grid reconstruction model, and to superimpose the voxel grid reconstruction model onto the body surface grid generated by the breast three-dimensional point cloud data to obtain a breast three-dimensional reconstruction model under unpressurized conditions.
[0122] Example 3
[0123] Based on the same inventive concept, embodiments of the present invention provide a computer-readable storage medium storing a computer program / instruction thereon, which, when executed by a processor, implements the three-dimensional reconstruction method of breast tissue in an unpressurized state as described in Embodiment 1 above.
[0124] Example 4
[0125] Based on the same inventive concept, this invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the three-dimensional reconstruction method of breast tissue in an unpressurized state as described in Embodiment 1 above.
[0126] Example 5
[0127] Based on the same inventive concept, this embodiment of the invention provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the three-dimensional reconstruction method of breast tissue in an unpressurized state as described in Embodiment 1 above.
[0128] In this embodiment, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that the method or process does not depend on the specific order of steps described herein. Other sequences of steps are also possible, as understood by those skilled in the art. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation on the claims. Furthermore, the claims for the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of the invention.
[0129] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0130] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of the invention. Those skilled in the art will understand that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0131] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0132] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
Claims
1. A method for three-dimensional reconstruction of breast tissue under unpressurized conditions, characterized in that, The method includes: The locations of multiple sampling points in the breast scanning area were determined, and the scanning was performed at each sampling point starting from zero pressure and increasing to the target pressure in a fixed step size, so as to obtain the breast ultrasound image at each sampling point under each pressure. Using the breast ultrasound image at each sampling point under zero pressure as a reference, the breast ultrasound images at corresponding sampling points under different pressures are registered, and the deformation data of each tissue layer is extracted; the deformation data includes three-dimensional spatial coordinates, pressure and three-dimensional displacement vector, and each tissue layer includes skin layer, fat layer and glandular layer. Based on the deformation data of each tissue layer, with three-dimensional spatial coordinates and pressure as inputs and three-dimensional displacement vectors as outputs, the Gaussian radial basis function is linearly solved and spatially interpolated to obtain the tissue deformation field model in the breast. Based on the tissue deformation field model in the breast, deformation compensation is performed on each pixel in each frame of breast ultrasound image, and the probe pose corresponding to each frame of breast ultrasound image is corrected to obtain the compensated breast ultrasound image sequence and the corrected probe pose sequence, respectively. Based on the corrected probe pose sequence, the compensated ultrasound image sequence is mapped onto a three-dimensional voxel grid to obtain a voxel grid reconstruction model. The voxel grid reconstruction model is then superimposed onto the body surface grid generated from the three-dimensional point cloud data of the breast to obtain a three-dimensional reconstruction model of the breast under unpressurized conditions.
2. The method for three-dimensional reconstruction of breast tissue under unpressurized conditions as described in claim 1, characterized in that, The deformation data for each tissue layer includes the deformation data of N discrete points in each tissue layer; Based on deformation data from various tissue layers, using three-dimensional spatial coordinates and pressure as input and three-dimensional displacement vectors as output, the Gaussian radial basis function is linearly solved and spatially interpolated to obtain a tissue deformation field model within the breast, including: Using the three-dimensional spatial coordinates and pressure of N discrete points in each tissue layer as input values and the three-dimensional displacement vector as output values, and substituting them into the following Gaussian radial basis function formula, we obtain N deformation equations concerning the weighting coefficients of the Gaussian radial basis function: , In the formula, This represents the three-dimensional displacement vector of the j-th discrete point in the i-th tissue layer, i.e., the output value. This represents the three-dimensional spatial coordinates and pressure of the j-th discrete point in the i-th tissue layer, i.e., the input value. This represents the three-dimensional spatial coordinates and pressure of the k-th discrete point in the i-th tissue layer. This represents the k-th weight coefficient of the i-th organizational layer. This represents the shape parameter of the i-th tissue layer; By linearly solving the N deformation equations, the weighting coefficients of the Gaussian radial basis function are determined, and spatial interpolation is performed to obtain the tissue deformation field model within the breast.
3. The method for three-dimensional reconstruction of breast tissue under unpressurized conditions as described in claim 2, characterized in that, The deformation field model of breast tissue is obtained by spatial interpolation using the following formula: , In the formula, This represents the three-dimensional displacement vector of the point to be determined in the i-th tissue layer. This represents the k-th weight coefficient of the i-th organizational layer determined by linear solution. Let represent the three-dimensional spatial coordinates and pressure of the point to be determined in the i-th tissue layer.
4. The method for three-dimensional reconstruction of breast tissue under unpressurized conditions as described in claim 1 or 3, characterized in that, Based on the aforementioned tissue deformation field model within the breast, deformation compensation is performed on each pixel in each frame of breast ultrasound images according to the following formula to obtain a compensated breast ultrasound image sequence: , , In the formula, This represents the three-dimensional spatial coordinates of any pixel in a breast ultrasound image under P pressure. This represents the three-dimensional spatial coordinates of the compensated pixel. Represents the three-dimensional spatial coordinates in the tissue deformation field model within the breast tissue. The three-dimensional displacement vector corresponding to the pressure P.
5. The method for three-dimensional reconstruction of breast tissue under unpressurized conditions as described in claim 4, characterized in that, Based on the aforementioned tissue deformation field model within the breast, the probe pose corresponding to each frame of breast ultrasound image is corrected according to the following formula to obtain the corrected probe pose sequence: , , In the formula, This indicates the corrected probe pose. This indicates the probe pose corresponding to the breast ultrasound image. This represents the displacement correction vector of the probe in three-dimensional space. The rotation matrix representing the probe pose corresponding to the breast ultrasound image. This represents the displacement component in the z-direction corresponding to the spatial position of the probe in the tissue deformation field model within the breast tissue.
6. The method for three-dimensional reconstruction of breast tissue under unpressurized conditions as described in claim 1, characterized in that, Based on the corrected probe pose sequence, the compensated ultrasound image sequence is mapped onto a three-dimensional voxel mesh to obtain a voxel mesh reconstruction model, including: Based on the spatial coverage of the corrected probe pose sequence, the spatial resolution and size of the three-dimensional voxel grid are determined, and a three-dimensional voxel grid for storing gray values and tissue category labels is established. The pixel gray values and spatial coordinates of each frame of compensated breast ultrasound image are mapped to a three-dimensional voxel grid to obtain the pixel gray values of the corresponding voxel grid. The weighted average of multiple pixel gray values falling into the same voxel grid is calculated to obtain the final gray value of the voxel grid. Each frame of the compensated breast ultrasound image is segmented to obtain the tissue category label of each pixel and mapped to a three-dimensional voxel grid. The tissue type label that falls into the same voxel grid the most times is determined as the final tissue category of the voxel grid. Based on the final grayscale value and the final tissue category of the voxel mesh, a voxel mesh reconstruction model is obtained.
7. A three-dimensional reconstruction device for breast tissue under unpressurized conditions, characterized in that, include: The multi-pressure breast ultrasound imaging module is used to determine the location of multiple sampling points in the breast scanning area, and scans at each sampling point starting from zero pressure and increasing to the target pressure in a fixed step size to obtain breast ultrasound images at each sampling point under each pressure. The deformation data module for each tissue layer is used to register breast ultrasound images at corresponding sampling points under different pressures, using the breast ultrasound image at each sampling point under zero pressure as a reference, and extract the deformation data of each tissue layer; the deformation data includes three-dimensional spatial coordinates, pressure and three-dimensional displacement vector, and each tissue layer includes skin layer, fat layer and glandular layer. The breast tissue deformation field model module is used to obtain the breast tissue deformation field model by linearly solving the Gaussian radial basis function and spatial interpolation based on the deformation data of each tissue layer, with three-dimensional spatial coordinates and pressure as inputs and three-dimensional displacement vectors as outputs; The deformation compensation and probe correction module is used to perform deformation compensation on each pixel in each frame of breast ultrasound image based on the deformation field model of the breast tissue, and to correct the probe pose corresponding to each frame of breast ultrasound image, so as to obtain the compensated breast ultrasound image sequence and the corrected probe pose sequence respectively. The breast three-dimensional reconstruction model module is used to map the compensated ultrasound image sequence to a three-dimensional voxel grid based on the corrected probe pose sequence to obtain a voxel grid reconstruction model, and then superimpose the voxel grid reconstruction model onto the body surface grid generated by the breast three-dimensional point cloud data to obtain a breast three-dimensional reconstruction model under unpressurized conditions.
8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When executed by a processor, the computer program / instruction implements the three-dimensional reconstruction method of breast tissue in an unpressurized state as described in any one of claims 1-6.
9. A computer program product, comprising a computer program / instructions, characterized in that, When executed by a processor, the computer program / instruction implements the three-dimensional reconstruction method of breast tissue in an unpressurized state as described in any one of claims 1-6.
10. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the three-dimensional reconstruction method of breast tissue in an unpressurized state as described in any one of claims 1-6.
