Three-dimensional reconstruction method and system for breast nodule

Abstract

The present disclosure provides a breast nodule three-dimensional reconstruction method and system, the method comprising acquiring multiple frames of breast ultrasound images and preprocessing, the preprocessing comprising filtering denoising processing and image enhancement processing; a lightweight convolutional neural network structure is used to extract features from the images to obtain feature maps; a segmentation module is used to segment the feature maps; SGBM stereo matching operation is performed on the segmented images to obtain disparity information between the images, and the disparity information is adaptively corrected to realize three-dimensional reconstruction of the breast nodule; a dynamic calibration mechanism is established and an adaptive adjustment algorithm is designed to continuously optimize and calibrate the three-dimensional reconstruction operation. Through image preprocessing, optimization algorithm, dynamic calibration and adaptive adjustment, the present disclosure overcomes the problems of high image quality requirement, susceptibility to influence, large calculation resource consumption and low three-dimensional reconstruction precision in the prior art.

Description

Technical Field

[0001] This disclosure relates to the field of medical image processing technology, and in particular to a method and system for three-dimensional reconstruction of breast nodules. Background Technology

[0002] With the continuous development of medical imaging technology, breast ultrasound examination, due to its convenience and non-invasiveness, has been widely used in the field of breast disease screening and plays a vital role in clinical diagnosis. To more accurately assess key information such as the morphology, size, and location of breast nodules, three-dimensional reconstruction technology for breast nodules has emerged.

[0003] Currently, existing 3D reconstruction techniques for breast nodules typically rely on a series of image processing algorithms and related models. Specifically, in the image acquisition stage, multiple frames of 2D breast ultrasound images are obtained using conventional breast ultrasound equipment. These acquired images form the foundational data for subsequent processing. In the feature extraction stage, traditional convolutional neural network (CNN) architectures, such as ResNet, are often used as feature extractors. These networks extract features from the input breast ultrasound images through multiple convolutional and pooling layers, aiming to capture relevant feature information of breast tissue and nodules, such as edges and textures. The next step is segmentation. At this stage, common approaches include using traditional segmentation methods such as thresholding and region growing, or employing deep learning network architectures such as UNet to segment the breast nodule region. These methods distinguish the nodule region in the breast ultrasound image from the surrounding breast tissue, providing relatively accurate nodule location and contour information for subsequent 3D reconstruction. Finally, in the 3D reconstruction stage, most existing technologies use block matching-based stereo vision algorithms, such as the Semi-Global Block Matching (SGBM) algorithm. This algorithm obtains disparity information between images by matching and analyzing adjacent frame images, and then combines relevant parameters such as camera calibration to realize the 3D reconstruction of breast nodules, so as to present the morphology and positional relationship of breast nodules in 3D space.

[0004] While existing three-dimensional reconstruction techniques for breast nodules have made some progress, many problems still exist. On the one hand, the image quality requirements are high and easily affected. Breast ultrasound images have inherent defects such as speckle noise due to their imaging principle, and the image quality varies between different ultrasound devices. Existing technologies lack a systematic and targeted preprocessing step before processing. Although some methods may perform basic filtering methods such as simple grayscale adjustment or mean filtering, these operations have limited effectiveness in removing speckle noise and improving image contrast, making it difficult to fully meet the high-quality image requirements of subsequent complex image processing algorithms. On the other hand, the computational resources are high. Traditional algorithms used in feature extraction, segmentation, and 3D reconstruction, such as ResNet convolutional neural networks and SGBM stereo matching algorithms, involve a large number of matrix operations and complex computational logic. When processing large-scale image data, they consume a lot of CPU time and memory, and may be slow or even unable to run on devices with limited hardware resources, limiting their application in broader scenarios such as primary healthcare units. In addition, the pre-calibrated standard SGBM curves relied upon for 3D reconstruction based on SGBM algorithms may not match due to changes in the speckle characteristics of ultrasound equipment with usage time, environment, and individual differences, making it difficult to guarantee the accuracy and stability of 3D reconstruction results.

[0005] Therefore, it is necessary to improve and innovate existing technologies to overcome these shortcomings and enhance the performance and practicality of three-dimensional reconstruction technology for breast nodules. Summary of the Invention

[0006] In view of this, the purpose of this disclosure is to overcome the problems of high image quality requirements and susceptibility to influence, large computational resource consumption and low three-dimensional reconstruction accuracy of the existing technology, and to provide a method and system for three-dimensional reconstruction of breast nodules.

[0007] To achieve the above objectives, this disclosure provides a method for three-dimensional reconstruction of breast nodules, including:

[0008] Step S1: Acquire multiple frames of breast ultrasound images and preprocess each frame of breast ultrasound image, including filtering and noise reduction and image enhancement.

[0009] Step S2: Use a lightweight convolutional neural network structure to extract features from the preprocessed breast ultrasound image to obtain a feature map;

[0010] Step S3: The extracted feature map is segmented using the segmentation module. During the segmentation process, a loss function is used to measure the difference between the segmentation result and the true label. The parameters of the segmentation module are continuously adjusted based on the difference calculated by the loss function through the backpropagation algorithm.

[0011] Step S4: Perform SGBM stereo matching on the segmented images to obtain disparity information between images, and perform adaptive curve correction on the disparity information to realize three-dimensional reconstruction of breast nodules;

[0012] Step S5: Establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation in step S4. The dynamic calibration mechanism includes periodically recalibrating the speckle characteristics of the equipment used to acquire breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters. The adaptive adjustment algorithm includes adjustment based on matching conditions and adjustment based on reconstruction results.

[0013] In one embodiment, step S1, the filtering and denoising process for each frame of breast ultrasound image includes adaptive median filtering and Gaussian filtering, wherein the method of adaptive median filtering includes:

[0014] An initial window is preset. It is determined whether the maximum difference in pixel grayscale values ​​within the initial window centered on the current pixel exceeds a preset threshold. If not, the initial window is used to perform median filtering. If so, the window size is gradually increased until the maximum difference in pixel grayscale values ​​within the window does not exceed the preset threshold.

[0015] In one embodiment, in step S1, the image enhancement processing for each frame of breast ultrasound image includes histogram equalization based on local contrast and adaptive contrast stretching. The histogram equalization based on local contrast method includes dividing the breast ultrasound image into several non-overlapping sub-regions, calculating the gray-level histogram of each sub-region and its cumulative distribution function, assigning a higher weight to the cumulative distribution function of the current sub-region and a lower weight to the cumulative distribution functions of adjacent sub-regions using a weighted average method to calculate the final cumulative distribution function value used for mapping, and then remapping the gray-level values ​​of the pixels within that sub-region. The adaptive contrast stretching method includes calculating the statistical characteristics of the gray-level histogram of the breast ultrasound image, determining whether the image's gray-level distribution is biased towards dark areas, bright areas, or a uniform distribution based on the statistical characteristics, and automatically determining the stretching based on the determination result. If the distribution is biased towards dark areas, the starting gray-level value range and the new gray-level value range to which the stretching is applied are automatically determined.

[0016] In one embodiment, in step S2, the lightweight convolutional neural network structure is MobileNet.

[0017] In one embodiment, in step S2, the lightweight convolutional neural network structure extracts image features through depthwise separable convolution operations, the method including:

[0018] Each channel of the input breast ultrasound image is individually depthwise convolved, and the output is a set of feature maps with the same number of channels as the input image.

[0019] Perform a 1×1 convolution on each pixel of the depthwise convolution output to adjust the number of channels.

[0020] In one embodiment, the method for performing SGBM stereo matching on the segmented images and obtaining disparity information between images in step S4 includes:

[0021] Extract key feature points from the segmented image, which includes the current frame image and the next frame image;

[0022] The key feature points extracted from the current frame image are matched with the key feature points extracted from the subsequent frame image. During the matching process, feature descriptors are used to describe the key feature points, and the matching relationship between the key feature points is determined by calculating the distance between the feature descriptors. If the distance between the feature descriptors of two key feature points is less than a preset distance threshold, then the two key feature points match; otherwise, the two key feature points do not match.

[0023] After extracting and matching key feature points, the matching cost is calculated by sampling. For each pair of matched key feature points, the absolute difference is calculated every few pixels and then summed to approximate the matching cost. The disparity information is calculated based on the positional difference of the matched region in the current frame and the next frame.

[0024] In one embodiment, the method for establishing a dynamic calibration mechanism in step S5 includes:

[0025] Acquire Q-frame calibration sample images, and for each frame of calibration sample image I... i (i = 1, 2, 3, ..., Q), select a local region centered on each pixel P(x, y);

[0026] Calculate the speckle statistical features within the local region to obtain speckle feature values, and record the speckle feature values ​​of each pixel in each frame of the calibration sample image to form a speckle feature vector V for each frame of the calibration sample image. i (i = 1, 2, 3, ..., Q);

[0027] Calculate the Euclidean distance between speckle feature vectors of calibration sample images collected at different time points, and determine whether the Euclidean distance between speckle feature vectors is greater than the set distance threshold. If it is, recalibration is required; otherwise, recalibration is not required.

[0028] Calculate the average speckle contrast of all multi-frame calibration sample images that require recalibration. Average size of speckle Average correlation coefficient with speckle and the combined eigenvalues As a new speckle characteristic parameter for current breast ultrasound equipment;

[0029] Based on the new speckle characteristic parameters, and combined with the original standard SGBM curve calculation model, a new standard SGBM curve adapted to the speckle characteristics of the current equipment is obtained.

[0030] In one embodiment, the method for adjusting based on matching conditions in step S5 includes:

[0031] For each pair of matched current frame regions and subsequent frame regions, calculate the first evaluation index value for both, where the first evaluation index value includes the mean similarity value and the structural similarity index.

[0032] The calculated first evaluation index value is compared with the set threshold. If the average similarity value is lower than the preset lower threshold, the adjustment range of the coherence coefficient in the adaptive curve correction process needs to be increased. If the structural similarity index exceeds the preset upper and lower thresholds, the resampling ratio of the coherence coefficient needs to be adjusted.

[0033] In one embodiment, the method for adjusting based on the reconstruction effect in step S5 includes:

[0034] The second evaluation index value is calculated according to the preset evaluation index for the reconstruction effect. The second evaluation index value includes the error rate between the reconstructed nodule morphology and the actual anatomical structure, as well as the visual quality evaluation index of the reconstructed image.

[0035] The calculated error rate is compared with the set threshold. If the error rate exceeds the preset upper limit threshold, the calculation method of the SGBM curve in the adaptive curve correction process is changed. If the visual quality evaluation index of the reconstructed image is not good, the entire adaptive curve correction process is redesigned.

[0036] Based on the same inventive concept, this disclosure also provides a three-dimensional reconstruction system for breast nodules, comprising:

[0037] The image preprocessing module is used to acquire multiple frames of breast ultrasound images and preprocess each frame of breast ultrasound image. The preprocessing includes filtering and noise reduction and image enhancement.

[0038] The feature extraction module is used to extract features from preprocessed breast ultrasound images using a lightweight convolutional neural network structure to obtain feature maps.

[0039] The segmentation module is used to segment the feature map obtained by feature extraction. During the segmentation process, a loss function is used to measure the difference between the segmentation result and the true label. The parameters of the segmentation module are continuously adjusted based on the difference calculated by the loss function through the backpropagation algorithm to obtain an image result that accurately segments the breast nodule region.

[0040] The three-dimensional reconstruction module is used to perform SGBM stereo matching on images with accurately segmented breast nodule regions, while acquiring disparity information between images and performing adaptive curve correction on the disparity information to achieve three-dimensional reconstruction of breast nodules.

[0041] The calibration and optimization control module is used to establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation. The dynamic calibration mechanism includes periodically recalibrating the speckle characteristics of the equipment used to acquire breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters. The adaptive adjustment algorithm includes adjustment based on matching conditions and adjustment based on reconstruction results.

[0042] Based on the same inventive concept, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the three-dimensional reconstruction method for breast nodules described in any of the above embodiments.

[0043] Based on the same inventive concept, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the three-dimensional reconstruction method for breast nodules described in any of the above embodiments.

[0044] As can be seen from the above, the three-dimensional reconstruction method and system for breast nodules provided in this disclosure have many significant beneficial effects in the three-dimensional reconstruction of breast nodules. First, through a carefully designed image preprocessing stage, effective filtering and denoising techniques are used to not only effectively reduce the impact of noise on subsequent processing, but also significantly improve the clarity and contrast of the image, providing higher-quality input data for feature extraction, segmentation, and three-dimensional reconstruction, thereby ensuring the accuracy of each stage of processing. Second, the optimization algorithm and hardware acceleration measures play an important role. On the one hand, the optimization algorithm reduces computational complexity; on the other hand, the hardware acceleration device improves operating efficiency, enabling the system to complete the three-dimensional reconstruction of breast nodules more quickly and stably when processing large amounts of breast ultrasound image data or running on devices with limited hardware resources. Furthermore, the construction of a dynamic calibration mechanism and the design of an adaptive adjustment algorithm specifically address the limitations of the adaptive curve correction method. By adjusting the correction parameters and methods in real time, the correction effect and the accuracy of three-dimensional reconstruction of breast nodules are effectively improved, ensuring the accuracy and reliability of the three-dimensional reconstruction results. This provides strong support for the diagnosis and three-dimensional visualization of breast nodules, demonstrating high practical value and promising prospects for promotion. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in this disclosure or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating a method for three-dimensional reconstruction of breast nodules according to one embodiment;

[0047] Figure 2 A comparison chart showing the processing efficiency between the improved group and the standard group;

[0048] Figure 3 A comparison chart showing the reconstruction accuracy between the improved group and the standard group;

[0049] Figure 4 A comparison chart showing how the improved group and the standard group respond to ultrasound equipment malfunctions;

[0050] Figure 5 A comparison of the improved group and the conventional group in response to sudden changes in the imaging environment;

[0051] Figure 6 This is a modular structure diagram of a three-dimensional reconstruction system for breast nodules according to one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0053] To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and components have been omitted.

[0054] like Figure 1 The flowchart of a three-dimensional reconstruction method for breast nodules according to one embodiment shown includes steps S1 to S5:

[0055] Step S1: Acquire multiple frames of breast ultrasound images and preprocess each frame of breast ultrasound image, including filtering and noise reduction and image enhancement.

[0056] Step S2: Use a lightweight convolutional neural network structure to extract features from the preprocessed breast ultrasound image to obtain a feature map;

[0057] Step S3: The extracted feature map is segmented using the segmentation module. During the segmentation process, a loss function is used to measure the difference between the segmentation result and the true label. The parameters of the segmentation module are continuously adjusted based on the difference calculated by the loss function through the backpropagation algorithm.

[0058] Step S4: Perform SGBM stereo matching on the segmented images to obtain disparity information between images, and perform adaptive curve correction on the disparity information to achieve three-dimensional reconstruction of breast nodules;

[0059] Step S5: Establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation in step S4. The dynamic calibration mechanism includes periodically recalibrating the speckle characteristics of the equipment used to acquire breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters. The adaptive adjustment algorithm includes adjustment based on matching conditions and adjustment based on reconstruction results.

[0060] This disclosure provides a three-dimensional reconstruction method for breast nodules, which has many significant benefits in this area. First, through a carefully designed image preprocessing stage, effective filtering, denoising, and image enhancement techniques are employed. This not only effectively reduces the impact of noise on subsequent processing but also significantly improves image clarity and contrast, providing higher-quality input data for feature extraction, segmentation, and three-dimensional reconstruction, thus ensuring the accuracy of each stage. Second, the optimized algorithm and hardware acceleration measures play a crucial role. On the one hand, the optimized algorithm reduces computational complexity; on the other hand, the hardware acceleration device improves operating efficiency, enabling faster and more stable completion of the three-dimensional reconstruction task of breast nodules even when processing large amounts of breast ultrasound image data or running on devices with limited hardware resources. Third, the construction of a dynamic calibration mechanism and the design of an adaptive adjustment algorithm specifically address the limitations of adaptive curve correction methods. By adjusting the correction parameters and methods in real time, the correction effect and the accuracy of three-dimensional reconstruction of breast nodules are effectively improved, ensuring the accuracy and reliability of the three-dimensional reconstruction results. This provides strong support for the diagnosis and three-dimensional visualization of breast nodules, demonstrating high practical value and promising prospects for widespread application.

[0061] As a preferred implementation, in step S1, the filtering and denoising process for each frame of breast ultrasound image includes adaptive median filtering. Specifically, the adaptive median filtering method includes: checking whether the maximum difference in pixel grayscale values ​​within a small initial window (e.g., 3×3) centered on the current pixel exceeds a preset threshold. If it does not exceed the threshold, it indicates that the region is relatively smooth, and this small window can be directly used for median filtering. If it exceeds the threshold, it means that the region may have noise or rich details, and the filtering window needs to be expanded. In this case, the window size is gradually increased (e.g., increasing the side length by 2 pixels each time, from 3×3 to 5×5, 7×7, etc.), and the maximum difference in pixel grayscale values ​​within the new window is checked again to see if it meets the condition, until a suitable window size is found so that the difference is within the threshold range, and then median filtering is performed. This implementation of adaptive median filtering can remove noise more accurately according to the actual situation of different regions of the image. While preserving image edge and detail information, it can better adapt to the local complexity of the image, improving the uniformity of the denoising effect and the overall image quality.

[0062] Furthermore, in step S1, the filtering and denoising process for each frame of breast ultrasound image also includes Gaussian filtering. Gaussian filtering is based on the weighted average principle of the Gaussian function, which is effective in removing Gaussian noise from the image and can preserve image details such as edges to a certain extent. The detailed process is as follows: First, a Gaussian kernel matrix needs to be generated according to the set parameters. The size of the Gaussian kernel matrix is ​​usually a square with an odd number of side lengths, such as 3×3 or 5×5, and its element values ​​are calculated based on the Gaussian function. The expression for the Gaussian function is: Where (x, y) represents the coordinate position in the Gaussian kernel matrix, and σ is the standard deviation, which determines the shape of the Gaussian function, i.e., the "width" of the Gaussian kernel. For each pixel to be processed in the image, a neighborhood of the same size as the Gaussian kernel is selected centered on that pixel. For example, for pixel (x, y), if the above-mentioned 3×3 Gaussian kernel is used, the nine surrounding pixel values ​​(coordinate range from (x-1, y-1) to (x+1, y+1)) centered on (x, y) are selected. Each pixel value in the neighborhood is multiplied by the element value at the corresponding position in the Gaussian kernel matrix, and then these products are added together. The sum is the new value of pixel (x, y) after Gaussian filtering. This implementation performs this operation on each pixel in the image, which can filter the image according to the weighted average method of the Gaussian function, remove Gaussian noise, make the image smoother, and at the same time, because it is a weighted average, it can also better preserve the edge and other details of the image.

[0063] In a preferred implementation, step S1 involves image enhancement processing for each frame of breast ultrasound image, including histogram equalization based on local contrast. Specifically, the histogram equalization method based on local contrast includes: first, dividing the breast ultrasound image into several non-overlapping sub-regions (e.g., 8×8 or 16×16 blocks); for each sub-region, calculating the gray-level histogram and cumulative distribution function (CDF); then, during gray-level remapping, considering both the CDF of the current sub-region and the CDF information of adjacent sub-regions. For example, a weighted average can be used, giving the CDF of the current sub-region a higher weight (e.g., 0.7) and the CDF of adjacent sub-regions a lower weight (e.g., 0.3) to calculate the final CDF value used for mapping, and then remapping the gray-level values ​​of the pixels within that sub-region. This implementation better considers the contrast requirements of local image regions, avoiding the local contrast imbalance problem that may occur with traditional methods, allowing each local region of the image to receive more appropriate contrast enhancement, thereby improving the overall clarity and visibility of the image.

[0064] Further, in step S1, the image enhancement processing for each frame of breast ultrasound image includes adaptive contrast stretching. Specifically, the adaptive contrast stretching method includes: first, calculating some statistical characteristics of the image's gray-level histogram, such as the mean, standard deviation, skewness, and kurtosis of the gray-level values; then, based on these statistical characteristics, it can be determined whether the image's gray-level distribution is biased towards dark areas, bright areas, or relatively uniform. If the image's gray-level distribution is biased towards dark areas (e.g., gray levels are greater than a certain threshold and the mean is small), the initial gray-level value range for stretching is automatically determined to be from the minimum gray-level value of the image to an intermediate value calculated based on the standard deviation (e.g., mean + standard deviation). The new range to be stretched can be set from 0 to an intermediate value calculated based on the maximum and minimum gray-level values ​​of the image (e.g., (maximum value - minimum value) / 2). Similarly, if the image's gray-level distribution is biased towards bright areas (gray levels are less than a certain threshold and the mean is large), the initial gray-level value range for stretching and the new range to be stretched are also determined according to the corresponding rules based on the statistical characteristics. If the grayscale distribution of the image is relatively uniform, it can be stretched according to a preset default range. This implementation method of adaptive contrast stretching requires no manual intervention and can automatically determine appropriate stretching parameters based on the image's own grayscale histogram characteristics. This improves the automation and flexibility of the image enhancement process, enabling different types of ultrasound images to achieve more suitable contrast enhancement and further improve image quality.

[0065] The image preprocessing method employed in this embodiment works synergistically through a carefully designed combination, producing unique effects that effectively improve image quality and have a highly positive impact on subsequent steps of 3D reconstruction of breast nodules. Specifically, in terms of noise removal, it effectively removes various common noise types found in ultrasound images, resulting in cleaner images. Regarding contrast enhancement, the combined use of histogram equalization and contrast stretching achieves a more significant and targeted contrast improvement than existing single grayscale adjustment methods, making breast nodules stand out more in the image. In terms of detail rendering, adaptive median filtering and Gaussian filtering preserve image details while removing noise, and subsequent contrast stretching further enhances these details. This allows the model to more accurately capture the features of breast nodules in subsequent feature extraction, segmentation, and 3D reconstruction steps, improving segmentation accuracy and thus enhancing the precision of 3D reconstruction, enabling a more realistic representation of the true shape and size of breast nodules.

[0066] As a preferred implementation, in step S2, the lightweight convolutional neural network structure used in this embodiment is MobileNet, preferably MobileNetV2. It employs depthwise separable convolution operations to extract image features. First, each channel of the input breast ultrasound image undergoes a depthwise convolution, outputting a set of feature maps with the same number of channels as the input image. Then, each pixel output by the depthwise convolution is subjected to a 1×1 convolution to adjust the number of channels. Compared to traditional convolution operations, this significantly reduces the computational load, especially when the number of channels and the output channel number are large. This embodiment reduces the computational complexity of the feature extraction stage while ensuring the extraction of effective image features, thereby reducing the consumption of computing resources. Furthermore, in terms of hardware, hardware acceleration devices such as graphics processing units (GPUs) and field-programmable gate arrays (FPGAs) can be used to accelerate the entire processing and improve operational efficiency through their parallel computing capabilities.

[0067] As a preferred implementation, in step S3, the extracted features can be segmented using network architectures such as UNet or traditional segmentation methods (such as thresholding, region growing, etc.) to divide the breast ultrasound image into different regions, with a focus on accurately segmenting the region where the breast nodules are located. During the segmentation process, a loss function (such as a combination of DiceLoss and BCELoss) is used to measure the difference between the segmentation result and the true label, and the parameters of the segmentation module are continuously optimized through the backpropagation algorithm to accurately segment the region where the breast nodules are located.

[0068] In a preferred implementation, in step S4, key feature points are first extracted from the segmented images (current frame and subsequent frame images). The key feature points extracted from the current frame image are then matched with those extracted from the subsequent frame image. During matching, feature descriptors (such as SIFT feature descriptors) are used to describe the key feature points, and the matching relationship between the key feature points is determined by calculating the distance between the feature descriptors (such as Euclidean distance). If the distance between the feature descriptors of two key feature points is less than a preset distance threshold, the two key feature points match; otherwise, they do not match. After the extraction and matching of key feature points are completed, the matching cost is calculated using a sampling method. For each pair of matched key feature points, the absolute difference is calculated every few pixels and summed to approximate the matching cost. Simultaneously, disparity information is calculated based on the positional differences of the matched regions in the current frame and subsequent frame images. This implementation, through a feature-based stereo matching method, can significantly reduce the number of elements that need to be matched, thereby reducing the computational load.

[0069] Specifically, the implementation process of the feature-based stereo matching algorithm described above is as follows:

[0070] 1. Key Feature Point Extraction: Input the segmented current frame image I curr and the next frame image I next and preset feature extraction parameters (including corner response threshold T) response Neighborhood size N size (etc.); for the current frame image I curr Using a selected feature extraction method (such as the Harris corner extraction algorithm or the SIFT feature point extraction algorithm), with each pixel (x, y) as the center, based on the neighborhood size N... size Determine the corresponding neighborhood region; calculate the feature response function value of the pixel within its neighborhood region. If the calculated feature response function value R(x, y) is greater than the corner response threshold T... response Then, the pixel (x, y) is marked as a potential key feature point and stored in the potential key feature point set P of the current frame image. curr In the middle. Similarly, for the subsequent frame image I next Repeat the above steps to obtain the set of potential key feature points P of the subsequent frame image. next .

[0071] 2. Feature descriptor generation: For each potential key feature point p in the current frame image... curr ∈P curr The selected feature descriptor generation method (such as SIFT feature descriptor generation method) is used to generate its corresponding feature descriptor D. curr (p curr This process involves analyzing and encoding image information about the region surrounding a key feature point to generate a vector that describes the features of that feature point. Similarly, for each potential key feature point p in a subsequent frame image... next ∈P next Similarly, the above method is used to generate its corresponding feature descriptor D. next (p next ).

[0072] 3. Feature point matching: Input the feature descriptor subset D of the current frame image. curr ={D curr (p curr )|p curr ∈P curr} and the feature descriptor subset D of the subsequent frame image next ={D next (p next )|p next ∈P next}, preset distance threshold T distance ; Traverse the feature descriptor subset D of the current frame image curr Each feature descriptor D in curr(p curr ): Calculate the feature descriptor set D of the current feature descriptor and the subsequent frame image. next Each feature descriptor D next (p next The distance between two feature descriptor vectors D. Taking Euclidean distance as an example, for two feature descriptor vectors D... curr (p curr ) and D next (p next The formula for calculating the Euclidean distance between them is: Where n represents the dimension of the feature descriptor vector; if the calculated distance d(p curr p nest Less than the distance threshold T distance Then it is considered that feature point p in the current frame image curr Feature point p in the subsequent frame image nest If they match, record the matching feature points and store them in the matching feature point set A.

[0073] 4. Calculate the matching cost: For each pair of matching feature points (p) in the matching feature point set A... curr p nest A simplified method for calculating the matching cost can be used, such as sampling to calculate the sum of absolute differences (SAD). Assuming a region (e.g., a 5×5 region) is defined centered on each matching feature point, the absolute difference is calculated and summed every b pixels (e.g., b=3) to obtain the matching cost for that pair of matching feature points.

[0074] Once the matching relationships between regions in different images are determined, disparity can be calculated based on the positional differences of the matching regions in the two images. By performing this disparity calculation on each matching region in the image, disparity information for different points on the entire image can be obtained. This disparity information exists in the form of disparity values ​​for each matching point (or matching region), reflecting the positional shift of the breast nodule in images from different viewpoints. It serves as the foundational data for subsequent 3D reconstruction and disparity correction. However, this disparity information may contain certain errors or be inaccurate, requiring further correction and optimization. The purpose of adaptive curve correction is to improve the accuracy of disparity information and the rationality of related parameters, thereby enabling 3D reconstruction based on this disparity information to more accurately represent the morphology and positional relationships of the breast nodule in 3D space.

[0075] Preferably, in this embodiment, adaptive curve correction first uses a simplified speckle model to approximate the SGBM curve. Through speckle characteristic analysis of a large number of breast ultrasound images, a simplified speckle model is established. For each region in the image, the SGBM curve for that region is calculated based on its center coordinates (x, y) and the simplified speckle model. Then, coherence coefficient adjustment and simplification are performed. A threshold is set when calculating the coherence coefficient. When the change in the coherence coefficient is within a certain threshold range, no subsequent adjustments or resampling are performed; the result from the previous round is directly used, or simple linear interpolation is applied. For example, if the threshold for coherence coefficient change is set to 0.05, when the difference between the calculated coherence coefficient and the previous round's coherence coefficient is less than 0.05, the previous round's coherence coefficient value is directly used. If the difference is greater than or equal to 0.05, adjustments and resampling are performed using conventional methods. After adaptive curve correction, more accurate disparity information and related parameters are obtained, further improving the accuracy of 3D reconstruction.

[0076] Specifically, the implementation process of the above adaptive curve correction is as follows:

[0077] 1. Establishing SGBM curves using a simplified speckle model: First, input a large dataset of breast ultrasound images I dataset The image size is M×N (M is the height, N is the width). Iterate through each image I in the dataset. i ∈I dataset (i = 1, 2, ..., |I dataset For each image, divide it into non-overlapping small regions, for example, regions of size w×w. Let the image be horizontally divisible into... The region can be divided vertically into several areas. There are regions, and for each subregion R ij (i = 1, 2, ..., m; j = 1, 2, ..., n), calculate the statistics related to its speckle characteristics, such as the standard deviation σ of the pixel gray values ​​within the region. ij With mean μ ij The speckle contrast is Another method involves using edge detection algorithms (such as Canny edge detection) to detect the edges of the speckle, and then calculating the average diameter or area of ​​the region enclosed by the edges as the speckle size S. ij The estimated value is then obtained. A quadratic function is then used to approximate the distribution of the speckle pattern, assuming the quadratic function is of the form f(x, y) = ax. 2 +by 2+cxy+dx+ey+f, where (x, y) are the coordinates within the region, and a, b, c, d, e, f are the coefficients to be determined. Based on the speckle characteristic statistics of each small region, methods such as multiple linear regression are used to fit and determine the coefficients of the quadratic function. For example, the center coordinates (x, y) of each region are... ij y ij )(in The simplified speckle model coefficients a, b, c, d, e, and f are used as sample data and substituted into a quadratic function to construct a system of linear equations. Solving this system of equations determines the coefficients a, b, c, d, e, and f. Next, the simplified speckle model coefficients a, b, c, d, e, and f, along with the center coordinates (x, y) of each region in the image, are input. For each region, its center coordinates (x, y) are substituted into the simplified speckle model quadratic function f(x, y) to calculate the SGBM curve value for that region. curve (x, y).

[0078] 2. Simplified Coherence Coefficient Adjustment: First, input the image data after SGBM stereo matching (such as matching region information), and the initial coherence coefficient value K0 (an initial value can be set based on experience or relevant research). Based on the initial setting and relevant calculation rules (the specific calculation method can be determined according to the actual SGBM stereo matching and subsequent processing methods), calculate the first-round coherence coefficient value K1. For example, if a calculation method based on pixel differences and region matching is used, combined with information such as SGBM curve values, the coherence coefficient can be calculated. Then, input the coherence coefficient value K1 from the previous round. prev The set coherence coefficient variation threshold ΔK is used to recalculate the coherence coefficient value K in each iteration (starting from the second iteration) based on the current image data and relevant calculation rules. current And calculate the difference ΔK between the current coherence coefficient value and the previous coherence coefficient value. current =|K current -K prev Next, determine ΔK. current The relationship between ΔK and its magnitude, if ΔK current If the change in the coherence coefficient is less than ΔK, meaning it falls within a set threshold range, the coherence coefficient value from the previous round is directly used as the result for this round, i.e., K. current =K prev If ΔK current If the value is ≥ΔK, then complex operations such as adjustment and resampling are performed according to conventional methods. For example, this may involve re-evaluating and adjusting the coherence coefficient based on more detailed information and disparity information of the image, recalculating the SGBM curve, and then obtaining a new coherence coefficient value as the result of this round.

[0079] This implementation method utilizes calibrated SGBM curve values ​​and coherence coefficient values, along with disparity information obtained from previous SGBM stereo matching and relevant information such as camera calibration (e.g., camera intrinsic and extrinsic parameters), to perform three-dimensional reconstruction of breast nodules using a specific three-dimensional reconstruction algorithm (e.g., an algorithm based on the principle of triangulation).

[0080] As a preferred implementation, the method for establishing a dynamic calibration mechanism in step S5 specifically includes the following steps:

[0081] Step S511: Acquire Q-frame calibration sample images. For each frame of calibration sample image I... i (i = 1, 2, 3, ..., Q), select a local region centered on each pixel P(x, y) (e.g., a 5×5 or 7×7 window centered on the pixel, with a window size of W×W). For example, the speckle characteristics of the device used to acquire breast ultrasound images can be recalibrated periodically (e.g., every certain time period or after a certain number of imaging operations). Select a set of standard objects with known geometry and size (e.g., a specially designed breast ultrasound calibration module with regular geometry), place them in the imaging area of ​​the breast ultrasound device, and acquire multiple frames of breast ultrasound images as calibration samples according to the normal ultrasound imaging operation procedure.

[0082] Step S512: Calculate the speckle statistical features in the local area to obtain speckle feature values, and record the speckle feature values ​​of each pixel in each frame of the calibration sample image to form the speckle feature vector V of each frame of the calibration sample image. i (i = 1, 2, 3, ..., Q);

[0083] Step S513: Calculate the Euclidean distance between the speckle feature vectors of the calibration sample images collected at different time points, and determine whether the Euclidean distance between the speckle feature vectors is greater than the set distance threshold. If it is, recalibration is required; otherwise, recalibration is not required.

[0084] Step S514: Calculate the average speckle contrast of all multi-frame calibration sample images that need to be recalibrated. Average size of speckle Average correlation coefficient with speckle and the combined eigenvalues As a new speckle characteristic parameter for current breast ultrasound equipment;

[0085] Step S515: Based on the new speckle characteristic parameters, recalculate using the original standard SGBM curve calculation model to obtain a standard SGBM curve adapted to the speckle characteristics of the current equipment, which will be used for subsequent adaptive curve correction operations. For example, in the original standard SGBM curve calculation model, the part related to speckle contrast is f(C), the part related to speckle size is g(S), and the part related to speckle correlation is h(R). Therefore, the updated standard SGBM curve calculation function can be set as follows:

[0086] The aforementioned dynamic calibration mechanism brings several significant benefits. Firstly, it improves image quality and feature extraction accuracy. By periodically collecting, analyzing, and determining new parameters for the speckle characteristics of the equipment used to acquire breast ultrasound images, it accurately reflects equipment characteristics, making subsequent ultrasound images more consistent with reality. Simultaneously, it optimizes the feature extraction process, allowing speckle-based feature extraction algorithms to more accurately locate and extract effective features related to breast nodules. Secondly, this mechanism improves stereo matching and 3D reconstruction accuracy. Updating the standard SGBM curve based on new speckle characteristic parameters better adapts it to SGBM stereo matching, improving disparity calculation accuracy and stereo matching precision. This provides more reliable basic data for 3D reconstruction of breast nodules, more accurately presenting the morphology and location of nodules in 3D space, assisting doctors in accurate diagnosis. Thirdly, it enhances the system's adaptability to equipment changes, addressing equipment aging, environmental influences, and differences between different devices. It promptly detects and adjusts for speckle characteristic changes caused by these factors, ensuring effective system operation under different equipment and equipment conditions. Finally, this mechanism also provides long-term and stable diagnostic support. Through continuous dynamic calibration, it ensures the stability and reliability of the image analysis basis, reduces diagnostic errors caused by changes in equipment or image parameters in long-term breast nodule diagnosis, promotes the application of this technology in clinical practice, and thus provides better breast nodule diagnosis services for a wider range of patients.

[0087] As a preferred implementation, in step S5, the method for adjusting based on matching conditions includes: First, calculating the first evaluation index value for each pair of matched current frame regions and subsequent frame regions, wherein the first evaluation index value includes the mean similarity value and the structural similarity index SSIM; then, comparing the calculated first evaluation index value with a set threshold. If the mean similarity value is lower than a preset lower threshold, it indicates that the matching effect is poor, and the adjustment range of the coherence coefficient during the adaptive curve correction process needs to be increased. For example, if the original coherence coefficient adjustment range is s, it can be updated to s+Δs; if the structural similarity index SSIM exceeds the preset upper and lower thresholds, it indicates that the matching effect is not ideal, and the resampling ratio of the coherence coefficient needs to be adjusted. For example, if the original coherence coefficient resampling ratio is r, it can be updated to r+Δr or r-Δr depending on the specific situation.

[0088] As another preferred implementation, in step S5, the method for adjusting based on reconstruction effect includes: first, calculating a second evaluation index value according to a preset index for evaluating reconstruction effect, wherein the second evaluation index value includes the error rate between the reconstructed nodule morphology and the actual anatomical structure and the visual quality evaluation index of the reconstructed image; then, comparing the calculated error rate (such as the nodule volume error rate) with a set threshold; if the error rate exceeds the preset upper threshold, changing the calculation method of the SGBM curve in the adaptive curve correction process; if the visual quality evaluation index of the reconstructed image is poor, such as the sharpness index being lower than the preset lower threshold or the contrast index exceeding the preset upper and lower thresholds, then the entire adaptive curve correction process needs to be redesigned.

[0089] The aforementioned adaptive adjustment algorithm brings numerous significant benefits. In optimizing matching performance and improving stereo matching accuracy, it accurately evaluates the matching effect by calculating assessment indicators such as the average similarity of pixels within the matching region and the structural similarity index. Then, based on the comparison results of these assessment indicators and set thresholds, it specifically adjusts the relevant parameters of the adaptive curve correction, thereby optimizing the matching effect and improving the stereo matching accuracy of SGBM, laying a solid foundation for subsequent 3D reconstruction. Regarding improving the quality and accuracy of 3D reconstruction results, after completing the 3D reconstruction of breast nodules, it comprehensively measures the reconstruction effect by calculating various aspects, including the error rate between nodule volume, area, shape, and actual anatomical structure, as well as visual quality assessment indicators of the reconstructed image. Furthermore, it flexibly adjusts the adaptive curve correction method based on the comparison results of indicators and thresholds, effectively improving the reconstruction effect. This makes the reconstructed breast nodules closer to the actual anatomical structure and improves the visual quality of the image, providing doctors with accurate and clear diagnostic information. Simultaneously, the algorithm enhances adaptability and stability, flexibly adjusting according to different situations in the matching and reconstruction stages of the image, adapting to various changes. Continuous monitoring and dynamic adjustment ensure stable operation of the algorithm, providing stable and reliable technical support for breast nodule diagnosis. In addition, it greatly contributes to the accuracy and reliability of clinical diagnosis. With improved stereo matching accuracy and 3D reconstruction quality, doctors can obtain accurate nodule information based on more accurate and clear reconstructed images that match the actual anatomical structure, reducing misdiagnosis and missed diagnosis, and better meeting patients' needs for breast nodule diagnosis.

[0090] The following comparison of the three-dimensional reconstruction method for breast nodules provided in this embodiment (hereinafter referred to as the "improved method") with existing conventional methods demonstrates the advantages of the improved method in terms of processing efficiency, reconstruction accuracy, and adaptability to complex situations.

[0091] This experiment selected 100 patients with breast nodules from different medical institutions as subjects. Each patient underwent multiple breast ultrasound examinations, resulting in 300 sets of breast ultrasound image data as experimental samples. These samples covered breast nodule images of different sizes, shapes, locations, and acquired using different imaging devices to ensure the generality and representativeness of the experimental results. The experimental samples were randomly divided into two groups: one group used existing conventional methods for three-dimensional reconstruction of breast nodules (hereinafter referred to as the "conventional group"), and the other group used the improved three-dimensional reconstruction method for breast nodules described in this embodiment (hereinafter referred to as the "improved group"). Under the same experimental environment, the two groups were processed separately, and the relevant experimental data were recorded.

[0092] 1. Processing Efficiency Comparison: On one hand, in the image preprocessing stage, the conventional group performs routine operations such as filtering and denoising, and image enhancement on each group of breast ultrasound images. Because the existing filtering algorithms may be relatively complex and not optimized, a significant amount of time is spent on processing image noise and enhancing contrast, averaging approximately 30 minutes. In contrast, the improved group, by employing optimized filtering algorithms (such as adaptive median filtering) and more efficient image enhancement techniques (such as histogram equalization based on local contrast and adaptive contrast stretching), reduces the average image preprocessing time to approximately 15 minutes, half that of the conventional group, significantly improving preprocessing efficiency. On the other hand, after image preprocessing, the conventional group performs 3D reconstruction of breast nodules, including a series of operations such as SGBM stereo matching and adaptive curve correction. The entire process takes an average of approximately 90 minutes of computation per group of images. The SGBM stereo matching stage, due to its pixel-based matching method, involves a large computational load and consumes considerable time. The improved group replaced the pixel-based matching method with a feature-based stereo matching method, and combined this with simplified matching cost calculation, simplified SGBM curve calculation, and simplified coherence coefficient adjustment, resulting in a significant reduction in the total computation time for 3D reconstruction. For each set of images, the average total computation time was approximately 35 minutes, a reduction of nearly 61.1% compared to the conventional group. Figure 2 It is clear that the improved group significantly outperforms the conventional group in both image preprocessing and total computation time for 3D reconstruction, demonstrating the remarkable advantage of the improved method in processing efficiency.

[0093] 2. Comparison of Reconstruction Accuracy: On one hand, after completing the 3D reconstruction of breast nodules, the conventional group compared the reconstructed nodule volume with the actual anatomical structure (obtainable through subsequent surgical pathology results or other more accurate measurement methods) to calculate the error rate between the reconstructed nodule volume and the actual nodule volume. Statistical analysis of all sample data showed that the average nodule volume error rate in the conventional group was approximately 18%, indicating that the existing method has a certain deviation in reconstructing breast nodule volume, which may affect the doctor's accurate judgment of nodule size. The improved group, due to optimized image preprocessing, more accurate stereo matching, and adaptive adjustment algorithms, significantly reduced the error rate between the reconstructed nodule volume and the actual nodule volume. Statistically, the average nodule volume error rate in the improved group was approximately 8%, a reduction of approximately 55.6% compared to the conventional group. On the other hand, the conventional group calculated the similarity index between the reconstructed nodule shape and the actual nodule shape (measured using methods such as Hausdorff distance). The average nodule shape similarity index in the conventional group was approximately 0.6, indicating that the reconstructed nodule shape by the existing method differs from the actual shape and is not accurately reconstructed. The improved group, through optimized 3D reconstruction algorithms, particularly the fine-tuning of parameters in the adaptive curve correction stage, achieved an average nodule shape similarity index of approximately 0.85 after reconstruction, representing an improvement of about 41.7% compared to the conventional group. (Calculation method: (0.85-0.6) / 0.6×100%). Figure 3 It is evident that the improved group outperformed the conventional group in both nodule volume error rate and nodule shape similarity index, indicating that the improved three-dimensional reconstruction method for breast nodules has a significant improvement in reconstruction accuracy.

[0094] 3. Comparison of Adaptability to Complex Situations: On the one hand, the conventional group uses existing methods to handle situations where the simulated ultrasound equipment malfunctions, causing significant changes in speckle characteristics. Due to the lack of a mechanism for rapid recalibration of speckle characteristics, the time required to resume normal 3D reconstruction after equipment failure is long, averaging about 6 hours, and the initial 3D reconstruction accuracy error rate after recalibration is high, averaging about 15%. In contrast, the improved group provides an emergency solution for rapid recalibration of speckle characteristics in its improved method, shortening the recalibration cycle and employing a simplified speckle characteristic acquisition method. Under the same equipment failure simulation conditions, the improved group requires an average of about 2 hours to resume normal 3D reconstruction, a reduction of approximately 66.7% compared to the conventional group. Simultaneously, the initial 3D reconstruction accuracy error rate after recalibration averages about 8%, a reduction of approximately 46.7% compared to the conventional group. On the other hand, when sudden changes in the imaging environment (such as temperature and humidity changes) affect image matching, the existing method lacks a mechanism for adaptive adjustment based on environmental parameters. Under simulated environmental abrupt changes, the conventional group exhibited a significant fluctuation in the 3D reconstruction accuracy error rate after the abrupt change, averaging approximately 20%, with a final stable 3D reconstruction accuracy error rate averaging approximately 15%. The improved group, however, proposed a method based on adaptive adjustment of environmental parameters, adjusting the parameters and methods of adaptive curve correction in real time according to environmental changes. Under the same simulated environmental abrupt changes, the improved group showed an average fluctuation of approximately 5% in the 3D reconstruction accuracy error rate after the abrupt change, a reduction of approximately 75% compared to the conventional group. Furthermore, the final stable 3D reconstruction accuracy error rate averaged approximately 8%, a reduction of approximately 46.7% compared to the conventional group. Figure 4 and Figure 5 It is clear that the improved group outperformed the conventional group in dealing with complex situations such as ultrasound equipment failure and sudden changes in the imaging environment. This was evident in the time required to resume normal 3D reconstruction, the accuracy error rate of the first 3D reconstruction after recalibration, the range of change in the accuracy error rate of 3D reconstruction after environmental changes, and the accuracy error rate of 3D reconstruction after final stabilization. This fully demonstrates the advantages of the improved breast nodule 3D reconstruction method in adapting to complex situations.

[0095] Therefore, the three-dimensional reconstruction method for breast nodules provided in this embodiment is significantly superior to existing conventional methods in terms of processing efficiency, reconstruction accuracy, and adaptability to complex situations. These advantages not only improve the efficiency and quality of three-dimensional reconstruction of breast nodules, but also better address various complex situations that may arise in practical applications, providing stronger support for the diagnosis and treatment of breast nodules. It has significant clinical application value and promising prospects for promotion.

[0096] Correspondingly, such as Figure 6 As shown in the embodiments of this disclosure, a three-dimensional reconstruction system for breast nodules is also provided, comprising:

[0097] The image preprocessing module is used to acquire multiple frames of breast ultrasound images and preprocess each frame of breast ultrasound image. The preprocessing includes filtering and noise reduction and image enhancement.

[0098] The feature extraction module is used to extract features from preprocessed breast ultrasound images using a lightweight convolutional neural network structure to obtain feature maps.

[0099] The segmentation module is used to segment the feature map obtained by feature extraction. During the segmentation process, a loss function is used to measure the difference between the segmentation result and the true label. The parameters of the segmentation module are continuously adjusted based on the difference calculated by the loss function through the backpropagation algorithm to obtain an image result that accurately segments the breast nodule region.

[0100] The three-dimensional reconstruction module is used to perform SGBM stereo matching on images with accurately segmented breast nodule regions, while acquiring disparity information between images and performing adaptive curve correction on the disparity information to achieve three-dimensional reconstruction of breast nodules.

[0101] The calibration and optimization control module is used to establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation. The dynamic calibration mechanism includes periodically recalibrating the speckle characteristics of the equipment used to acquire breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters. The adaptive adjustment algorithm includes adjustment based on matching conditions and adjustment based on reconstruction results.

[0102] This disclosure provides a three-dimensional reconstruction method for breast nodules, which has many significant benefits in this area. First, through a carefully designed image preprocessing stage, effective filtering, denoising, and image enhancement techniques are employed. This not only effectively reduces the impact of noise on subsequent processing but also significantly improves image clarity and contrast, providing higher-quality input data for feature extraction, segmentation, and three-dimensional reconstruction, thus ensuring the accuracy of each stage. Second, the optimized algorithm and hardware acceleration measures play a crucial role. On the one hand, the optimized algorithm reduces computational complexity; on the other hand, the hardware acceleration device improves operating efficiency, enabling faster and more stable completion of the three-dimensional reconstruction task of breast nodules even when processing large amounts of breast ultrasound image data or running on devices with limited hardware resources. Third, the construction of a dynamic calibration mechanism and the design of an adaptive adjustment algorithm specifically address the limitations of adaptive curve correction methods. By adjusting the correction parameters and methods in real time, the correction effect and the accuracy of three-dimensional reconstruction of breast nodules are effectively improved, ensuring the accuracy and reliability of the three-dimensional reconstruction results. This provides strong support for the diagnosis and three-dimensional visualization of breast nodules, demonstrating high practical value and promising prospects for widespread application.

[0103] The system described above is used to implement the corresponding three-dimensional reconstruction method for breast nodules in the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0104] It should be noted that the accompanying drawings of the embodiments of this disclosure only involve structures related to the embodiments of this disclosure, and other structures can be referred to in general design.

[0105] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0106] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0107] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the communication method based on asymmetric encryption as described in any of the above embodiments.

[0108] The electronic device in this embodiment is used to implement the corresponding three-dimensional reconstruction method of breast nodules in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0109] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the three-dimensional reconstruction method of breast nodules as described in any of the above embodiments.

[0110] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the three-dimensional reconstruction method of breast nodules as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0111] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this disclosure as described above, which are not provided in detail for the sake of brevity.

[0112] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0113] This disclosure is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for three-dimensional reconstruction of breast nodules, characterized in that, include: Step S1: Acquire multiple frames of breast ultrasound images and preprocess each frame of breast ultrasound image, including filtering and noise reduction and image enhancement. Step S2: Use a lightweight convolutional neural network structure to extract features from the preprocessed breast ultrasound image to obtain a feature map; Step S3: The extracted feature map is segmented using the segmentation module. During the segmentation process, a loss function is used to measure the difference between the segmentation result and the true label. The parameters of the segmentation module are continuously adjusted based on the difference calculated by the loss function through the backpropagation algorithm. Step S4: Perform SGBM stereo matching on the segmented images to obtain disparity information between images, and perform adaptive curve correction on the disparity information to realize three-dimensional reconstruction of breast nodules; Step S5: Establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation in step S4. The dynamic calibration mechanism includes periodically recalibrating the speckle characteristics of the equipment used to acquire breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters. The adaptive adjustment algorithm includes adjustment based on matching conditions and adjustment based on reconstruction results. Adaptive curve correction methods include: A simplified speckle model was used to approximate the calculation of the SGBM curve. By analyzing the speckle characteristics of a large number of breast ultrasound images, a simplified speckle model was established. For each region in the image, the SGBM curve of that region was calculated based on the center coordinates (x,y) of that region and the simplified speckle model. To simplify the coherence coefficient adjustment, a threshold is set when calculating the coherence coefficient. When the change in the coherence coefficient is within the set threshold range, no further adjustment or resampling is performed. The result from the previous round is directly used or a simple linear interpolation is performed. When the change in the coherence coefficient is outside the set threshold range, the adjustment and resampling are performed according to the conventional method.

2. The method for three-dimensional reconstruction of breast nodules according to claim 1, characterized in that: In step S1, the filtering and denoising process for each frame of breast ultrasound image includes adaptive median filtering and Gaussian filtering. The adaptive median filtering method includes: An initial window is preset. It is determined whether the maximum difference in pixel grayscale values ​​within the initial window centered on the current pixel exceeds a preset threshold. If not, the initial window is used to perform median filtering. If so, the window size is gradually increased until the maximum difference in pixel grayscale values ​​within the window does not exceed the preset threshold.

3. The method for three-dimensional reconstruction of breast nodules according to claim 2, characterized in that: In step S1, image enhancement processing for each frame of breast ultrasound image includes histogram equalization based on local contrast and adaptive contrast stretching. The histogram equalization based on local contrast method involves dividing the breast ultrasound image into several non-overlapping sub-regions, calculating the gray-level histogram for each sub-region and its cumulative distribution function, and assigning a higher weight to the cumulative distribution function of the current sub-region and a lower weight to the cumulative distribution functions of adjacent sub-regions using a weighted average method to calculate the final cumulative distribution function value used for mapping, thereby remapping the gray-level values ​​of the pixels within that sub-region. The adaptive contrast stretching method involves calculating the statistical characteristics of the gray-level histogram of the breast ultrasound image, determining whether the image's gray-level distribution is biased towards dark areas, bright areas, or a uniform distribution based on these statistical characteristics, and automatically determining the initial gray-level value range and the new gray-level value range to which the image is stretched if the distribution is biased towards dark areas.

4. The method for three-dimensional reconstruction of breast nodules according to any one of claims 1-3, characterized in that: In step S2, the lightweight convolutional neural network structure is MobileNet.

5. The method for three-dimensional reconstruction of breast nodules according to claim 4, characterized in that: In step S2, the lightweight convolutional neural network structure extracts image features through depthwise separable convolution operations, the method including: Each channel of the input breast ultrasound image is individually depthwise convolved, and the output is a set of feature maps with the same number of channels as the input image. Perform a 1×1 convolution on each pixel of the depthwise convolution output to adjust the number of channels.

6. The method for three-dimensional reconstruction of breast nodules according to claim 4, characterized in that: In step S4, the method for performing SGBM stereo matching on the segmented images to obtain disparity information between images includes: Extract key feature points from the segmented image, which includes the current frame image and the next frame image; The key feature points extracted from the current frame image are matched with the key feature points extracted from the subsequent frame image. During the matching process, feature descriptors are used to describe the key feature points, and the matching relationship between the key feature points is determined by calculating the distance between the feature descriptors. If the distance between the feature descriptors of two key feature points is less than a preset distance threshold, then the two key feature points match; otherwise, the two key feature points do not match. After extracting and matching key feature points, the matching cost is calculated by sampling. For each pair of matched key feature points, the absolute difference is calculated every few pixels and then summed to approximate the matching cost. The disparity information is calculated based on the positional difference of the matched region in the current frame and the next frame.

7. The method for three-dimensional reconstruction of breast nodules according to claim 6, characterized in that: In step S5, the method for establishing a dynamic calibration mechanism includes: collection Frame calibration sample images, for each frame calibration sample image For each pixel Select a local area centered on the target region; Calculate the speckle statistical features within the local region to obtain speckle feature values, and record the speckle feature values ​​of each pixel in each frame of the calibration sample image to form a speckle feature vector for each frame of the calibration sample image. ; Calculate the Euclidean distance between speckle feature vectors of calibration sample images collected at different time points, and determine whether the Euclidean distance between speckle feature vectors is greater than the set distance threshold. If it is, recalibration is required; otherwise, recalibration is not required. Calculate the average speckle contrast of all multi-frame calibration sample images that require recalibration. Average size of speckle Average correlation coefficient with speckle and the synthesized eigenvalues As a new speckle characteristic parameter for current breast ultrasound equipment; Based on the new speckle characteristic parameters, and combined with the original standard SGBM curve calculation model, a new standard SGBM curve adapted to the speckle characteristics of the current equipment is obtained.

8. The method for three-dimensional reconstruction of breast nodules according to claim 7, characterized in that, In step S5, the method for adjusting based on the matching situation includes: For each pair of matched current frame regions and subsequent frame regions, calculate the first evaluation index value for both, where the first evaluation index value includes the mean similarity value and the structural similarity index. The calculated first evaluation index value is compared with the set threshold. If the average similarity value is lower than the preset lower threshold, the adjustment range of the coherence coefficient in the adaptive curve correction process needs to be increased. If the structural similarity index exceeds the preset upper and lower thresholds, the resampling ratio of the coherence coefficient needs to be adjusted.

9. The method for three-dimensional reconstruction of breast nodules according to claim 7, characterized in that, In step S5, the method for adjusting based on the reconstruction effect includes: The second evaluation index value is calculated according to the preset evaluation index for the reconstruction effect. The second evaluation index value includes the error rate between the reconstructed nodule morphology and the actual anatomical structure, as well as the visual quality evaluation index of the reconstructed image. The calculated error rate is compared with the set threshold. If the error rate exceeds the preset upper limit threshold, the calculation method of the SGBM curve in the adaptive curve correction process is changed. If the visual quality evaluation index of the reconstructed image is not good, the entire adaptive curve correction process is redesigned.

10. A three-dimensional reconstruction system for breast nodules, characterized in that, include: The image preprocessing module is used to acquire multiple frames of breast ultrasound images and preprocess each frame of breast ultrasound image. The preprocessing includes filtering and noise reduction and image enhancement. The feature extraction module is used to extract features from preprocessed breast ultrasound images using a lightweight convolutional neural network structure to obtain feature maps. The segmentation module is used to segment the feature map obtained by feature extraction. During the segmentation process, a loss function is used to measure the difference between the segmentation result and the true label. The parameters of the segmentation module are continuously adjusted based on the difference calculated by the loss function through the backpropagation algorithm to obtain an image result that accurately segments the breast nodule region. The three-dimensional reconstruction module is used to perform SGBM stereo matching on images with accurately segmented breast nodule regions, while acquiring disparity information between images and performing adaptive curve correction on the disparity information to achieve three-dimensional reconstruction of breast nodules. The calibration and optimization control module is used to establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation. The dynamic calibration mechanism includes periodically recalibrating the speckle characteristics of the equipment used to acquire breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters. The adaptive adjustment algorithm includes adjustment based on matching conditions and adjustment based on reconstruction effect. Adaptive curve correction methods include: A simplified speckle model was used to approximate the calculation of the SGBM curve. By analyzing the speckle characteristics of a large number of breast ultrasound images, a simplified speckle model was established. For each region in the image, the SGBM curve of that region was calculated based on the center coordinates (x,y) of that region and the simplified speckle model. To simplify the coherence coefficient adjustment, a threshold is set when calculating the coherence coefficient. When the change in the coherence coefficient is within the set threshold range, no further adjustment or resampling is performed. The result from the previous round is directly used or a simple linear interpolation is performed. When the change in the coherence coefficient is outside the set threshold range, the adjustment and resampling are performed according to the conventional method.

Images