Breast nodule three-dimensional reconstruction method and system

Abstract

The invention provides a breast nodule three-dimensional reconstruction method and system, and the method comprises the steps: obtaining a plurality of frames of breast ultrasonic images, and carrying out the preprocessing, including filtering and denoising processing and image enhancement processing; performing feature extraction on the image by adopting a lightweight convolutional neural network structure to obtain a feature map; carrying out segmentation processing on the feature map by utilizing a segmentation module; sGBM stereo matching operation is carried out on the segmented images, parallax information between the images is obtained, adaptive curve correction is carried out on the parallax information, and three-dimensional reconstruction of breast nodules is realized; and establishing a dynamic calibration mechanism and designing an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation. Through image preprocessing, algorithm optimization, dynamic calibration and adaptive adjustment, the problems that in the prior art, the image quality is high in requirement and prone to being affected, the computing resource consumption is large, and the three-dimensional reconstruction precision is low are solved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of medical image processing, and in particular to a method and system for three-dimensional reconstruction of breast nodules. Background Art

[0002] With the continuous development of medical imaging technology, breast ultrasound examination has been widely used in the field of breast disease screening due to its convenience and non-invasiveness, and plays an extremely important role in the clinical diagnosis process. In order to more accurately evaluate key information such as the shape, size and location of breast nodules, breast nodule 3D reconstruction technology has emerged.

[0003] At present, the existing 3D reconstruction technology of breast nodules is usually achieved by relying on a series of image processing algorithms and related models. Specifically, in the image acquisition stage, conventional breast ultrasound equipment is mainly used to obtain multiple frames of 2D breast ultrasound images, and these acquired images constitute the basic data source for subsequent processing work. In the feature extraction stage, in most cases, traditional convolutional neural network (CNN) architectures such as ResNet are used as feature extractors. These networks extract features from the input breast ultrasound images through multiple convolutional layers and pooling layers, aiming to capture the relevant feature information of breast tissue and nodules in the image, such as edges, textures and other features. The next step is the segmentation step. At this stage, the common practice is either to use traditional segmentation methods such as threshold segmentation and region growing, or to use deep learning network architectures such as UNet to segment the breast nodule area. These methods distinguish the nodule area in the breast ultrasound image from the surrounding breast tissue, providing relatively accurate nodule location and contour information for subsequent 3D reconstruction. Finally, in the three-dimensional reconstruction stage, most existing technologies use block matching-based stereo vision algorithms, such as the Semi-Global Block Matching (SGBM) algorithm, which obtains the disparity information between images by matching and analyzing adjacent frame images, and then combines relevant parameters such as camera calibration to achieve three-dimensional reconstruction of breast nodules, so as to present the morphology and positional relationship of breast nodules in three-dimensional space.

[0004] Although the existing three-dimensional reconstruction technology of breast nodules has made certain progress, there are still many problems. On the one hand, the image quality requirements are high and easily affected. Breast ultrasound images have inherent defects such as speckle noise due to the imaging principle, and the imaging quality of different ultrasound equipment is also different. The existing technology lacks a systematic and targeted preprocessing link before processing. Although some methods may perform simple grayscale adjustment or use basic filtering methods such as mean filtering, these operations have limited effectiveness in removing speckle noise and improving image contrast, and it is difficult to fully meet the needs of subsequent complex image processing algorithms for high-quality images; on the other hand, the computing resources are consumed a lot. Traditional algorithms used in feature extraction, segmentation and three-dimensional reconstruction, such as convolutional neural networks such as ResNet and SGBM stereo matching algorithms, involve a large number of matrix operations and complex computing logic, consume a lot of CPU time and memory when processing large-scale image data, and will be slow or even unable to run on devices with limited hardware resources, limiting its application in a wider range of scenarios such as primary medical units; in addition, the pre-calibrated standard SGBM curve relied on for three-dimensional reconstruction based on the SGBM algorithm will not match the speckle characteristics of the ultrasound equipment due to changes in usage time, environment, individual differences, etc., making it difficult to ensure the accuracy and stability of the three-dimensional reconstruction results.

[0005] Therefore, it is necessary to improve and innovate the existing technology to overcome these defects and improve the performance and practicality of breast nodule three-dimensional reconstruction technology. Summary of the invention

[0006] In view of this, the purpose of the present disclosure is to overcome the problems of the prior art in terms of high and easily affected image quality, large consumption of computing resources, and low three-dimensional reconstruction accuracy, and to provide a method and system for three-dimensional reconstruction of breast nodules.

[0007] Based on the above objectives, the present disclosure provides a method for three-dimensional reconstruction of breast nodules, comprising:

[0008] Step S1: acquiring multiple frames of breast ultrasound images, and preprocessing each frame of the breast ultrasound image, wherein the preprocessing includes filtering, denoising and image enhancement;

[0009] Step S2: using a lightweight convolutional neural network structure to extract features from the preprocessed breast ultrasound image to obtain a feature map;

[0010] Step S3: Use the segmentation module to perform segmentation processing on the extracted feature map. During the segmentation process, the loss function is used to measure the difference between the segmentation result and the true label. The parameters of the segmentation module are continuously adjusted according to the difference calculated by the loss function through the back propagation algorithm;

[0011] Step S4: performing SGBM stereo matching operation on the segmented images to obtain disparity information between the images, and performing adaptive curve correction on the disparity information to achieve three-dimensional reconstruction of breast nodules;

[0012] Step S5: Establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation in step S4, wherein the dynamic calibration mechanism includes regularly recalibrating the speckle characteristics of the device used to acquire breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters, and designing the adaptive adjustment algorithm includes adjustment based on matching conditions and adjustment based on reconstruction effects.

[0013] In one embodiment, in step S1, filtering and denoising each frame of breast ultrasound image includes adaptive median filtering and Gaussian filtering, wherein the method of adaptive median filtering includes:

[0014] An initial window is preset to determine whether the maximum difference of the pixel grayscale values ​​within the initial window centered on the current pixel point exceeds a preset threshold. If not, the initial window is used to perform a median filtering operation. If so, the window size is gradually increased until the maximum difference of the pixel grayscale values ​​within the window does not exceed the preset threshold.

[0015] In one of the embodiments, in step S1, image enhancement processing is performed on each frame of breast ultrasound image, including histogram equalization processing based on local contrast and adaptive contrast stretching processing, wherein the method of histogram equalization processing based on local contrast includes dividing the breast ultrasound image into a plurality of non-overlapping sub-regions, respectively counting the grayscale histogram of each sub-region and calculating the cumulative distribution function, and using a weighted average method to give a higher weight to the cumulative distribution function of the current sub-region and a lower weight to the cumulative distribution function of the adjacent sub-region, so as to calculate the final cumulative distribution function value for mapping, and then remap the grayscale values ​​of the pixels in the sub-region; the method of adaptive contrast stretching processing includes calculating the statistical features of the grayscale histogram of the breast ultrasound image, judging whether the grayscale distribution of the image is biased towards dark distribution, bright distribution or uniform distribution based on the statistical features, and automatically determining the stretching based on the judgment result. If it is judged to be biased towards dark distribution, the starting grayscale value range of the stretching and the new grayscale value range to be stretched are automatically determined.

[0016] In one of the embodiments, in step S2, the lightweight convolutional neural network structure is MobileNet.

[0017] In one embodiment, in step S2, the lightweight convolutional neural network structure extracts image features through a depth-separable convolution operation, and the method includes:

[0018] Perform a deep convolution on each channel of the input breast ultrasound image and output a set of feature maps with the same number of input image channels;

[0019] A 1×1 convolution is performed on each pixel of the depthwise convolution output to adjust the number of channels.

[0020] In one embodiment, in step S4, the method of performing SGBM stereo matching operation on the segmented images and obtaining disparity information between the images includes:

[0021] Extract key feature points from the segmented image, wherein the image includes a current frame image and a subsequent frame image;

[0022] Match the key feature points extracted from the current frame image with the key feature points extracted from the next frame image. During the matching, feature descriptors are used to describe the key feature points, and the matching relationship between the key feature points is determined by calculating the distance between the feature descriptors. If the distance between the feature descriptors of the two key feature points is less than a preset distance threshold, the two key feature points match, otherwise, the two key feature points do not match;

[0023] After completing the extraction and matching of key feature points, the matching cost is calculated by sampling. For each pair of matching key feature points, the absolute difference is calculated every few pixels and summed up to approximate the matching cost. The disparity information is calculated based on the position difference between the matched area in the current frame image and the next frame image.

[0024] In one embodiment, in step S5, the method for establishing a dynamic calibration mechanism includes:

[0025] Collect Q frames of calibration sample images, and for each frame of calibration sample image I i (i=1,2,3…,Q), select a local area with each pixel point P(x,y) as the center;

[0026] Calculate the speckle statistical features in the local area to obtain the speckle feature value, and record the speckle feature value of each pixel in each frame of the calibration sample image to form a speckle feature vector V for each frame of the calibration sample image. i (i=1,2,3…,Q);

[0027] Calculate the Euclidean distance of the speckle feature vectors between the calibration sample images collected at different time points, and determine whether the Euclidean distance between the speckle feature vectors is greater than the set distance threshold. If so, recalibration is required, otherwise, recalibration is not required;

[0028] Calculate the average speckle contrast of all the calibration sample images that need to be recalibrated. Average size of speckle The average correlation coefficient of the speckle correlation And the integrated eigenvalues As a new speckle characteristic parameter for current breast ultrasound equipment;

[0029] According to the new speckle characteristic parameters, the original standard SGBM curve calculation model is combined to recalculate and obtain the standard SGBM curve that adapts to the speckle characteristics of the current equipment.

[0030] In one embodiment, in step S5, the method for adjusting based on the matching situation includes:

[0031] For each pair of matched current frame regions and subsequent frame regions, calculating a first evaluation index value of the two, wherein the first evaluation index value includes a similarity mean and a structural similarity index;

[0032] The calculated first evaluation index value is compared with the set threshold. If the similarity mean is lower than the preset lower limit threshold, it is necessary to increase the adjustment range of the coherence coefficient during the adaptive curve correction process; if the structural similarity index exceeds the preset upper and lower limit thresholds, it is necessary to adjust the resampling ratio of the coherence coefficient.

[0033] In one embodiment, in step S5, the method based on reconstruction effect adjustment includes:

[0034] Calculating a second evaluation index value according to a preset index for evaluating the reconstruction effect, wherein the second evaluation index value includes an error rate between the reconstructed nodule morphology and the actual anatomical structure and a visual quality evaluation index of the reconstructed image;

[0035] The calculated error rate is compared with the set threshold. If the error rate exceeds the preset upper threshold, the calculation method of the SGBM curve in the adaptive curve correction process is changed. If the visual quality evaluation index of the reconstructed image is not good, the entire adaptive curve correction process is redesigned.

[0036] Based on the same inventive concept, the present disclosure also provides a breast nodule three-dimensional reconstruction system, comprising:

[0037] An image preprocessing module is used to obtain multiple frames of breast ultrasound images and preprocess each frame of breast ultrasound image, wherein the preprocessing includes filtering, denoising and image enhancement.

[0038] A feature extraction module is used to use the preprocessed breast ultrasound image as input, and use a lightweight convolutional neural network structure to extract features to obtain a feature map;

[0039] The segmentation module is used to segment the feature map obtained by feature extraction using the segmentation module. During the segmentation process, the difference between the segmentation result and the true label is measured using the loss function. The parameters of the segmentation module are continuously adjusted according to the difference calculated by the loss function through the back propagation algorithm to obtain an image result of accurately segmenting the breast nodule area.

[0040] A three-dimensional reconstruction module is used to perform SGBM stereo matching operation on the image with the breast nodule area accurately segmented, obtain disparity information between images, and perform adaptive curve correction on the disparity information to achieve three-dimensional reconstruction of the breast nodule;

[0041] A calibration and optimization control module is used to establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation, wherein the dynamic calibration mechanism includes regularly recalibrating the speckle characteristics of the equipment used to obtain breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters, and designing an adaptive adjustment algorithm including adjustment based on matching conditions and adjustment based on reconstruction effects.

[0042] Based on the same inventive concept, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for three-dimensional reconstruction of breast nodules described in any of the above embodiments is implemented.

[0043] Based on the same inventive concept, the present disclosure also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the breast nodule three-dimensional reconstruction method described in any of the above embodiments.

[0044] From the above, it can be seen that the present invention provides a method and system for three-dimensional reconstruction of breast nodules, which have many significant beneficial effects in the three-dimensional reconstruction of breast nodules. First, through the carefully designed image preprocessing link, the effective filtering denoising and image enhancement technology are used, which not only effectively reduces the impact of noise on subsequent processing, but also greatly improves the clarity and contrast of the image, and provides better input data for feature extraction, segmentation and three-dimensional reconstruction, thereby ensuring the accuracy of processing in each link; secondly, the optimization algorithm and hardware acceleration measures adopted play an important role. On the one hand, the calculation complexity is reduced by the optimization algorithm; on the other hand, the operation efficiency is improved with the help of hardware acceleration equipment, so that when processing a large amount of breast ultrasound image data or running on a device with limited hardware resources, it can complete the three-dimensional reconstruction task of breast nodules more quickly and stably; furthermore, the construction of a dynamic calibration mechanism and the design of an adaptive adjustment algorithm specifically solve the limitations of the adaptive curve correction method, and by adjusting the correction parameters and methods in real time, the correction effect and the accuracy of the three-dimensional reconstruction of breast nodules are effectively improved, ensuring the accuracy and reliability of the three-dimensional reconstruction results, and providing strong support for the diagnosis and three-dimensional visualization of breast nodules, showing a high practical value and promotion prospects as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 A flowchart of a method for three-dimensional reconstruction of breast nodules according to an embodiment;

[0047] Figure 2 This is a comparison chart of the treatment efficiency between the improved group and the conventional group;

[0048] Figure 3 This is a comparison chart of the reconstruction accuracy between the improved group and the conventional group;

[0049] Figure 4 This is a comparison chart of how the improvement group and the conventional group coped with ultrasound equipment failures;

[0050] Figure 5 This is a comparison chart of the improvement group and the conventional group in coping with the sudden change in the imaging environment;

[0051] Figure 6 The module structure diagram of a breast nodule three-dimensional reconstruction system according to one embodiment. DETAILED DESCRIPTION

[0052] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0053] In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of some known functions and known components.

[0054] like Figure 1 A method flow of a breast nodule three-dimensional reconstruction method in one embodiment shown includes steps S1 to S5:

[0055] Step S1: acquiring multiple frames of breast ultrasound images, and preprocessing each frame of the breast ultrasound image, wherein the preprocessing includes filtering, denoising and image enhancement;

[0056] Step S2: using a lightweight convolutional neural network structure to extract features from the preprocessed breast ultrasound image to obtain a feature map;

[0057] Step S3: Use the segmentation module to perform segmentation processing on the extracted feature map. During the segmentation process, the loss function is used to measure the difference between the segmentation result and the true label. The parameters of the segmentation module are continuously adjusted according to the difference calculated by the loss function through the back propagation algorithm;

[0058] Step S4: performing SGBM stereo matching operation on the segmented images to obtain disparity information between images, and performing adaptive curve correction on the disparity information to achieve three-dimensional reconstruction of breast nodules;

[0059] Step S5: Establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation in step S4, wherein the dynamic calibration mechanism includes regularly recalibrating the speckle characteristics of the equipment used to acquire breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters, and designing an adaptive adjustment algorithm including adjustment based on matching conditions and adjustment based on reconstruction effects.

[0060] The present invention discloses a method for 3D reconstruction of breast nodules, which has many significant beneficial effects in the 3D reconstruction of breast nodules. Firstly, through a carefully designed image preprocessing link and the use of effective filtering denoising and image enhancement technology, not only the influence of noise on subsequent processing is effectively reduced, but also the clarity and contrast of the image are greatly improved, and better input data is provided for feature extraction, segmentation and 3D reconstruction, thereby ensuring the accuracy of processing in each link; secondly, the optimization algorithm and hardware acceleration measures adopted play an important role. On the one hand, the calculation complexity is reduced by the optimization algorithm; on the other hand, the operation efficiency is improved with the help of hardware acceleration equipment, so that when processing a large amount of breast ultrasound image data or running on a device with limited hardware resources, the 3D reconstruction task of breast nodules can be completed more quickly and stably; furthermore, the construction of a dynamic calibration mechanism and the design of an adaptive adjustment algorithm specifically solve the limitations of the adaptive curve correction method, and the correction effect and the accuracy of 3D reconstruction of breast nodules are effectively improved by adjusting the correction parameters and methods in real time, ensuring the accuracy and reliability of the 3D reconstruction results, providing strong support for the diagnosis and 3D visualization of breast nodules, and showing a high practical value and promotion prospect as a whole.

[0061] As a preferred implementation, in step S1, filtering and denoising each frame of breast ultrasound image includes adaptive median filtering. Specifically, the method of adaptive median filtering includes: whether the maximum difference of the pixel grayscale value in a small initial window (such as 3×3) centered on the current pixel point exceeds a preset threshold. If it does not exceed the threshold, it means that the area is relatively smooth, and the small window can be directly used for median filtering; if it exceeds the threshold, it means that there may be noise or rich details in the area, and the filter window needs to be expanded. At this time, the window size is gradually increased (such as increasing the side length of 2 pixels each time, from 3×3 to 5×5, 7×7, etc.), and the maximum difference of the pixel grayscale value in the new window is judged again whether it meets the conditions, until a suitable window size is found so that the difference is within the threshold range, and then the median filtering operation is performed. The adaptive median filtering of this implementation can remove noise more accurately according to the actual situation of different areas of the image, while retaining the edge and detail information of the image, it can better adapt to the local complexity of the image, improve the uniformity of the denoising effect and the overall image quality.

[0062] Furthermore, in step S1, the filtering and denoising of each frame of breast ultrasound image also includes Gaussian filtering. Gaussian filtering is based on the weighted average principle of Gaussian function, which has a good effect on removing Gaussian noise in the image and can retain image edge and other detail information to a certain extent. The detailed process is as follows: First, a Gaussian kernel matrix needs to be generated according to the set parameters. The size of the Gaussian kernel matrix is ​​usually a square with odd sides, such as 3×3, 5×5, etc., and its element values ​​are calculated according to the Gaussian function. Among them, the expression of the Gaussian function is: Where (x, y) represents the coordinate position in the Gaussian kernel matrix, and σ is the standard deviation, which determines the shape of the Gaussian function, that is, the "width" of the Gaussian kernel. For each pixel to be processed in the image, a neighborhood with the same size as the Gaussian kernel is selected with the pixel as the center. For example, for the pixel (x, y), if the above-mentioned 3×3 Gaussian kernel is used, 9 pixel values ​​around (x, y) are selected (the coordinate range is (x-1, y-1) to (x+1, y+1)). Multiply each pixel value in the neighborhood with the element value of the corresponding position in the Gaussian kernel matrix, and then add these products. The sum obtained is the new value of the pixel (x, y) after Gaussian filtering. By performing such an operation on each pixel in the image, this embodiment can filter the image according to the weighted average method of the Gaussian function to remove Gaussian noise and make the image smoother. At the same time, because it is a weighted average, it can also relatively better retain the edge and other detail information of the image.

[0063] As a preferred implementation, in step S1, the image enhancement processing of each frame of breast ultrasound image includes histogram equalization processing based on local contrast. Specifically, the method of using histogram equalization processing based on local contrast includes: first, dividing the breast ultrasound image into a number of non-overlapping sub-regions (such as 8×8 or 16×16 small blocks), for each sub-region, respectively counting the grayscale histogram of each sub-region and calculating the cumulative distribution function CDF; then, when remapping the grayscale value, comprehensively consider the CDF information of the current sub-region and the CDF information of the adjacent sub-region. For example, a weighted average method can be used to give a higher weight (such as 0.7) to the CDF of the current sub-region and a lower weight (such as 0.3) to the CDF of the adjacent sub-region to calculate the final CDF value for mapping, and then remap the grayscale value of the pixel points in the sub-region. This implementation can better consider the contrast requirements of the local area of ​​the image, avoid the local contrast imbalance problem that may occur in the traditional method, so that each local area of ​​the image can obtain a more appropriate contrast enhancement, thereby improving the overall clarity and visibility of the image.

[0064] Further, in step S1, the image enhancement processing for each frame of breast ultrasound image includes adaptive contrast stretching processing. Specifically, the method of using adaptive contrast stretching processing includes: first, calculating some statistical features of the grayscale histogram of the image, such as the mean, standard deviation, skewness and kurtosis of the grayscale value; then, based on these statistical features, it can be judged whether the grayscale distribution of the image is biased towards the dark part, the bright part or relatively uniform. If the grayscale distribution of the image is biased towards the dark part (for example, the grayscale is greater than a certain threshold and the mean is small), the starting grayscale value range of the stretch is automatically determined to be from the minimum value of the image grayscale value to an intermediate value calculated according to the standard deviation (such as the mean + standard deviation), and the new range stretched can be set from 0 to a certain intermediate value calculated according to the maximum and minimum values ​​of the image grayscale value (such as (maximum value-minimum value) / 2). Similarly, if the grayscale distribution of the image is biased towards the bright part (the grayscale is less than a certain threshold and the mean is large), the starting grayscale value range of the stretch and the new range stretched are also determined according to the corresponding rules based on the statistical features. If the grayscale distribution of the image is relatively uniform, it can be stretched according to the preset default range. The adaptive contrast stretching of this embodiment does not require manual intervention, and can automatically determine the appropriate stretching parameters according to the grayscale histogram characteristics of the image itself, thereby improving the automation and flexibility of the image enhancement process, so that different types of ultrasound images can obtain more appropriate contrast enhancement, further improving image quality.

[0065] The image preprocessing method adopted in this embodiment works together through a carefully designed combination method, producing a unique effect, effectively improving the image quality and having a very positive impact on the subsequent steps of three-dimensional reconstruction of breast nodules. Specifically, in terms of noise removal, it can effectively remove various types of noise commonly seen in ultrasound images, making the image purer; in terms of contrast enhancement, through the successive coordination of histogram equalization and contrast stretching, it can achieve a more significant and targeted contrast enhancement than the single grayscale adjustment method in the prior art, making the breast nodules more prominent in the image; in terms of detail presentation, adaptive median filtering and Gaussian filtering retain the details of the image while removing noise, and subsequent operations such as contrast stretching further highlight the details, so that in the subsequent steps of feature extraction, segmentation and three-dimensional reconstruction, the model can more accurately capture the characteristics of breast nodules, improve the accuracy of segmentation, and then improve the accuracy of three-dimensional reconstruction, and can more realistically present the true shape and size of breast nodules.

[0066] As a preferred implementation, in step S2, the lightweight convolutional neural network structure used in this implementation is MobileNet, and MobileNetV2 can be preferably selected, which uses a deep separable convolution operation to extract image features. First, each channel of the input breast ultrasound image is individually deep convolved to output a set of feature maps with the same number of channels as the input image; then a 1×1 convolution is performed on each pixel point of the deep convolution output to adjust the number of channels. Compared with the traditional convolution operation, the amount of calculation is greatly reduced, especially when the number of channels and the number of output channels are large, the reduction in the amount of calculation is more obvious. Under the premise of ensuring that effective image features can be extracted, this implementation reduces the computational complexity of the feature extraction stage, thereby reducing the consumption of computing resources. Moreover, in terms of hardware, hardware acceleration devices such as graphics processing units (GPUs) and field programmable gate arrays (FPGAs) can be used to accelerate the entire processing process and improve operating efficiency by using their parallel computing capabilities.

[0067] As a preferred implementation, in step S3, the extracted features can be segmented using a network architecture such as UNet or a traditional segmentation method (such as threshold segmentation, region growing, etc.) to segment the breast ultrasound image into different regions, with the focus on accurately segmenting the region where the breast nodules are located. During the segmentation process, a loss function (such as a combination of DiceLoss and BCELoss) is used to measure the difference between the segmentation result and the true label, and the parameters of the segmentation module are continuously optimized through the back propagation algorithm to accurately segment the region where the breast nodules are located.

[0068] As a preferred implementation, in step S4, firstly, key feature points are extracted from the segmented images (current frame and subsequent frame images); the key feature points extracted from the current frame image are matched with the key feature points extracted from the subsequent frame image. During the matching, feature descriptors (such as SIFT feature descriptors) are used to describe the key feature points, and the matching relationship between the key feature points is determined by calculating the distance between the feature descriptors (such as Euclidean distance). If the distance between the feature descriptors of the two key feature points is less than a preset distance threshold, the two key feature points match, otherwise, the two key feature points do not match; after the extraction and matching of the key feature points are completed, the matching cost is calculated by sampling. For each pair of matching key feature points, the absolute difference is calculated and summed every few pixels to approximately calculate the matching cost. At the same time, the disparity information is calculated based on the position difference of the matching area in the current frame image and the subsequent frame image. This implementation method can greatly reduce the number of elements that need to be matched through a feature-based stereo matching method, thereby reducing the amount of calculation.

[0069] Specifically, the implementation process of the above feature-based stereo matching algorithm is as follows:

[0070] 1. Key feature point extraction: Input the segmented current frame image I curr and the next frame image I next And the preset feature extraction parameters (including corner response threshold T response , neighborhood size N size etc.); for the current frame image I curr , using the selected feature extraction method (such as Harris corner point extraction algorithm or SIFT feature point extraction algorithm), taking each pixel (x, y) as the center, according to the neighborhood size N size Determine the corresponding neighborhood area; calculate the characteristic response function value of the pixel point in its neighborhood area. If the calculated characteristic response function value R(x, y) is greater than the corner point response threshold T response , then the pixel point (x, y) is marked as a potential key feature point and stored in the potential key feature point set P of the current frame image. curr Similarly, for the subsequent frame image I next Repeat the above steps to obtain the potential key feature point set P of the next frame image. next .

[0071] 2. Feature descriptor generation: For each potential key feature point p of the current frame image curr ∈P curr , using the selected feature descriptor generation method (such as SIFT feature descriptor generation method) to generate its corresponding feature descriptor D curr (p curr ). This process involves analyzing and encoding the image information of the area around the key feature point to generate a vector that can describe the characteristics of the feature point. Similarly, for each potential key feature point p in the subsequent frame image, next ∈P next , the above method is also used to generate the corresponding feature descriptor D next (p next ).

[0072] 3. Feature point matching: Input the feature descriptor set D of the current frame image curr ={D curr (p curr )|p curr ∈P curr} and the feature descriptor set D of the subsequent frame image next ={D next (p next )|p next ∈P next}, preset distance threshold T distance ; Traverse the feature descriptor set D of the current frame image curr Each feature descriptor D in curr(p curr ): Calculate the feature descriptor and the feature descriptor set D of the next frame image next Each feature descriptor D next (p next ). Taking the Euclidean distance as an example, for two feature descriptor vectors D curr (p curr ) and D next (p next ), the Euclidean distance between them is calculated as Where n represents the dimension of the feature descriptor vector; if the calculated distance d(p curr , p nest ) is less than the distance threshold T distance , then it is considered that the feature point p in the current frame image curr and the feature point p in the next frame image nest If they are matched, the pair of matched feature points is recorded and stored in the matching feature point set A.

[0073] 4. Calculate the matching cost: For each pair of matching feature points (p curr , p nest ), a simplified matching cost calculation method can be used, such as sampling and calculating the sum of absolute differences (SAD). Assume that a region (such as a 5×5 region) is determined with each matching feature point as the center, and the absolute difference is calculated and summed every b pixels (for example, b=3) to obtain the matching cost of the pair of matching feature points.

[0074] Once the matching relationship between the various regions in different images is determined, the disparity can be calculated based on the position difference of the matching regions in the two images. By performing such disparity calculations on each matching region in the image, the disparity information of different points on the entire image can be obtained. These disparity information exist in the form of disparity values ​​for each matching point (or matching region). They reflect the positional offset of breast nodules in images of different viewing angles and are the basic data for subsequent three-dimensional reconstruction and correction of disparity information. However, these disparity information may have certain errors or be inaccurate, and further correction and optimization processing are required. The purpose of adaptive curve correction is to improve the accuracy of disparity information and the rationality of related parameters, so that the three-dimensional reconstruction based on these disparity information can more accurately present the morphology and positional relationship of breast nodules in three-dimensional space.

[0075] As a preferred embodiment, the adaptive curve correction of this embodiment firstly adopts a simplified speckle model to approximate the SGBM curve. By analyzing the speckle characteristics of a large number of breast ultrasound images, a simplified speckle model is established. For each area in the image, the SGBM curve of the area is calculated according to the center coordinates (x, y) of the area and the simplified speckle model. Then, the coherence coefficient adjustment is simplified. When calculating the coherence coefficient, a threshold is set. When the change of the coherence coefficient is within a certain threshold range, no subsequent adjustment and resampling and other complex operations are performed. The result of the previous round is directly used or simple linear interpolation and other processing are performed. For example, the threshold of the coherence coefficient change is set to 0.05. When the difference between the calculated coherence coefficient and the coherence coefficient of the previous round is less than 0.05, the coherence coefficient value of the previous round is directly used. If the difference is greater than or equal to 0.05, the adjustment and resampling and other operations are performed according to the conventional method. After the adaptive curve correction process, more accurate disparity information and related parameters are obtained, and the accuracy of three-dimensional reconstruction is further improved.

[0076] Specifically, the implementation process of the above adaptive curve correction is as follows:

[0077] 1. Simplified speckle model to establish SGBM curve: First, input a large number of breast ultrasound image data sets I dataset , the image size is M×N (M is the height, N is the width), traverse each image I in the data set i ∈I dataset (i=1,2,…,|I dataset |), for each image, divide it into small non-overlapping regions, such as a region of size w×w. Assume that the image can be divided horizontally into Areas can be divided vertically into regions, for each small region R ij (i = 1, 2, ..., m; j = 1, 2, ..., n), calculate the statistics related to its speckle characteristics, such as calculating the standard deviation σ of the pixel gray value in the area ij With mean μ ij , the speckle contrast is The edge of the speckle is detected by edge detection algorithms (such as Canny edge detection), and the average diameter or area is calculated based on the area surrounded by the edge as the speckle size S. ij Then a quadratic function is used to approximate the distribution of speckles, and the quadratic function is assumed to be f(x, y) = ax 2 +by 2+cxy+dx+ey+f, where (x, y) is the coordinate in the area, a, b, c, d, e, f are the coefficients to be determined, and the coefficients of the quadratic function are determined by fitting according to the speckle characteristic statistics of each small area using methods such as multivariate linear regression. For example, the center coordinates (x ij ,y ij )(in ) and the corresponding speckle characteristic statistics are used as sample data, substituted into the quadratic function form, and a linear equation system is constructed. The coefficients a, b, c, d, e, and f are determined by solving the linear equation system. Next, the simplified speckle model coefficients a, b, c, d, e, and f, and the center coordinates (x, y) of each area in the image are input. For each area in the image, its center coordinates (x, y) are substituted into the simplified speckle model quadratic function f(x, y), and the SGBM curve value SGBM of the area is calculated. curve (x, y).

[0078] 2. Simplified adjustment of coherence coefficient: First, input the image data after SGBM stereo matching (such as matching area information, etc.), the initial coherence coefficient value K 0 (An initial value can be set based on experience or related research), according to the initial setting and related calculation rules (the specific calculation method can be determined according to the actual SGBM stereo matching and subsequent processing method), calculate the first round of coherence coefficient value K 1 For example, if a calculation method based on pixel difference and region matching is used, the coherence coefficient is calculated by combining information such as SGBM curve value. Then input the coherence coefficient value K of the previous round prev , set the coherence coefficient change threshold ΔK, in each round of iteration (starting from the second round), recalculate the coherence coefficient value K according to the current image data and related calculation rules current , and calculate the difference ΔK between the current coherence coefficient value and the previous coherence coefficient value current =|K current -K prev |. Then determine ΔK current The relationship between ΔK and the magnitude of ΔK is as follows: current <ΔK, that is, the change of the coherence coefficient is within the set threshold range, and the coherence coefficient value of the previous round is directly used as the result of this round, that is, K current =K prev , if ΔK current ≥ΔK, complex operations such as adjustment and resampling are performed according to conventional methods. For example, it may involve re-evaluating and adjusting the coherence coefficient according to more detailed information of the image, disparity information, etc., recalculating the SGBM curve, etc., and then obtaining a new coherence coefficient value as the result of this round.

[0079] This embodiment utilizes the corrected SGBM curve values, coherence coefficient values ​​and other related parameters, combined with the disparity information obtained by the previous SGBM stereo matching and related information such as camera calibration (such as camera intrinsic parameters, extrinsic parameters, etc.), to perform three-dimensional reconstruction of breast nodules through a specific three-dimensional reconstruction algorithm (such as an algorithm based on the principle of triangulation).

[0080] As a preferred implementation, in step S5, the method for establishing a dynamic calibration mechanism specifically includes the following steps:

[0081] Step S511: Collect Q frames of calibration sample images. For each frame of calibration sample image I i (i=1,2,3…,Q), select a local area with each pixel point P(x,y) as the center (such as a 5×5 or 7×7 window centered on the pixel, assuming the window size is W×W), as an example, the speckle characteristics of the device used to obtain breast ultrasound images can be recalibrated regularly (such as every certain time period or after a certain number of imaging operations), select a set of standard objects with known geometric shapes and sizes (such as a specially made breast ultrasound calibration module with regular geometric shapes), place it in the imaging area of ​​the breast ultrasound device, and obtain multiple frames of breast ultrasound images as calibration samples according to the normal ultrasound imaging operation process;

[0082] Step S512: Calculate the speckle statistical features in the local area to obtain a speckle feature value, and record the speckle feature value of each pixel in each frame of the calibration sample image to form a speckle feature vector V for each frame of the calibration sample image. i (i=1,2,3…,Q);

[0083] Step S513: calculating the Euclidean distance between the speckle feature vectors of the calibration sample images collected at different time points, and determining whether the Euclidean distance between the speckle feature vectors is greater than a set distance threshold, if so, recalibration is required, if not, recalibration is not required;

[0084] Step S514: Calculate the average speckle contrast of all the multiple calibration sample images that need to be recalibrated Average size of speckle The average correlation coefficient of the speckle correlation And the integrated eigenvalues As a new speckle characteristic parameter for current breast ultrasound equipment;

[0085] Step S515: Recalculate according to the new speckle characteristic parameters in combination with the original standard SGBM curve calculation model to obtain a standard SGBM curve adapted to the speckle characteristics of the current device for subsequent adaptive curve correction operations. For example, in the original standard SGBM curve calculation model, the part related to speckle contrast is f(C), the part related to speckle size is g(S), and the part related to speckle correlation is h(R), then the updated standard SGBM curve calculation function can be set to

[0086] The above dynamic calibration mechanism brings many significant beneficial effects. First, it can improve the image quality and feature extraction accuracy. By regularly collecting, analyzing and determining new parameters for the speckle characteristics of the equipment used to obtain breast ultrasound images, it can accurately reflect the characteristics of the equipment, so that the ultrasound images obtained subsequently are more in line with the actual situation. At the same time, it optimizes the feature extraction process, allowing the feature extraction algorithm based on speckle characteristics to more accurately locate and extract effective features related to breast nodules. Secondly, this mechanism can improve the accuracy of stereo matching and three-dimensional reconstruction. Based on the new speckle characteristic parameters, the standard SGBM curve is updated to better adapt to SGBM stereo matching, improve the accuracy of disparity calculation and stereo matching accuracy, and then provide more reliable basic data for the three-dimensional reconstruction of breast nodules, more accurately present the morphology, position and other information of nodules in three-dimensional space, and help doctors make accurate diagnoses. Furthermore, it enhances the system's adaptability to equipment changes, can cope with equipment aging, environmental impacts and differences between different equipment, and promptly discover and adjust the changes in speckle characteristics caused by these factors, ensuring that the system can operate effectively under different equipment and equipment conditions. Finally, this mechanism also provides long-term and stable diagnostic support. Through continuous dynamic calibration, it ensures the stability and reliability of the image analysis basis, reduces diagnostic errors caused by changes in equipment or image parameters for long-term breast nodule diagnosis, and promotes the promotion of this technology in clinical applications, thereby providing patients with better breast nodule diagnosis services.

[0087] As a preferred implementation, in step S5, the method for adjusting based on the matching situation includes: first, calculating the first evaluation index value of each pair of matching current frame area and subsequent frame area, wherein the first evaluation index value includes the similarity mean and the structural similarity index SSIM; then, comparing the calculated first evaluation index value with the set threshold value, if the similarity mean is lower than the preset lower limit threshold value, it means that the matching effect is poor, and it is necessary to increase the adjustment amplitude of the coherence coefficient in the adaptive curve correction process, for example, assuming that the original coherence coefficient adjustment amplitude is s, it can be updated to s+Δs; if the structural similarity index SSIM exceeds the preset upper and lower limit thresholds, it means that the matching effect is not ideal, and it is necessary to adjust the resampling ratio of the coherence coefficient. For example, assuming that the original coherence coefficient resampling ratio is r, it can be updated to r+Δr or r-Δr according to the specific situation.

[0088] As another preferred implementation, in step S5, the method based on reconstruction effect adjustment includes: first, calculating a second evaluation index value according to a preset index for evaluating the reconstruction effect, wherein the second evaluation index value includes the error rate between the reconstructed nodule morphology and the actual anatomical structure and a visual quality evaluation index of the reconstructed image; then, comparing the calculated error rate (such as the nodule volume error rate) with the set threshold; if the error rate exceeds the preset upper limit threshold, changing the calculation method of the SGBM curve in the adaptive curve correction process; if the visual quality evaluation index of the reconstructed image is not good, such as the clarity index is lower than the preset lower limit threshold or the contrast index exceeds the preset upper and lower limit thresholds, etc., the entire adaptive curve correction process needs to be redesigned.

[0089] The above-mentioned adaptive adjustment algorithm has brought many significant beneficial effects. In terms of optimizing the matching effect and improving the accuracy of stereo matching, it accurately evaluates the matching effect by calculating evaluation indicators such as the mean similarity of pixels in the matching area and the structural similarity index, and then adjusts the adaptive curve correction related parameters in a targeted manner based on the comparison results between the evaluation indicators and the set threshold, thereby optimizing the matching effect and improving the SGBM stereo matching accuracy, laying a solid foundation for subsequent three-dimensional reconstruction. In terms of improving the quality of three-dimensional reconstruction and enhancing the accuracy of reconstruction results, after completing the three-dimensional reconstruction of breast nodules, it comprehensively measures the reconstruction effect by calculating the error rate between the volume, area, shape and actual anatomical structure of the nodules and the visual quality evaluation indicators of the reconstructed images, and flexibly adjusts the adaptive curve correction method based on the comparison results between the indicators and the threshold, effectively improving the reconstruction effect, making the reconstructed breast nodules closer to the actual anatomical structure and improving the visual quality of the image, providing doctors with accurate and clear diagnostic basis. At the same time, the algorithm also enhances adaptability and stability, can flexibly adjust according to different situations of the image in the matching and reconstruction stages, adapt to various changes, and ensures the stable operation of the algorithm through continuous monitoring and dynamic adjustment, providing stable and reliable technical support for the diagnosis of breast nodules. In addition, it has greatly contributed to the accuracy and reliability of clinical diagnosis. With improved stereo matching accuracy and three-dimensional reconstruction quality, doctors can obtain accurate nodule information based on more accurate and clear reconstructed images that are consistent with the actual anatomical structure, reduce misdiagnosis and missed diagnosis, and better meet patients' breast nodule diagnosis needs.

[0090] The following compares the breast nodule three-dimensional reconstruction method provided in this embodiment (hereinafter referred to as the "improved method") with the existing conventional method to demonstrate the advantages of the improved method in terms of processing efficiency, reconstruction accuracy, and adaptability to complex situations.

[0091] This experiment selected 100 patients with breast nodules from different medical institutions as experimental subjects. Each patient underwent multiple breast ultrasound examinations, and a total of 300 groups of breast ultrasound image data were obtained as experimental samples. These samples cover breast nodule images of different sizes, shapes, positions and collected by different imaging devices to ensure the universality and representativeness of the experimental results. The experimental samples were randomly divided into two groups, one group using the existing conventional method for three-dimensional reconstruction of breast nodules (hereinafter referred to as the "conventional group"), and the other group using the improved three-dimensional reconstruction method of breast nodules in this embodiment (hereinafter referred to as the "improved group"). Under the same experimental environment, the two groups were processed separately and the relevant experimental data were recorded.

[0092] 1. Comparison of processing efficiency: On the one hand, in the image preprocessing stage, the conventional group performed conventional operations such as filtering, denoising and image enhancement on each group of breast ultrasound images. Since the filtering algorithm of the existing method may be relatively complex and not optimized enough, it takes a lot of time to process image noise and enhance contrast, with an average time of about 30 minutes. The improved group adopted optimized filtering algorithms (such as adaptive median filtering) and more efficient image enhancement techniques (such as histogram equalization based on local contrast and adaptive contrast stretching), and the average image preprocessing time was about 15 minutes, which was half the time of the conventional group, greatly improving the preprocessing efficiency. On the other hand, after completing the image preprocessing, the conventional group performed three-dimensional reconstruction of breast nodules, including a series of operations such as SGBM stereo matching and adaptive curve correction. The average total calculation time for each group of images in the whole process was about 90 minutes. Among them, the SGBM stereo matching stage used a pixel-based matching method, which had a large amount of calculation and took a lot of time. The improved group used a feature-based stereo matching method instead of a pixel-based matching method, and combined it with simplified matching cost calculation, simplified SGBM curve calculation, and simplified coherence coefficient adjustment, which greatly reduced the total calculation time for 3D reconstruction. For each set of images, the average total calculation time was about 35 minutes, which was nearly 61.1% shorter than that of the conventional group. Figure 2 It can be clearly seen that the improved group is significantly better than the conventional group in terms of image preprocessing and total calculation time of 3D reconstruction, reflecting the significant advantages of the improved method in processing efficiency.

[0093] 2. Comparison of reconstruction accuracy: On the one hand, after completing the three-dimensional reconstruction of breast nodules, the conventional group compared the reconstructed nodule volume with the actual anatomical structure (which can be obtained through subsequent surgical pathology results or other more accurate measurement methods) to calculate the error rate between the reconstructed nodule volume and the actual nodule volume. After statistical analysis of all sample data, the average error rate of the nodule volume in the conventional group was about 18%, which indicates that the existing method has certain deviations in reconstructing the volume of breast nodules, which may affect the doctor's accurate judgment of the nodule size. In the improved group, due to the use of optimized image preprocessing, more accurate stereo matching, and adaptive adjustment algorithms, the error rate between the reconstructed nodule volume and the actual nodule volume in the improved group was significantly reduced. According to statistics, the average error rate of the nodule volume in the improved group was about 8%, which was about 55.6% lower than that in the conventional group. On the other hand, the conventional group calculated the similarity index between the reconstructed nodule shape and the actual nodule shape (which can be measured by methods such as Hausdorff distance). The average similarity index of the nodule shape in the conventional group was about 0.6, indicating that there is a certain difference between the nodule shape reconstructed by the existing method and the actual shape, and the restoration of the nodule morphology is not accurate enough. The improved group used an optimized 3D reconstruction algorithm, especially fine-tuning the parameters in the adaptive curve correction phase. The average similarity index of the reconstructed nodules in the improved group was about 0.85, which was about 41.7% higher than that in the conventional group. (Calculation method: (0.85-0.6) / 0.6×100%). Figure 3 It can be clearly seen that the improved group is superior to the conventional group in terms of nodule volume error rate and nodule shape similarity index, indicating that the improved breast nodule three-dimensional reconstruction method has significantly improved the reconstruction accuracy.

[0094] 3. Comparison of adaptability to complex situations: On the one hand, when the simulated ultrasound equipment fails and the speckle characteristics change greatly, the conventional group uses the existing method to deal with it. Due to the lack of a mechanism for quickly recalibrating the speckle characteristics, it takes a long time to restore normal 3D reconstruction after the equipment fails, about 6 hours on average, and the first 3D reconstruction accuracy error rate after recalibration is high, about 15% on average. In the improved method, the improved group provides an emergency solution for rapid recalibration of speckle characteristics for ultrasound equipment failure, shortens the recalibration cycle and adopts a simple speckle characteristic acquisition method. Under the same equipment failure simulation, the time required for the improved group to restore normal 3D reconstruction is about 2 hours on average, which is about 66.7% shorter than that of the conventional group. At the same time, the first 3D reconstruction accuracy error rate after recalibration is about 8% on average, which is about 46.7% lower than that of the conventional group. On the other hand, when the imaging environment of the conventional group changes suddenly (such as temperature and humidity changes, etc.) and affects the image matching effect, the existing method does not have a mechanism for adaptive adjustment based on environmental parameters. In the case of simulated environmental mutation, the error rate of 3D reconstruction accuracy after the environmental mutation in the conventional group changed greatly, averaging about 20%, and the average error rate of 3D reconstruction accuracy after final stabilization was about 15%. The improved group proposed a method based on adaptive adjustment of environmental parameters, which adjusts the parameters and methods of adaptive curve correction in real time according to environmental changes. In the same simulation of environmental mutation, the error rate of 3D reconstruction accuracy after the environmental mutation in the improved group changed by an average of about 5%, which was about 75% lower than that of the conventional group. At the same time, the average error rate of 3D reconstruction accuracy after final stabilization was about 8%, which was about 46.7% lower than that of the conventional group. Figure 4 and Figure 5 It can be clearly seen that the improved group is significantly better than the conventional group in dealing with complex situations such as ultrasound equipment failure and sudden changes in the imaging environment, whether in terms of the time required to resume normal three-dimensional reconstruction work, the first three-dimensional reconstruction accuracy error rate after recalibration, the amplitude of the three-dimensional reconstruction accuracy error rate after the sudden change in the environment, and the three-dimensional reconstruction accuracy error rate after final stabilization. This fully demonstrates the advantage of the improved breast nodule three-dimensional reconstruction method in its adaptability to complex situations.

[0095] Therefore, the three-dimensional reconstruction method of breast nodules provided in this embodiment is significantly superior to existing conventional methods in terms of processing efficiency, reconstruction accuracy, and adaptability to complex situations. These advantages can not only improve the work efficiency and quality of three-dimensional reconstruction of breast nodules, but also better cope with various complex situations that may arise in practical applications, and provide more powerful support for the diagnosis and treatment of breast nodules. It has important clinical application value and promotion prospects.

[0096] Correspondingly, such as Figure 6 As shown, the embodiment of the present disclosure also provides a breast nodule three-dimensional reconstruction system, including:

[0097] An image preprocessing module is used to obtain multiple frames of breast ultrasound images and preprocess each frame of breast ultrasound image, wherein the preprocessing includes filtering, denoising and image enhancement.

[0098] A feature extraction module is used to use the preprocessed breast ultrasound image as input, and use a lightweight convolutional neural network structure to extract features to obtain a feature map;

[0099] The segmentation module is used to segment the feature map obtained by feature extraction using the segmentation module. During the segmentation process, the difference between the segmentation result and the true label is measured using the loss function. The parameters of the segmentation module are continuously adjusted according to the difference calculated by the loss function through the back propagation algorithm to obtain an image result of accurately segmenting the breast nodule area.

[0100] A three-dimensional reconstruction module is used to perform SGBM stereo matching operation on the image with the breast nodule area accurately segmented, obtain disparity information between images, and perform adaptive curve correction on the disparity information to achieve three-dimensional reconstruction of the breast nodule;

[0101] A calibration and optimization control module is used to establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation, wherein the dynamic calibration mechanism includes regularly recalibrating the speckle characteristics of the equipment used to obtain breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters, and designing an adaptive adjustment algorithm including adjustment based on matching conditions and adjustment based on reconstruction effects.

[0102] The present invention discloses a method for 3D reconstruction of breast nodules, which has many significant beneficial effects in the 3D reconstruction of breast nodules. Firstly, through a carefully designed image preprocessing link and the use of effective filtering denoising and image enhancement technology, not only the influence of noise on subsequent processing is effectively reduced, but also the clarity and contrast of the image are greatly improved, and better input data is provided for feature extraction, segmentation and 3D reconstruction, thereby ensuring the accuracy of processing in each link; secondly, the optimization algorithm and hardware acceleration measures adopted play an important role. On the one hand, the calculation complexity is reduced by the optimization algorithm; on the other hand, the operation efficiency is improved with the help of hardware acceleration equipment, so that when processing a large amount of breast ultrasound image data or running on a device with limited hardware resources, the 3D reconstruction task of breast nodules can be completed more quickly and stably; furthermore, the construction of a dynamic calibration mechanism and the design of an adaptive adjustment algorithm specifically solve the limitations of the adaptive curve correction method, and the correction effect and the accuracy of 3D reconstruction of breast nodules are effectively improved by adjusting the correction parameters and methods in real time, ensuring the accuracy and reliability of the 3D reconstruction results, providing strong support for the diagnosis and 3D visualization of breast nodules, and showing a high practical value and promotion prospect as a whole.

[0103] The system of the above-mentioned embodiment is used to implement the corresponding breast nodule three-dimensional reconstruction method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0104] It should be noted that the drawings of the embodiments of the present disclosure only involve structures related to the embodiments of the present disclosure, and other structures may refer to general designs.

[0105] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0106] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0107] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the communication method based on asymmetric encryption described in any of the above embodiments is implemented.

[0108] The electronic device of this embodiment is used to implement the corresponding breast nodule three-dimensional reconstruction method in any of the aforementioned embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0109] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the breast nodule three-dimensional reconstruction method as described in any of the above embodiments.

[0110] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the three-dimensional reconstruction method of breast nodules as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0111] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0112] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0113] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for three-dimensional reconstruction of breast nodules, characterized in that: include: Step S1: acquiring multiple frames of breast ultrasound images, and preprocessing each frame of the breast ultrasound image, wherein the preprocessing includes filtering, denoising and image enhancement; Step S2: using a lightweight convolutional neural network structure to extract features from the preprocessed breast ultrasound image to obtain a feature map; Step S3: Use the segmentation module to segment the extracted feature map. During the segmentation process, the loss function is used to measure the difference between the segmentation result and the true label. The parameters of the segmentation module are continuously adjusted according to the difference calculated by the loss function through the back propagation algorithm. Step S4: performing SGBM stereo matching operation on the segmented images to obtain disparity information between the images, and performing adaptive curve correction on the disparity information to achieve three-dimensional reconstruction of breast nodules; Step S5: Establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation in step S4, wherein the dynamic calibration mechanism includes regularly recalibrating the speckle characteristics of the device used to acquire breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters, and designing the adaptive adjustment algorithm includes adjustment based on matching conditions and adjustment based on reconstruction effects.

2. The method for three-dimensional reconstruction of breast nodules according to claim 1, characterized in that: In step S1, filtering and denoising each frame of breast ultrasound image includes adaptive median filtering and Gaussian filtering, wherein the method of adaptive median filtering includes: An initial window is preset to determine whether the maximum difference of the pixel grayscale values ​​within the initial window centered on the current pixel point exceeds a preset threshold. If not, the initial window is used to perform a median filtering operation. If so, the window size is gradually increased until the maximum difference of the pixel grayscale values ​​within the window does not exceed the preset threshold.

3. The method for three-dimensional reconstruction of breast nodules according to claim 2, characterized in that: In step S1, image enhancement processing is performed on each frame of breast ultrasound image, including histogram equalization processing based on local contrast and adaptive contrast stretching processing, wherein the method of histogram equalization processing based on local contrast includes dividing the breast ultrasound image into a plurality of non-overlapping sub-regions, respectively counting the grayscale histogram of each sub-region and calculating the cumulative distribution function, using a weighted average method to give a higher weight to the cumulative distribution function of the current sub-region and a lower weight to the cumulative distribution function of the adjacent sub-region, so as to calculate the final cumulative distribution function value for mapping, and then remap the grayscale values ​​of the pixels in the sub-region; the method of adaptive contrast stretching processing includes calculating the statistical features of the grayscale histogram of the breast ultrasound image, judging whether the grayscale distribution of the image is biased towards dark distribution, bright distribution or uniform distribution based on the statistical features, and automatically determining the stretching based on the judgment result. If it is judged to be biased towards dark distribution, the starting grayscale value range of the stretching and the new grayscale value range to be stretched are automatically determined.

4. The method for three-dimensional reconstruction of breast nodules according to any one of claims 1 to 3, characterized in that: In step S2, the lightweight convolutional neural network structure is MobileNet.

5. The method for three-dimensional reconstruction of breast nodules according to claim 4, characterized in that: In step S2, the lightweight convolutional neural network structure extracts image features through a depth-separable convolution operation, and the method includes: Perform a deep convolution on each channel of the input breast ultrasound image and output a set of feature maps with the same number of input image channels; A 1×1 convolution is performed on each pixel of the depthwise convolution output to adjust the number of channels.

6. The method for three-dimensional reconstruction of breast nodules according to claim 4, characterized in that: In step S4, the method of performing SGBM stereo matching operation on the segmented images to obtain disparity information between the images includes: Extract key feature points from the segmented image, wherein the image includes a current frame image and a subsequent frame image; Match the key feature points extracted from the current frame image with the key feature points extracted from the next frame image. During the matching, feature descriptors are used to describe the key feature points, and the matching relationship between the key feature points is determined by calculating the distance between the feature descriptors. If the distance between the feature descriptors of the two key feature points is less than a preset distance threshold, the two key feature points match, otherwise, the two key feature points do not match; After completing the extraction and matching of key feature points, the matching cost is calculated by sampling. For each pair of matching key feature points, the absolute difference is calculated every few pixels and summed up to approximate the matching cost. The disparity information is calculated based on the position difference between the matched area in the current frame image and the next frame image.

7. The method for three-dimensional reconstruction of breast nodules according to claim 6, characterized in that: In step S5, the method for establishing a dynamic calibration mechanism includes: Collect Q frames of calibration sample images, and for each frame of calibration sample image I i (i=1,2,3…,Q), select a local area with each pixel point P(x,y) as the center; Calculate the speckle statistical features in the local area to obtain the speckle feature value, and record the speckle feature value of each pixel in each frame of the calibration sample image to form a speckle feature vector V for each frame of the calibration sample image. i (i=1,2,3…,Q); Calculate the Euclidean distance of the speckle feature vectors between the calibration sample images collected at different time points, and determine whether the Euclidean distance between the speckle feature vectors is greater than the set distance threshold. If so, recalibration is required, otherwise, recalibration is not required; Calculate the average speckle contrast of all the calibration sample images that need to be recalibrated. Average speckle size The average correlation coefficient of the speckle correlation And the integrated eigenvalues As a new speckle characteristic parameter for current breast ultrasound equipment; According to the new speckle characteristic parameters, the original standard SGBM curve calculation model is combined to recalculate and obtain the standard SGBM curve that adapts to the speckle characteristics of the current equipment.

8. The method for three-dimensional reconstruction of breast nodules according to claim 7, characterized in that: In step S5, the method for adjusting based on the matching situation includes: For each pair of matched current frame regions and subsequent frame regions, calculating a first evaluation index value of the two, wherein the first evaluation index value includes a similarity mean and a structural similarity index; The calculated first evaluation index value is compared with the set threshold. If the similarity mean is lower than the preset lower limit threshold, it is necessary to increase the adjustment range of the coherence coefficient during the adaptive curve correction process; if the structural similarity index exceeds the preset upper and lower limit thresholds, it is necessary to adjust the resampling ratio of the coherence coefficient.

9. The method for three-dimensional reconstruction of breast nodules according to claim 7, characterized in that: In step S5, the method based on reconstruction effect adjustment includes: Calculating a second evaluation index value according to a preset index for evaluating the reconstruction effect, wherein the second evaluation index value includes an error rate between the reconstructed nodule morphology and the actual anatomical structure and a visual quality evaluation index of the reconstructed image; The calculated error rate is compared with the set threshold. If the error rate exceeds the preset upper threshold, the calculation method of the SGBM curve in the adaptive curve correction process is changed. If the visual quality evaluation index of the reconstructed image is not good, the entire adaptive curve correction process is redesigned.

10. A breast nodule three-dimensional reconstruction system, characterized in that: include: An image preprocessing module is used to obtain multiple frames of breast ultrasound images and preprocess each frame of breast ultrasound image, wherein the preprocessing includes filtering, denoising and image enhancement. A feature extraction module is used to use the preprocessed breast ultrasound image as input, and use a lightweight convolutional neural network structure to extract features to obtain a feature map; The segmentation module is used to perform segmentation processing on the feature map obtained by feature extraction using the segmentation module. During the segmentation process, the difference between the segmentation result and the true label is measured using the loss function. The parameters of the segmentation module are continuously adjusted according to the difference calculated by the loss function through the back propagation algorithm to obtain an image result of accurately segmenting the breast nodule area. A three-dimensional reconstruction module is used to perform SGBM stereo matching operation on the image with the breast nodule area accurately segmented, obtain disparity information between images, and perform adaptive curve correction on the disparity information to achieve three-dimensional reconstruction of the breast nodule; A calibration and optimization control module is used to establish a dynamic calibration mechanism and design an adaptive adjustment algorithm to continuously optimize and calibrate the three-dimensional reconstruction operation, wherein the dynamic calibration mechanism includes regularly recalibrating the speckle characteristics of the equipment used to obtain breast ultrasound images, determining new speckle characteristic parameters, and recalculating the standard SGBM curve based on the new speckle characteristic parameters, and designing an adaptive adjustment algorithm including adjustment based on matching conditions and adjustment based on reconstruction effects.

Images