Preoperative ultrasonic intelligent searching and positioning method for CR focus after neoadjuvant chemotherapy of breast cancer

Abstract

The invention provides a pre-operative ultrasonic intelligent searching and positioning method for a CR focus after neoadjuvant chemotherapy of breast cancer, and relates to the field of focus searching and positioning, and the method comprises the steps: carrying out the preprocessing of an input ultrasonic image; extracting the primary features of the pre-processed area outside the lesion, performing multi-scale processing, and constructing a feature pyramid; and extracting multi-scale features, and fusing the multi-scale features through an adaptive weight allocation strategy. Enhancing contextual information of an area outside the focus in the fusion feature map by using an attention mechanism; segmenting the enhanced feature map by using a full convolutional layer to generate a segmentation mask of the CR focus region; and comparing image features of the area outside the lesion obtained by scanning of the probe with the learned feature mode in real time, and positioning the corresponding lesion area which is completely relieved. The method can well adapt to the characteristics of poor occupation effect, fuzzy boundary and strip-shaped distribution of the CR focus, and can accurately identify and locate the position of the CR focus in the ultrasonic image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of lesion positioning, and particularly relates to a method for preoperative ultrasound intelligent positioning of CR lesions after neoadjuvant chemotherapy of breast cancer. BACKGROUND

[0002] Breast cancer is one of the most common malignant tumors in women. With the progress of early screening and treatment technology, the treatment methods for breast cancer are also developing. Neoadjuvant chemotherapy (NAC) as an important treatment method aims to reduce the size of the tumor before surgery to improve the success rate of surgical resection. Existing studies have shown that NAC can effectively improve the survival rate and quality of life of patients with locally advanced or invasive breast cancer. However, despite the success of NAC in clinical application, how to accurately evaluate the efficacy and locate lesions remains a pressing problem.

[0003] During the process of neoadjuvant chemotherapy, doctors usually rely on imaging examinations (such as ultrasound, CT, MRI, etc.) to evaluate the response of the tumor. After several cycles of chemotherapy, the lesion may significantly shrink, even to the complete remission (CR) state. However, CR lesions are often difficult to identify in conventional ultrasound images, mainly due to their poor occupying effect, blurred boundaries, and patchy distribution. This makes it extremely difficult for doctors to locate the lesions before surgery, affecting the success rate of surgery and the prognosis of patients.

[0004] Currently, conventional ultrasound examination has significant limitations in identifying and locating CR lesions. First, as the lesion shrinks, the features of the lesion in the ultrasound image become less and less obvious, often confusing with the surrounding normal tissue. Second, existing imaging evaluation methods lack specificity and cannot effectively use the contrast information between pre- and post-chemotherapy ultrasound images to improve the accuracy of lesion positioning. In addition, existing technologies lack effective analysis methods for locating lesions that have disappeared, making it impossible for doctors to accurately determine the true location and range of the lesion before surgery. Therefore, it is of practical significance to develop a new method for locating CR lesions after neoadjuvant chemotherapy of breast cancer based on the comparison of ultrasound image features outside the lesion. SUMMARY

[0005] This invention primarily addresses the challenge of identifying and locating complete remission (CR) lesions in breast cancer on ultrasound images. As neoadjuvant chemotherapy progresses, lesion volume may significantly shrink or even disappear completely. However, in conventional ultrasound images, CR lesions, due to their poor mass effect, blurred boundaries, and patchy distribution, are often confused with surrounding normal tissue, making it difficult for physicians to accurately determine the true location and extent of the lesion before surgery. Furthermore, existing imaging assessment methods lack specificity and fail to effectively utilize the comparative information from ultrasound images before and after chemotherapy, further increasing the difficulty of lesion localization. Therefore, developing a new method to improve the accuracy of CR lesion localization on ultrasound images has become a crucial issue urgently needing to be addressed in the field of neoadjuvant chemotherapy for breast cancer.

[0006] To address the related technical issues, this invention provides a method for preoperative ultrasound-based intelligent location and placement of CR lesions after neoadjuvant chemotherapy for breast cancer.

[0007] To achieve the above objectives, the technical solution adopted in this invention is: a method for preoperative ultrasound-based intelligent location and localization of CR lesions after neoadjuvant chemotherapy for breast cancer, comprising the following steps:

[0008] Step S1: Preprocess the extralesional region in the ultrasound images of breast cancer after three cycles of neoadjuvant chemotherapy, wherein the extralesional region is the breast tissue region surrounding the original lesion.

[0009] Step S2: Use multiple convolutional layers to extract primary features of the preprocessed lesion region, the primary features including pixel value histogram, texture information and local structural features;

[0010] Step S3: Perform multi-scale processing on the primary features to construct a feature pyramid;

[0011] Step S4: Extract multi-scale features from the feature maps of different scales constructed in step S3, and fuse the multi-scale features through an adaptive weight allocation strategy;

[0012] Step S5: Enhance the contextual information of the region outside the lesion in the fused feature map obtained in step S4 using an attention mechanism;

[0013] Step S6: Use a fully convolutional layer to segment the enhanced feature map and generate a segmentation mask for the CR lesion region;

[0014] Step S7: During the preoperative ultrasound localization process, the image features of the area outside the lesion obtained by the probe scanning are compared with the learned feature patterns in real time to locate the corresponding lesion area that has been completely relieved.

[0015] Optionally, step S1 includes:

[0016] Step S1-1: Normalize the grayscale of the input ultrasound image, normalizing the pixel values ​​of the ultrasound image to... or Within the range;

[0017] Step S1-2: Apply a filter to remove random noise from the ultrasound image obtained in step S1-1;

[0018] Step S1-3: Enhance image contrast using histogram equalization or adaptive histogram equalization;

[0019] Steps S1-4: Use morphological operations to improve image quality.

[0020] Optionally, in step S2, the local structural features are obtained by extracting lesion edge and corner information through a convolutional layer.

[0021] Optionally, step S3 specifically includes:

[0022] Step S3-1: Extract multi-scale features through depthwise separable convolution and pointwise convolution;

[0023] Step S3-2: Upsample the multi-scale features to the original resolution and then fuse them using adaptive weights.

[0024] Optionally, in step S3-1, the extraction of multi-scale features is based on the cascaded operation of depthwise convolution and pointwise convolution, combined with an activation function.

[0025] In step S3-2, the adaptive weight fusion of multi-scale features is achieved by weighting and summing the upsampled features based on the learned weights.

[0026] Optionally, step S4 specifically includes:

[0027] Step S4-1: Extract features from feature maps at different scales;

[0028] Step S4-2: Apply attention mechanisms to emphasize important features;

[0029] Step S4-3: Dynamically fuse features using a gating mechanism;

[0030] Step S4-4: Introduce skip connections to enhance feature representation.

[0031] Optionally, step S5 includes:

[0032] Step S5-1: Extract and transform features from the input feature map;

[0033] Step S5-2: Apply a gating mechanism to dynamically adjust feature importance;

[0034] Step S5-3: Fuse the adjusted features with the original features;

[0035] Step S5-4: Assign weights to the fused feature maps using an attention mechanism;

[0036] Step S5-5: Sum all weighted feature maps to obtain the comprehensive feature representation;

[0037] Steps S5-6: Perform further feature mapping and boundary prediction on the comprehensive feature representation;

[0038] Step S5-7: Introduce the original features into the boundary prediction through residual connections;

[0039] Step S5-8: Apply the activation function to generate the final boundary probability map.

[0040] Optionally, step S6 includes:

[0041] Step S6-1: Perform global average pooling on the feature map after multi-scale fusion to extract global context information;

[0042] Step S6-2: Use convolutional layers to extract local contextual information;

[0043] Step S6-3: Concatenate the global and local context information and generate attention weights using a multilayer perceptron;

[0044] Step S6-4: Recalibrate the feature map using attention weights

[0045] Beneficial effects:

[0046] This invention, through the aforementioned technical solution, can effectively improve the accuracy of identification and localization of complete remission (CR) lesions in ultrasound images after neoadjuvant chemotherapy for breast cancer, overcoming the limitations of existing technologies in lesion localization. This method is adaptable to the characteristics of CR lesions, such as poor mass effect, blurred boundaries, and patchy distribution.

[0047] 1. Through the above technical solution, firstly, the method of the present invention can achieve indirect localization of invisible lesions, thus solving the problem of direct identification of CR lesions due to changes in their morphological characteristics. Specifically, because CR lesions have poor space-occupying effect, blurred boundaries, and a strip-like distribution, their contrast with normal tissue in ultrasound images is extremely low, making it impossible to locate them by directly observing their morphology and boundaries like traditional tumors. The method of the present invention innovatively shifts the focus from the lesion itself to the area outside the lesion (i.e., the breast tissue area surrounding the original lesion). By analyzing the changes in tissue characteristics in this area, the location of the original lesion can be inferred. The method of the present invention breaks away from the traditional WYSIWYG image analysis paradigm, providing a feasible technical path for locating CR lesions that are almost invisible on images through indirect reasoning logic from the outside in.

[0048] Second, the method of this invention, through a systematic feature learning process, effectively ensures the accuracy and robustness of localization. Specifically, step S1 preprocesses the input image to provide a high-quality image foundation with less noise and better contrast for all subsequent steps. This effectively and directly improves the stability and reliability of feature extraction, which is a prerequisite for ensuring the overall system performance. Step S2 uses convolutional layers to automatically extract primary features, replacing the shallow features (such as hand-drawn textures) that traditional methods rely on manually designed. This allows the system to autonomously learn more effective feature representations related to CR lesions from the data, laying a richer and more relevant information foundation for subsequent analysis. Step S3 constructs a feature pyramid to simultaneously capture information at both the detail level (small-scale features, such as subtle structural distortions) and the semantic level (large-scale features, such as the overall arrangement pattern of tissue regions). This allows the system to more comprehensively understand the image content and avoid missing key clues by focusing only on a single scale. Step S4 uses an adaptive weight allocation strategy for fusion, meaning the system can dynamically and selectively strengthen scale features that are more important to the current localization task, while suppressing unimportant or noisy features. This effectively improves the quality and discriminative ability of feature representations. Step S5 simulates a doctor's focus through an attention mechanism, allowing the system to automatically focus on the contextual information of the region most relevant to the lesion, ignoring irrelevant background. This effectively improves the signal-to-noise ratio of the feature map, making subsequent segmentation and localization more accurate. The fully convolutional layer in Step S6 classifies each pixel, ultimately generating a segmentation mask that clearly indicates the possible region of the CR lesion. This provides doctors with clear visual guidance, transforming abstract features into concrete regions.

[0049] Third, the method of this invention can achieve real-time and practical preoperative localization, and has significant clinical value. Step S7 of this invention applies the aforementioned offline-trained model to a real preoperative scenario. By comparing the probe scan images with the learned feature patterns in real time, the system can instantly provide localization information on the screen. This effect transforms this method from an offline analysis tool into an intelligent auxiliary decision-making system that can be seamlessly integrated into the surgical procedure, directly guiding the surgeon's intraoperative operations and improving the accuracy and efficiency of the surgery.

[0050] 2. Other beneficial effects or advantages of the present invention will be described in detail in the specific embodiments. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating the steps of a method for preoperative ultrasound-based intelligent location and positioning of CR lesions in breast cancer after neoadjuvant chemotherapy, provided by an exemplary embodiment of the present invention.

[0053] Figures 2 to 5 This is a comparative schematic diagram of the initial and final images of CR lesion identification achieved by the preoperative ultrasound intelligent location method for CR lesions after neoadjuvant chemotherapy for breast cancer according to the present invention. In each figure, the left side is the initial image and the right side is the final image. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0055] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0056] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0057] like Figure 1As shown, this invention provides a method for preoperative ultrasound-based intelligent location of CR lesions in breast cancer after neoadjuvant chemotherapy. The method includes the following steps:

[0058] Step S1: Preprocessing of the extralesional region: In the ultrasound images of breast cancer patients after completing three cycles of neoadjuvant chemotherapy (NAC), the extralesional region (i.e., the breast tissue area surrounding the original lesion) is preprocessed to improve image quality and lay the foundation for subsequent feature extraction and comparison.

[0059] Step S2: Primary feature extraction of the extralesional region: Using multiple convolutional layers, primary features of the extralesional region ultrasound image after preprocessing in Step S1 are extracted; wherein, the primary features include pixel value histogram, texture information, and local structural features; in order to capture key information of the tissue surrounding the lesion and provide basic features for subsequent CR lesion locating.

[0060] Step S3: Construct a feature pyramid for the extralesional region: Perform multi-scale processing on the primary features of the extralesional region extracted in step S2 to construct a feature pyramid for the extralesional region; in order to integrate features of the extralesional region at different scales, thereby capturing the multi-scale morphological changes that may be caused by the potential area of ​​CR lesions and manifested in the extralesional region.

[0061] Step S4: Multi-scale feature fusion of the extralesional region: Extract multi-scale features of the extralesional region from the feature maps of different scales constructed in step S3, and fuse the multi-scale features through adaptive weight allocation; dynamically adjust the weights according to the importance of the extralesional region features in indicating the location of CR lesions, and enhance the sensitivity to the boundary region of potential CR lesions.

[0062] Step S5: Enhance the contextual information of the region outside the lesion: Use the attention mechanism to enhance the contextual information of the region outside the lesion in the fused feature map obtained in step S4; in order to improve the significance of features related to the location of potential CR lesions in the region outside the lesion, and enhance the model's ability to identify CR lesions (which have poor space-occupying effect, blurred boundaries and strip-like distribution).

[0063] Step S6: Generation of CR lesion region segmentation mask: Use a fully convolutional layer to segment the feature map obtained in step S5 to generate a segmentation mask for the breast cancer complete remission (CR) lesion region; realize the prediction of CR lesion region based on feature comparison of the region outside the lesion.

[0064] Step S7: Intraoperative Intelligent Location and Localization of CR Lesions: During the preoperative ultrasound localization process after breast cancer patients have completed a full course of neoadjuvant chemotherapy, the operator obtains an image of the area outside the lesion after NAC in the current section through dynamic scanning with a probe; the image is processed in real time using the model established in the aforementioned steps (S1-S6), its characteristics are analyzed and compared with the learned pattern; the predicted CR lesion location information is intelligently output (such as segmentation mask overlay display, coordinate indication, etc.), guiding the operator to accurately find and locate the corresponding lesion area that has achieved complete remission (CR) but is difficult to observe directly with conventional ultrasound.

[0065] Through the above technical solution, firstly, the method of the present invention can achieve indirect localization of invisible lesions, thus solving the problem of direct identification of CR lesions due to changes in their morphological characteristics. Specifically, because CR lesions have poor space-occupying effect, blurred boundaries, and a strip-like distribution, their contrast with normal tissue in ultrasound images is extremely low, making it impossible to locate them by directly observing their morphology and boundaries like traditional tumors. The method of the present invention innovatively shifts the focus from the lesion itself to the area outside the lesion (i.e., the breast tissue area surrounding the original lesion). By analyzing the changes in tissue characteristics in this area, the location of the original lesion can be inferred. The method of the present invention breaks away from the traditional WYSIWYG image analysis paradigm, providing a feasible technical path for locating CR lesions that are almost invisible on images through indirect reasoning logic from the outside in.

[0066] Second, the method of this invention, through a systematic feature learning process, effectively ensures the accuracy and robustness of localization. Specifically, step S1 preprocesses the input image to provide a high-quality image foundation with less noise and better contrast for all subsequent steps. This effectively and directly improves the stability and reliability of feature extraction, which is a prerequisite for ensuring the overall system performance. Step S2 uses convolutional layers to automatically extract primary features, replacing the shallow features (such as hand-drawn textures) that traditional methods rely on manually designed. This allows the system to autonomously learn more effective feature representations related to CR lesions from the data, laying a richer and more relevant information foundation for subsequent analysis. Step S3 constructs a feature pyramid to simultaneously capture information at both the detail level (small-scale features, such as subtle structural distortions) and the semantic level (large-scale features, such as the overall arrangement pattern of tissue regions). This allows the system to more comprehensively understand the image content and avoid missing key clues by focusing only on a single scale. Step S4 uses an adaptive weight allocation strategy for fusion, meaning the system can dynamically and selectively strengthen scale features that are more important to the current localization task, while suppressing unimportant or noisy features. This effectively improves the quality and discriminative ability of feature representations. Step S5 simulates a doctor's focus through an attention mechanism, allowing the system to automatically focus on the contextual information of the region most relevant to the lesion, ignoring irrelevant background. This effectively improves the signal-to-noise ratio of the feature map, making subsequent segmentation and localization more accurate. The fully convolutional layer in Step S6 classifies each pixel, ultimately generating a segmentation mask that clearly indicates the possible region of the CR lesion. This provides doctors with clear visual guidance, transforming abstract features into concrete regions.

[0067] Third, the method of this invention can achieve real-time and practical preoperative localization, and has significant clinical value. Step S7 of this invention applies the aforementioned offline-trained model to a real preoperative scenario. By comparing the probe scan images with the learned feature patterns in real time, the system can instantly provide localization information on the screen. This effect transforms this method from an offline analysis tool into an intelligent auxiliary decision-making system that can be seamlessly integrated into the surgical procedure, directly guiding the surgeon's intraoperative operations and improving the accuracy and efficiency of the surgery.

[0068] Furthermore, it should be noted that, firstly, in the method of the present invention, by performing multi-scale processing on the primary features and constructing a feature pyramid to achieve the fusion of features at different scales, the multi-morphological capture of CR lesions can be achieved, so as to be well adapted to the morphological characteristics of CR lesions.

[0069] Second, in the method of the present invention, multi-scale features are first extracted from feature maps of different scales, and multi-scale features are fused through adaptive weight allocation. This allows for dynamic adjustment of weights based on the importance of features, so as to enhance boundary information more effectively. At the same time, a sub-network is used to refine the boundary of CR lesions, which can provide more accurate boundary localization to address the fuzzy boundary characteristics of CR lesions.

[0070] Third, in the method of the present invention, the contextual information of CR lesions is enhanced by utilizing an attention mechanism, thereby improving the contrast between the lesions and the background, so as to effectively address the problem of similar intensity distribution of CR lesions.

[0071] Overall, the method of the present invention can effectively address the characteristics of variable morphology, blurred boundaries, and similar intensity distribution of CR lesions, thereby enabling more accurate identification and localization of the location and contour of CR lesions in ultrasound images. This allows for a more accurate assessment of the nature and status of CR lesions after NAC in breast cancer, which is beneficial not only for developing further surgical and treatment plans but also for accurately evaluating the efficacy of NAC in the patient.

[0072] In one embodiment of the present invention, step S1 may specifically include:

[0073] Step S1-1: Normalize the grayscale of the input ultrasound image, normalizing the pixel values ​​of the ultrasound image to... or Within a certain range; so as to facilitate neural network processing.

[0074] Step S1-2: Apply a filter to remove random noise from the ultrasound image obtained in step S1-1;

[0075] Steps S1-3: Use histogram equalization or adaptive histogram equalization to enhance image contrast; this helps to make the difference between the lesion and the background more obvious.

[0076] Steps S1-4: Use morphological operations (such as dilation, erosion, opening, closing, etc.) to improve image quality, especially the clarity of lesion boundaries.

[0077] In this way, by performing grayscale normalization, noise removal, contrast enhancement, and morphological operations in the above process, the quality of ultrasound images can be effectively improved, thereby enhancing segmentation efficiency and ensuring segmentation results.

[0078] In one embodiment of the present invention, step S3 may specifically include:

[0079] Step S3-1: Extract features at multiple scales using the following formula:

[0080]

[0081] In the formula, Indicates the first Features at each scale For activation function, The weights of the depthwise convolution. This represents the convolution operation. For depthwise convolution, For the input feature map, For the first The weight matrix used for depthwise convolution at each scale. This is the bias term corresponding to depthwise convolution. The weights are those corresponding to pointwise convolution. For pointwise convolution, For the first The weight matrix used for pointwise convolution at each scale. The bias term corresponding to pointwise convolution;

[0082] In this implementation, it should be noted that, firstly, in depthwise convolution, each input channel has an independent convolution kernel, so... The shape is typically [height, width, input_channels], where height and width are the spatial dimensions of the convolution kernel, and input_channels is the number of channels in the input feature map. The purpose of depthwise convolution is to extract spatial features independently on each input channel.

[0083] Second, in pointwise convolution, the kernel size is 1x1, and its purpose is to combine features across all input channels. Therefore, The shape is typically [1, 1, input_channels, output_channels], where input_channels is the number of channels after depthwise convolution, and output_channels is the number of channels after pointwise convolution. The purpose of pointwise convolution is to mix information across channels using a 1x1 convolution kernel to generate new feature representations.

[0084] Step S3-2: Fuse the multi-scale features extracted in step S3-1 according to the following formula:

[0085]

[0086] In the formula, and The weights obtained during the learning process , ,in, and These are the weights of the convolution operation. and It is a bias term. Indicates features The process of upsampling to the original resolution, Indicates the first Features at each scale For the first Each scale.

[0087] In this way, the above technical solution can construct a feature pyramid more adapted to CR lesions, effectively fusing features at different scales and strengthening important features through adaptive weight adjustment while suppressing unimportant features. This enables the capture of CR lesions of different morphologies and effectively improves the segmentation accuracy of CR lesions in ultrasound images.

[0088] In one embodiment of the present invention, step S4 may specifically include:

[0089] Step S4-1: Extract features from feature maps at different scales:

[0090]

[0091]

[0092] In the formula, This is the first-scale feature map. This refers to the second-scale feature map, where the first-scale feature map is the output of the first or second convolutional layer in the convolutional neural network, and the second-scale feature map is the output of the penultimate or last convolutional layer in the convolutional neural network. For the first Features at each scale These are the convolution weights corresponding to the first-scale feature map. These are the convolution weights corresponding to the second-scale feature map.

[0093] Step S4-2: Apply attention mechanisms to emphasize important features:

[0094]

[0095]

[0096] In the formula, It is a first-scale feature map The features in the image are obtained through attention weights from the SE (Squeeze-and-Excitation) layer. It is a second-scale feature map The features in the image are obtained through the attention weights of the SE layer.

[0097] In this embodiment, it should be noted that,

[0098] First-scale feature maps typically have higher spatial resolution, meaning they are denser in pixels and can capture finer details and texture information. At the same time, these feature maps are closer to the network's input layer, so they contain more information at the raw pixel level, such as edges and corners. Furthermore, first-scale feature maps may have fewer channels (feature dimensions) because they focus more on low-level visual features.

[0099] Second-scale feature maps have lower resolution and sparser pixels, resulting in a larger receptive field and the ability to capture broader contextual information. Furthermore, these feature maps are typically located in deeper layers of the network and contain more abstract information, such as object categories and scene layout.

[0100] In this way, by effectively fusing these feature maps of different scales, high-level semantic information can be introduced while preserving spatial details, thereby improving the performance of segmentation or detection tasks.

[0101] Step S4-3: Use a gating mechanism to dynamically fuse features:

[0102]

[0103]

[0104] In the formula, For gating signals, For activation function, It is the first Convolutional kernel weights in each feature fusion module This is the fused feature map.

[0105] Step S4-4: Introduce skip connections:

[0106]

[0107] In the formula, This is the final output feature map.

[0108] Through the above technical solution, this invention extracts multi-scale features using different convolutional layers, effectively enhancing the model's ability to identify CR lesions of different sizes and shapes. Simultaneously, through the SELayer (SE layer), the model can automatically learn the importance of different features and adjust their weights accordingly, improving the discriminative power of the feature representation. Furthermore, the gating mechanism of this invention allows the model to dynamically select and fuse features, which helps capture complex contextual information and improve segmentation accuracy. In addition, skip connections help alleviate the gradient vanishing problem and allow the model to retain more original information, thereby improving the detail and robustness of the segmentation.

[0109] Furthermore, in this embodiment, it should be specifically noted that...

[0110] In the feature pyramid construction step (i.e., step S3) of this invention, features at different scales have already been fused. However, the reason why features are extracted from different feature maps again in step S4 of this invention includes:

[0111] First, feature refinement and optimization. The main purpose of the feature pyramid construction step is to generate multi-scale feature representations. These features may contain rich spatial information, but they may not be refined enough or task-specific. The feature extraction in step S4 refines and optimizes these features, making them more suitable for the final segmentation task.

[0112] Second, selective feature emphasis. In the feature pyramid, although fusion has been performed, some key features may not have received sufficient emphasis. Step S4 can selectively emphasize features that are more important to the segmentation task through more complex operations (such as attention mechanisms).

[0113] Third, adaptive feature fusion. Step S4 dynamically adjusts the importance of features through an adaptive mechanism, which helps the model better adapt to the diversity and complexity of CR lesions. This adaptive fusion can be based on the current feature content and the needs of the segmentation task.

[0114] Fourth, multi-level feature representation. Feature representations in neural networks are typically hierarchical. Step S4 can further extract higher-level features, which may contain more abstract semantic information, helping to improve the accuracy and robustness of segmentation.

[0115] Fifth, residual and skip connections. Feature extraction in step S4 may include residual connections or skip connections, which help alleviate the vanishing gradient problem and allow the network to learn more complex mappings.

[0116] In summary, the feature extraction in step S4 is not simply a repetitive task, but rather a further refinement and optimization of features based on the feature pyramid, making them more suitable for the accurate segmentation of CR lesions. Through this multi-level, adaptive feature processing, the model can more effectively learn and predict the boundaries of CR lesions.

[0117] In one embodiment of the present invention, step S5 may specifically include:

[0118] Step S5-1: Process the input feature map Apply a series of convolutional layers Feature extraction and transformation are performed, that is, This yields intermediate feature representations; thus, it is possible to enhance feature representation capabilities and extract features that are more conducive to boundary refinement.

[0119] Step S5-2: Apply gating mechanism The importance of features can be dynamically adjusted; in this way, features useful for boundary refinement can be emphasized, while unimportant features can be suppressed.

[0120] Step S5-3: Perform element-wise multiplication (i.e., dot product) between the features adjusted in step S5-2 and the original features, and then pass the result through a convolutional layer. To merge, that is In this way, features from different levels can be integrated to improve the discriminative power of features.

[0121] Step S5-4: Assign weights to each fused feature map using an attention mechanism. In this way, the weights can be dynamically adjusted according to the importance of the features, thereby improving the accuracy of the boundaries.

[0122] Step S5-5: Sum all weighted feature maps to obtain the comprehensive feature representation, i.e., In this way, multi-scale features can be integrated to improve the continuity and accuracy of boundaries.

[0123] Steps S5-6: Through convolutional layers Further feature mapping and boundary prediction are performed on the comprehensive feature representation obtained in step S5-5, that is, In this way, a boundary probability map can be generated, refining the boundary information.

[0124] Step S5-7: Connect via residuals Original features This is directly incorporated into the final boundary prediction, that is, This can alleviate the vanishing gradient problem, preserve the original feature information, and improve the stability of network training.

[0125] Step S5-8: Apply activation function Generate the final boundary probability map , This allows for the introduction of nonlinearity, ensuring the accuracy of boundary prediction.

[0126] The above technical solution first enhances feature representation capabilities by extracting features more conducive to boundary refinement. Then, a gating mechanism emphasizes features useful for boundary refinement while suppressing unimportant features. Next, features at different levels are fused to improve boundary accuracy. Subsequently, an attention mechanism dynamically adjusts feature weights based on their importance, further enhancing boundary accuracy. Then, multi-scale features are integrated to improve boundary continuity and accuracy. Residual connections are used to incorporate original features into boundary prediction to alleviate the gradient vanishing problem. Finally, nonlinearity is introduced to generate the final boundary probability map, ensuring the accuracy of boundary prediction. In other words, this technical solution can effectively and more precisely refine the boundaries of CR lesions in ultrasound images, solving the boundary blurring problem and improving the accuracy of boundary localization.

[0127] In one embodiment of the present invention, step S6 may specifically include:

[0128] Step S6-1: Perform global average pooling on the feature map after multi-scale fusion to obtain a feature map containing global context information. The eigenvectors of are calculated using the following formula:

[0129]

[0130] In the formula, and These represent the height and width of the feature map, respectively. This indicates the location of the fused feature map. The eigenvalue at that location.

[0131] Step S6-2: Use a convolutional layer to extract local contextual information from the fused feature map. The extraction formula is as follows:

[0132]

[0133] In the formula, For activation function, For convolution kernel weights, For kernel bias, This represents the number of convolution kernels.

[0134] Step S6-3: Transfer the global context information obtained in step S6-1 and the local context information obtained in step S6-2 The data is stitched together, and attention weights are generated using a multilayer perceptron. The calculation formula is as follows:

[0135]

[0136] In the formula, For the weights of the multilayer perceptron, This is the bias for the multilayer perceptron. This represents a vector concatenation operation;

[0137] Step S6-4: Recalibrate the fused feature map using the attention weights generated in step S6-3.

[0138] Thus, through the aforementioned technical solution, global average pooling is first used to extract global contextual information from the feature map, which helps the network focus on the important features of the entire CR lesion region. Simultaneously, local convolution operations allow the network to focus on the local features of the CR lesion, thereby improving segmentation accuracy. Furthermore, attention weights enable the network to adaptively adjust the weights of the feature map based on global and local contextual information, highlighting the important features of the CR lesion. Finally, feature relabeling allows the network to focus more on the key regions of the CR lesion and suppress irrelevant information, thereby effectively improving segmentation performance.

[0139] In this invention, the Dice evaluation function is used to process the method of this invention on a dataset (including 10 ultrasound images of breast cancer NAC post-CR lesions under different conditions, numbered 1 to 10). The results are shown in Table 1 below.

[0140] Table 1. Processing results of the method of the present invention on a dataset.

[0141] Ultrasound image number DSC 1 0.81244 2 0.91339 3 0.96954 4 0.91487 5 0.94613 6 0.94566 7 0.95461 8 0.96535 9 0.89945 10 0.87164

[0142] The results show that the method of the present invention performs well in this dataset, with good image segmentation performance. It can be well adapted to the characteristics of CR lesions, such as variable morphology, blurred boundaries and similar intensity distribution. It can more accurately identify and locate the position and contour of CR lesions in ultrasound images, so as to make a more accurate assessment of the nature and status of CR lesions after NAC in breast cancer. This is not only beneficial for the formulation of further surgical and treatment plans, but also for the accurate assessment of the NAC efficacy of the patient.

[0143] The technical solution of the present invention will be further described below with reference to an exemplary embodiment.

[0144] This exemplary implementation uses a specific postoperative ultrasound image of a breast cancer patient as an example. It assumes the input image is a 512×512 pixel grayscale ultrasound image with pixel values ​​ranging from [0, 255]. The patient has completed three cycles of neoadjuvant chemotherapy (NAC), and the original lesion area has achieved complete remission (CR). This method aims to locate the CR lesion by intelligently analyzing the characteristics of the area outside the lesion (i.e., the breast tissue area surrounding the original lesion).

[0145] Step 1: Pretreatment of the area outside the lesion.

[0146] Step 1.1: Normalize the pixel values ​​of the input ultrasound image from [0, 255] to the range [0, 1]. The calculation formula is: ;in, The original image is in position pixel values, It is the normalized value.

[0147] Step 1.2: Apply a Gaussian filter (kernel size 5×5, standard deviation σ=1.0) to remove random noise. Filtered image. The calculation is as follows: ;in, This indicates a convolution operation.

[0148] Step 1.3: Enhance image contrast using Adaptive Histogram Equalization (CLAHE). Set the cropping limit to 2.0 and the grid size to 8×8. The output image is denoted as... .

[0149] Step 1.4: Apply a closing operation (dilation followed by erosion) to improve image quality, with a kernel size of 3×3. The output image is denoted as... This serves as the input for subsequent processing.

[0150] Step 2: Extraction of primary features of the area outside the lesion.

[0151] Multiple convolutional layers are used to extract primary features from the preprocessed image.

[0152] enter: (512×512×1);

[0153] Feature extraction is performed using three convolutional layers (kernel size 3×3, stride 1, padding 1):

[0154] First layer: Output channel 32, extract pixel value histogram features.

[0155] Second layer: Output channel 64, extract texture information (such as LBP equivalent features).

[0156] The third layer: output channel 128, extracting local structural features (edge ​​and corner information).

[0157] Output primary feature map (512×512×128).

[0158] Step 3: Construct a feature pyramid of the region outside the lesion.

[0159] Step 3.1: Multi-scale feature extraction.

[0160] We extract features at three scales (i=1,2,3) using depthwise separable convolution and pointwise convolution. Assume the input feature map... ;

[0161] Scale 1 (Original Resolution):

[0162]

[0163] in, The weights are depthwise convolutional weights (3×3×128). For pointwise convolution weights (1×1×128×64), output (512×512×64).

[0164] Scale 2 (downsampling by 2):

[0165] Similarly, using a convolution with a stride of 2, the output is... (256×256×128).

[0166] Scale 3 (downsampling by 4 times):

[0167] Similarly, output (128×128×256).

[0168] Step 3.2: Adaptive Weight Fusion

[0169] Multi-scale features are upsampled to the original resolution and then fused.

[0170]

[0171] in, , Similar calculations.

[0172] Assume the learned weights are: , , , , , .

[0173] Feature map after fusion (512×512×256) is used as the next input.

[0174] Step S4: Multi-scale feature fusion of the region outside the lesion (including feature extraction, attention mechanism, gating mechanism, and skip connections).

[0175] Step 4.1: Extract features from different scales

[0176] Extract first-scale (shallow) and second-scale (deep) features from the feature pyramid:

[0177] ,

[0178] For example, (512×512×64) originates from shallow layers. (512×512×128) comes from the deep layer.

[0179] Step 4.2: Apply attention mechanisms

[0180] Attention weights are calculated using the Squeeze-and-Excitation (SE) layer:

[0181] ,

[0182] Assumption and A weight map of the same size as the feature map.

[0183] Step 4.3: Gating Mechanism Fusion Features

[0184] Calculate the gated signal and fuse its features:

[0185]

[0186]

[0187] in, For the Sigmoid function, These are the kernel weights.

[0188] Step S4-4: Jump Connection

[0189]

[0190] Output fused feature map (512×512×256).

[0191] Step S5: Enhance the contextual information of the region outside the lesion. (Use attention mechanisms to enhance the contextual information and generate a boundary probability map.)

[0192] Step 5.1: Feature Extraction and Transformation

[0193] For the input feature map Apply convolutional layers This yields intermediate features.

[0194] Step 5.2: Gating Mechanism Adjustment Features

[0195] Application gating mechanism Dynamically adjust the importance of features.

[0196] Step 5.3: Feature Fusion

[0197] The adjusted features are multiplied with the original feature points and then fused:

[0198]

[0199] Step 5.4: Attention Weight Allocation

[0200] Assign weights to each fused feature map ,For example, , , .

[0201] Step 5.5: Feature Summation

[0202] The comprehensive features are represented as follows:

[0203]

[0204] Step 5.6: Boundary Prediction

[0205] Through convolutional layers Further mapping:

[0206]

[0207] Step 5.7: Residual Connection

[0208] Introducing original features:

[0209]

[0210] Step 5.8: Generate boundary probability map

[0211] Applying the Sigmoid activation function:

[0212]

[0213] Output boundary probability map B (512×512×1), with a value range of [0,1], representing the probability that each pixel belongs to the boundary of the CR lesion.

[0214] Step 6: Generate the CR lesion region segmentation mask.

[0215] Step 6.1: Global Average Pooling

[0216] For fused feature maps Perform global average pooling on (512×512×256):

[0217]

[0218] Output (1×1×256).

[0219] Step 6.2: Extraction of local contextual information

[0220] Use convolutional layers (kernel size 3×3, number of channels 256) to extract local features:

[0221]

[0222] Output (512×512×256).

[0223] Step 6.3: Generate attention weights

[0224] By combining global and local information, weights are generated using a multilayer perceptron (MLP).

[0225]

[0226] Output attention weight map (512×512×1).

[0227] Step 6.4: Feature Recalibration

[0228] Multiply the attention weights by the fused feature map:

[0229]

[0230] Finally, a fully convolutional layer (kernel size 1×1) is used to... The mapping is a segmentation mask M (512×512×1), where pixel values ​​greater than 0.5 are considered CR lesion areas.

[0231] Step 7: Intraoperative CR lesion intelligent location and positioning

[0232] During preoperative ultrasound localization, real-time probe scan images (e.g., 512×512 pixels) are acquired and input into a trained model (based on steps S1-S6). The model outputs a segmentation mask in real time, which is overlaid on the ultrasound image to guide the physician in locating CR lesion areas. For example, when the probe scans a certain area, the model outputs a mask probability of 0.8, indicating that the area has a high probability of being a CR lesion, and the system marks it with a colored outline (for example, see [reference]). Figures 2 to 5(As shown).

[0233] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for preoperative ultrasound-guided intelligent localization of CR lesions in breast cancer after neoadjuvant chemotherapy, characterized in that: Includes the following steps: Step S1: Preprocess the extralesional region in the ultrasound images of breast cancer after three cycles of neoadjuvant chemotherapy, wherein the extralesional region is the breast tissue region surrounding the original lesion. Step S2: Use multiple convolutional layers to extract primary features of the preprocessed lesion region, the primary features including pixel value histogram, texture information and local structural features; Step S3: Perform multi-scale processing on the primary features to construct a feature pyramid; Step S4: Extract multi-scale features from the feature maps of different scales constructed in step S3, and fuse the multi-scale features through an adaptive weight allocation strategy; Step S5: Enhance the contextual information of the region outside the lesion in the fused feature map obtained in step S4 using an attention mechanism; Step S6: Use a fully convolutional layer to segment the enhanced feature map and generate a segmentation mask for the CR lesion region; Step S7: During the preoperative ultrasound localization process, the image features of the area outside the lesion obtained by the probe scanning are compared with the learned feature patterns in real time to locate the corresponding lesion area that has been completely relieved.

2. The method for preoperative ultrasound-guided intelligent localization of CR lesions after neoadjuvant chemotherapy for breast cancer according to claim 1, characterized in that, Step S1 includes: Step S1-1: Normalize the grayscale of the input ultrasound image, normalizing the pixel values ​​of the ultrasound image to... or Within the range; Step S1-2: Apply a filter to remove random noise from the ultrasound image obtained in step S1-1; Step S1-3: Enhance image contrast using histogram equalization or adaptive histogram equalization; Steps S1-4: Use morphological operations to improve image quality.

3. The method for preoperative ultrasound-guided intelligent localization of CR lesions after neoadjuvant chemotherapy for breast cancer according to claim 1, characterized in that, In step S2, the local structural features are obtained by extracting lesion edge and corner information through convolutional layers.

4. The method for preoperative ultrasound-guided intelligent localization of CR lesions after neoadjuvant chemotherapy for breast cancer according to claim 1, characterized in that, Step S3 specifically includes: Step S3-1: Extract multi-scale features through depthwise separable convolution and pointwise convolution; Step S3-2: Upsample the multi-scale features to the original resolution and then fuse them using adaptive weights.

5. The method for preoperative ultrasound-guided intelligent localization of CR lesions after neoadjuvant chemotherapy for breast cancer according to claim 1, characterized in that, In step S3-1, the extraction of multi-scale features is based on the cascaded operation of depthwise convolution and pointwise convolution, combined with an activation function. In step S3-2, the adaptive weight fusion of multi-scale features is achieved by weighting and summing the upsampled features based on the learned weights.

6. The method for preoperative ultrasound-guided intelligent localization of CR lesions after neoadjuvant chemotherapy for breast cancer according to claim 1, characterized in that, Step S4 specifically includes: Step S4-1: Extract features from feature maps at different scales; Step S4-2: Apply attention mechanisms to emphasize important features; Step S4-3: Dynamically fuse features using a gating mechanism; Step S4-4: Introduce skip connections to enhance feature representation.

7. The method for preoperative ultrasound-guided intelligent localization of CR lesions after neoadjuvant chemotherapy for breast cancer according to claim 1, characterized in that, Step S5 includes: Step S5-1: Extract and transform features from the input feature map; Step S5-2: Apply a gating mechanism to dynamically adjust feature importance; Step S5-3: Fuse the adjusted features with the original features; Step S5-4: Assign weights to the fused feature maps using an attention mechanism; Step S5-5: Sum all weighted feature maps to obtain the comprehensive feature representation; Steps S5-6: Perform further feature mapping and boundary prediction on the comprehensive feature representation; Step S5-7: Introduce the original features into the boundary prediction through residual connections; Step S5-8: Apply the activation function to generate the final boundary probability map.

8. The method for preoperative ultrasound-guided intelligent localization of CR lesions after neoadjuvant chemotherapy for breast cancer according to claim 1, characterized in that, Step S6 includes: Step S6-1: Perform global average pooling on the feature map after multi-scale fusion to extract global context information; Step S6-2: Use convolutional layers to extract local contextual information; Step S6-3: Concatenate the global and local context information and generate attention weights using a multilayer perceptron; Step S6-4: Recalibrate the feature map using attention weights.

Images