Breast cancer MRI image enhancement and lesion automatic identification method based on attention mechanism

Abstract

The application discloses a breast cancer MRI image enhancement and lesion automatic identification method based on an attention mechanism and relates to the technical field of data processing and computer-aided diagnosis. The method comprises the following steps: acquiring multi-time-point breast MRI image samples and performing pretreatment; performing random window enhancement and random channel commutative enhancement on the pretreated samples, inputting the enhanced image samples into an improved convolutional neural network, and optimizing the deep metric learning loss function based on the proxy NCA to extract lesion embedding features; according to the lesion embedding features, a square matching game framework with the expert-labeled lesion state distribution as a target is constructed; according to the square matching game framework, an inverse reinforcement learning strategy without reinforcement learning is adopted, and the lesion identification model parameters are iteratively optimized through a no-regret square matching algorithm; the optimized model is used for lesion property judgment and the judgment result and the corresponding confidence score are output, so that the MRI image adaptive enhancement and accurate lesion automatic identification are realized.

Description

Technical Field

[0001] The embodiments of the present invention relate to the fields of data processing and computer-aided diagnosis technology, and in particular to a method for breast cancer MRI image enhancement and automatic lesion identification based on attention mechanism. Background Technology

[0002] Breast cancer is a common malignant tumor in women, and MRI, with its high soft tissue resolution, has become a core imaging tool for lesion detection and treatment monitoring. However, clinical breast cancer MRI imaging has many technical challenges: First, data is scarce and highly individualized, with irregular lesion shapes and significant scale differences, leading to insufficient model generalization ability; second, images contain numerous noise artifacts, complex tissue structures, and blurred lesion edges, making them easily confused with normal tissue; third, traditional image enhancement techniques are mostly designed for single problems and lack adaptability to multimodal channel feature consistency and local occlusion scenarios; fourth, lesion identification often adopts a step-by-step approach of segmentation followed by classification, which suffers from error accumulation, and traditional reinforcement learning (RL) driven identification strategies require global exploration, resulting in low efficiency and unstable accuracy.

[0003] In existing technologies, image enhancement largely relies on simple operations such as cropping and flipping, making it difficult to simulate real-world scenarios in clinical settings, such as local tissue occlusion and multimodal channel differences. The feature extraction stage lacks targeted enhancement of salient lesion features, resulting in scattered intra-class features and insufficient inter-class discrimination. Recognition strategy optimization often depends on traditional RL or single classification models, failing to fully utilize prior information annotated by clinical experts, leading to recognition efficiency and accuracy that cannot meet clinical needs. Therefore, there is an urgent need for an integrated method that combines multi-dimensional enhancement, efficient feature extraction, and accurate strategy optimization. Summary of the Invention

[0004] This invention aims to address the problems of numerous noise artifacts, complex tissue structures, blurred lesion edges, large differences in lesion scale, and unstable automatic identification performance in existing breast cancer MRI images. It provides a breast cancer MRI image enhancement and automatic lesion identification method based on an attention mechanism. It integrates the advantages of radiomics features and deep learning models, enhances the saliency of tumor regions through a multi-level attention mechanism, and achieves adaptive enhancement of MRI images and accurate automatic lesion identification by combining multi-source technology innovation.

[0005] In a first aspect, embodiments of the present invention provide a method for breast cancer MRI image enhancement and automatic lesion identification based on an attention mechanism, comprising:

[0006] Acquire multi-timepoint breast MRI image samples containing multiple treatment stages, and preprocess the image samples;

[0007] The preprocessed image samples are then subjected to random window enhancement and random channel commutative enhancement in sequence to obtain enhanced image samples.

[0008] The enhanced image samples are input into an improved convolutional neural network and optimized using a deep metric learning loss function based on surrogate NCA to extract lesion embedding features.

[0009] Based on the lesion embedding features, a moment matching game framework is constructed with the expert-annotated lesion state distribution as the optimization objective.

[0010] Based on the moment matching game framework, an inverse reinforcement learning strategy without reinforcement learning is adopted, and the parameters of the lesion recognition model containing the convolutional neural network are iteratively optimized through the regretless moment matching algorithm to obtain the optimized lesion recognition model.

[0011] The nature of the lesion is determined based on the optimized lesion identification model, and the lesion nature determination result and corresponding confidence score are output.

[0012] In a preferred embodiment, the step of sequentially performing random window enhancement and random channel commutative enhancement on the preprocessed image samples to obtain enhanced image samples includes:

[0013] Random occlusion is applied to the boundary region of the preprocessed image sample, and the average value of the pixels in the occluded region is used to fill it to obtain the image sample enhanced by random window.

[0014] Based on the image sample enhanced by the random window, any channel in the multimodal weighted channels contained therein is randomly selected, and the pixel value matrix of the selected channel is copied and used to replace the pixel value matrix of the other channels to generate an image sample with randomly interchangeable channels.

[0015] In a preferred embodiment, the improved convolutional neural network uses DenseNet as its backbone network and includes convolutional layers, a spatial attention module, and a channel attention module, wherein...

[0016] The backbone network is used to extract features from the enhanced image samples and generate feature maps;

[0017] The spatial attention module is used to perform max pooling and average pooling on the feature map respectively, and after fusing the pooling results, it is processed by convolution and activated by the Sigmoid function to generate a spatial attention weight map, which is used to perform spatial domain weighting on the feature map.

[0018] The channel attention module is used to perform global average pooling of the channel dimension on the feature map, and the pooling result is transformed by a multilayer perceptron and activated by the sigmoid function to generate a channel attention weight vector, which is used to perform channel domain weighting on the feature map.

[0019] In a preferred embodiment, the step of extracting lesion embedding features by optimizing a deep metric learning loss function based on surrogate NCA includes:

[0020] The feature map, after being weighted by the spatial attention module and the channel attention module, is then encoded using the improved convolutional neural network to output the lesion embedding feature vector corresponding to each image sample.

[0021] Construct a learnable surrogate weight matrix corresponding to all lesion categories, and calculate the normalized distance between the lesion embedding feature vector corresponding to each image sample and the surrogate vectors of each category in the learnable surrogate weight matrix;

[0022] Based on the normalized distance, the agent NCA loss value is calculated using cross-entropy loss;

[0023] With the goal of minimizing the surrogate NCA loss, the parameters of the convolutional neural network and the surrogate weight matrix are simultaneously updated iteratively through the gradient backpropagation algorithm to obtain lesion embedding features that are compact within classes and separated between classes.

[0024] As a preferred embodiment, the step of constructing a moment-matching game framework based on the lesion embedding features and optimizing the distribution of lesion states labeled by experts includes:

[0025] Based on the lesion embedding features, the identification strategy and expert annotation strategy of the lesion identification model are defined;

[0026] A moment matching game framework is constructed, wherein the objective function of the moment matching game framework is to minimize the cumulative reward gap between the identification strategy and the expert labeling strategy. The cumulative reward gap is calculated by the expected difference of the cumulative reward of each strategy under the reward function, and the reward function is generated by mapping the lesion embedding features and the clinical diagnostic labels.

[0027] As a preferred embodiment, the objective function of the moment matching game framework is:

[0028]

[0029]

[0030] in, The lesion identification strategy of the model : Strategy space, which contains the set of all possible lesion identification strategies; Reward function; : Reward function space; :Strategy In the reward function The expected cumulative reward; Expert strategy; Policy-based The generated trajectory Mathematical expectation operation; Lesion identification trajectory; Image sequence length; : No. The state at time 1, in single MRI scan recognition, is the 1st... The feature map output by the convolutional layer is the first layer in the sequence image. Frame feature representation; : No. In lesion identification, the actions at any given moment represent the model's transformation of features or decision-making operations. : No. Moment State With action The corresponding instant reward.

[0031] In a preferred embodiment, the step of employing an inverse reinforcement learning strategy without reinforcement learning according to the moment matching game framework, and iteratively optimizing the parameters of the lesion identification model containing the convolutional neural network using a regretless moment matching algorithm, includes:

[0032] Initialize an experience buffer, wherein the experience buffer is used to store the distribution of lesion states annotated by experts;

[0033] During the iteration process, the state and time step are randomly sampled from the distribution of lesion states annotated by experts, and after performing random actions on the sampled state, the historical best strategy is used to complete the subsequent identification trajectory.

[0034] Calculate the cumulative reward for the subsequent identified trajectory, and store the data triple consisting of the current state, the executed action, and the cumulative reward into the experience buffer;

[0035] The parameters of the lesion identification model are updated using a regretless algorithm based on all the data triples stored in the experience buffer.

[0036] In a preferred embodiment, the parameters of the lesion identification model are updated using a regretless algorithm based on all data triples stored in the experience buffer, including:

[0037] Using all the data triples as training samples, an optimization objective function with entropy regularization is constructed;

[0038] The optimization objective function is solved using a regretless algorithm, and the parameters of the lesion identification model are iteratively updated.

[0039] Calculate the cumulative expected reward gap of the lesion identification model strategy in two adjacent iterations. If the gap is less than a preset convergence threshold, stop parameter updating; otherwise, repeat the iteration until the model performance converges.

[0040] In a preferred embodiment, the nature of the lesion is determined based on the optimized lesion identification model, and the lesion nature determination result and corresponding confidence score are output, including:

[0041] The lesion embedding features are input into the classification module of the lesion recognition model to obtain the probability distribution of each lesion category;

[0042] The lesion category corresponding to the maximum probability value in the probability distribution is output as the lesion nature determination result, and the confidence score of the determination result is calculated based on the probability distribution.

[0043] Secondly, embodiments of the present invention also provide an attention-based MRI image enhancement and lesion automatic identification system for breast cancer, comprising:

[0044] The data acquisition and preprocessing module is used to acquire breast MRI image samples at multiple time points covering multiple treatment stages and to preprocess the image samples.

[0045] The data augmentation module is used to perform random window augmentation and random channel commutative augmentation on the preprocessed image samples in sequence to obtain the augmented image samples;

[0046] The lesion embedding feature extraction module is used to input the enhanced image samples into the improved convolutional neural network and extract lesion embedding features by optimizing the deep metric learning loss function based on surrogate NCA.

[0047] The moment matching game framework construction module is used to construct a moment matching game framework with the expert-annotated lesion state distribution as the optimization objective based on the lesion embedding features.

[0048] The lesion identification model optimization module is used to perform iterative optimization of the parameters of the lesion identification model containing the convolutional neural network according to the moment matching game framework, using an inverse reinforcement learning strategy without reinforcement learning, and through a regretless moment matching algorithm, to obtain the optimized lesion identification model.

[0049] The lesion nature determination module is used to determine the nature of lesions based on the optimized lesion recognition model and output the lesion nature determination result and the corresponding confidence score.

[0050] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0051] One or more processors;

[0052] Storage device for storing one or more programs;

[0053] When the one or more programs are executed by the one or more processors, the one or more processors implement the attention mechanism-based breast cancer MRI image enhancement and lesion automatic identification method described in any embodiment of the present invention.

[0054] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the attention-based MRI image enhancement and lesion automatic identification method for breast cancer as described in any embodiment of the present invention.

[0055] Compared with the prior art, the present invention achieves the following beneficial effects:

[0056] (1) This invention uses random window enhancement technology to simulate scenarios such as local tissue overlap and blood vessel obstruction in clinical MRI, forcing the model to focus on the core features of the lesion, improving the ability to capture key local features, and alleviating the problem of insufficient generalization ability caused by data scarcity.

[0057] (2) This invention uses random channel exchangeable enhancement technology to adapt to the feature differences of MRI multimodal weighted channels, so as to enhance the expression of lesion features in a single modality channel, reduce intramodal differences, and improve the model's adaptability to multimodal data.

[0058] (3) This invention proposes a deep metric learning optimization process based on ProxyNCA loss. By calculating the cross-entropy loss through the Euclidean distance between the embedded features and the class weight matrix, the network is guided to learn salient lesion features that are compact within the class and separated between the classes, thus solving the problem of insufficient lesion feature discrimination.

[0059] (4) This invention proposes an unregretful moment matching algorithm through an inverse reinforcement learning optimization strategy without reinforcement learning. It utilizes the lesion state distribution with expert annotation to optimize the learning strategy, avoids the global exploration of traditional reinforcement learning, and achieves accurate and efficient lesion identification and nature determination (such as tumor / non-tumor, pCR / non-pCR, tumor grade, etc.).

[0060] (5) This invention reduces the accumulation of errors in traditional step-by-step schemes by constructing an integrated architecture of image enhancement, feature extraction, strategy optimization and lesion determination, thereby improving the overall accuracy and stability of identification and providing a reliable technical tool for clinical auxiliary diagnosis and treatment response prediction. Attached Figure Description

[0061] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0062] Figure 1 This is a flowchart of the breast cancer MRI image enhancement and lesion automatic identification method based on attention mechanism provided in the embodiments of the present invention;

[0063] Figure 2 This is a framework diagram of the breast cancer MRI image enhancement and lesion automatic identification method based on attention mechanism provided in the embodiments of the present invention;

[0064] Figure 3 This is an image region of interest marker map provided in an embodiment of the present invention;

[0065] Figure 4 These are images of circulating tumor cells under immunofluorescence staining provided in embodiments of the present invention;

[0066] Figure 5 This is a schematic diagram of the structure of the attention mechanism-based breast cancer MRI image enhancement and lesion automatic identification system provided in an embodiment of the present invention;

[0067] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0068] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0069] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations (or steps) can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the figures. The process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0070] Example 1

[0071] like Figure 1The diagram shows a flowchart of a breast cancer MRI image enhancement and lesion automatic identification method 100 based on attention mechanism provided in Embodiment 1 of the present invention. The method 100 specifically includes the following steps:

[0072] S110. Obtain multi-timepoint breast MRI image samples containing multiple treatment stages, and preprocess the image samples.

[0073] Preferably, in this embodiment of the invention, multiple time-point MRI samples (including pre-treatment, during-treatment, and post-treatment) of multiple HER2-positive early or locally advanced breast cancer patients are collected to construct a dataset containing more than 600 effective samples. This dataset can be used alone for lesion identification or in combination with treatment response prediction models (such as multi-omics feature fusion prediction models) to provide data support for clinical treatment effect evaluation (such as pathological complete response rate pCR prediction).

[0074] Furthermore, the collected breast MRI image samples underwent routine preprocessing, including random resizing and cropping, 50% probability horizontal flipping, and random translation enhancement.

[0075] The target size for random size adjustment and cropping is 384×384 pixels (to meet DenseNet input requirements); the translation range for random translation enhancement is ±15 pixels, and the area exceeding the boundary after translation is zero-filled; the original MRI image is first subjected to artifact removal (motion artifacts are smoothed by Gaussian filtering and metal artifact areas are removed by thresholding), and then the pixel values ​​are normalized to the [0,1] interval.

[0076] S120. Perform random window enhancement and random channel exchangeable enhancement on the preprocessed image samples in sequence to obtain enhanced image samples.

[0077] Preferably, addressing the issues of scarce breast cancer MRI data, significant individual variability, and irregular lesion morphology, this invention, in step S110, integrates multi-dimensional enhancement technology on top of the conventional preprocessing of the acquired breast MRI image samples. The specific steps are as follows:

[0078] (a) Random window enhancement: Random occlusion is applied to the boundary region of the preprocessed image sample, and the average value of the pixels in the occlusion region is used to fill it to obtain the image sample enhanced by random window. The boundary region refers to the area 10% to 15% of the size from the edge of the image (e.g., the boundary region of a 384×384 pixel image is a ring-shaped area 38 to 57 pixels from the edge). The occlusion operation only randomly selects rectangular occlusion blocks in this region to avoid occluding the core area of ​​the lesion.

[0079] Specifically, for scenarios that may exist in clinical MRI, such as local tissue overlap and blood vessel obstruction, a masking technique is used to randomly mask the image boundary region, with the masked area accounting for a certain percentage. Follows a uniform distribution function Uni The occluded area is filled with the average of neighboring pixels to avoid introducing abrupt noise. This operation simulates the real clinical situation where the lesion is occluded by local normal tissue, forcing the model to focus on the core features of the lesion rather than redundant edge information, thus improving the ability to capture key local features of the lesion.

[0080] (b) Random Channel Exchangeable Enhancement: Based on the image sample enhanced by the random window, any channel among its multimodal weighted channels is randomly selected. The pixel value matrix of the selected channel is copied and used to replace the pixel value matrices of the other channels, generating a random channel exchangeable enhanced image sample. Specifically, the aforementioned MRI images typically contain multimodal weighted channels such as T1, T2, and DWI. Different modal channels emphasize different aspects of lesion characterization (e.g., T1-weighted highlights anatomical structures, while T2-weighted highlights edema areas). For multimodal channel data, a single channel is randomly selected. (in Mapping MRI T1-weighted channel, Mapping T2 weighted channels, Map the DWI (diffusion-weighted imaging) channel, copy the pixel information of that channel to replace the other two channels, and generate an enhanced image. ,in, : No. The index in MRI samples is The pixel matrix of the modal channel contains the specific features of the lesion in that modality (such as the anatomical features of the T1-weighted channel and the edema region features of the T2-weighted channel).

[0081] This random channel allows for the exchange of enhancement operations to strengthen the expression of lesion features in a single modality, reduce the representational differences between different modalities, retain the core lesion information of each channel, enhance the consistency of features between channels, and improve the model's adaptability to multimodal MRI data.

[0082] The aforementioned random window enhancement and random channel commutative enhancement can compensate for the insufficiency of the original dataset, enabling the subsequent recognition network to still have robust learning capabilities in data-scarce scenarios, thereby reducing the risk of overfitting.

[0083] S130. The enhanced image samples are input into the improved convolutional neural network, and the lesion embedding features are extracted by optimizing the deep metric learning loss function based on surrogate NCA.

[0084] Preferably, the improved convolutional neural network described above uses DenseNet as its backbone network, including convolutional layers, a spatial attention module, and a channel attention module. The backbone network is used to extract features from the enhanced image samples to generate feature maps. The spatial attention module performs max pooling and average pooling on the feature maps, merges the pooling results, performs convolution and activates them with a sigmoid function to generate a spatial attention weight map, which is then used to perform spatial domain weighting on the feature maps. The channel attention module performs global average pooling along the channel dimensions on the feature maps, transforms the pooling results using a multilayer perceptron and activates them with a sigmoid function to generate a channel attention weight vector, which is then used to perform channel domain weighting on the feature maps.

[0085] Specifically, an attention weight map is generated based on the improved DenseNet backbone network through a spatial attention mechanism. To enhance the characteristic response of the tumor region and suppress background noise interference, spatial attention is calculated as follows:

[0086]

[0087] in, Spatial attention weight map, dimension and convolutional layer output feature map Consistency is used to assign weights to each spatial location of the feature map, thereby enhancing the feature response of the tumor region and suppressing background noise interference. The output feature map of a certain layer in a convolutional neural network contains hierarchical features of breast cancer MRI images (such as edges, textures, lesion candidate region features, etc.). The sigmoid activation function maps input values ​​to... The interval is used to generate attention weights (the closer the weight is to 1, the more important the feature at the corresponding position / channel is). Two-dimensional convolution operation: used for spatial feature extraction and dimensionality adjustment of feature maps. Core parameters include kernel size, stride, and padding method to adapt to the scale characteristics of breast cancer lesion features; MaxPool: max pooling operation, which takes the maximum value of a local region of the feature map, retains key features while reducing the dimension of the feature map, improving model computational efficiency and translation invariance; AvgPool: average pooling operation, which takes the average value of a local region of the feature map, smooths the feature response, reduces noise interference, and assists in the global optimization of attention weights.

[0088] The channel attention mechanism adaptively adjusts the feature weights of each channel. The calculation is as follows:

[0089]

[0090] in, Channel attention weight vector, dimensions and feature map The number of channels is consistent, which is used to adaptively adjust the importance of each channel feature and highlight the channel features that are key to lesion identification; MLP: Multilayer Perceptron (fully connected network), which consists of two fully connected layers and is used to perform nonlinear transformation on the aggregated features of channels and learn the dependencies between channels; Feature map The average pooling result in the channel dimension is obtained by averaging all spatial locations of each channel to obtain a single-valued channel aggregated feature, which reflects the global response strength of the channel. : The weight matrix of the first fully connected layer of the multilayer perceptron, used to map the channel aggregated features to a low-dimensional feature space, with the dimension being "number of channels × hidden layer dimension".

[0091] The weight matrix of the second fully connected layer of the multilayer perceptron is used to map low-dimensional features back to the original channel number dimension and output channel attention weights, with the dimension being "hidden layer dimension × number of channels". Linear rectified activation function: used to introduce nonlinearity into the multilayer perceptron, enhance the model's ability to fit complex dependencies of channel features, and alleviate the gradient vanishing problem.

[0092] Furthermore, the feature maps weighted by the spatial attention module and the channel attention module are used for feature encoding through an improved convolutional neural network to output the lesion embedding feature vector corresponding to each image sample. A learnable surrogate weight matrix corresponding to all lesion categories is constructed, and the normalized distance between the lesion embedding feature vector corresponding to each image sample and the surrogate vectors of each category in the learnable surrogate weight matrix is ​​calculated. Based on the normalized distance, the surrogate NCA loss value is calculated through cross-entropy loss. With the goal of minimizing the surrogate NCA loss value, the parameters of the convolutional neural network and the surrogate weight matrix are iteratively updated simultaneously through the gradient backpropagation algorithm to obtain lesion embedding features that are compact within classes and separated between classes.

[0093] Specifically, ProxyNCA loss is used to optimize the deep metric learning process. Cross-entropy loss is calculated based on the Euclidean distance between embedded features and class weight matrices, guiding the network to learn salient lesion features that are compact within classes and separated between classes. The loss function is defined as:

[0094]

[0095] in, : The proxy NCA loss function is used to optimize the deep metric learning process and guide the network to learn the embedding features of breast cancer lesions that are compact within classes and separated between classes; : The number of samples in a single training batch, i.e., the number of breast cancer MRI image samples involved in the current loss calculation; : No. A sample of breast cancer MRI images, containing multidimensional features related to the lesion; :sample The embedded feature vector, after being processed by a convolutional neural network and feature encoding, has a fixed-length feature dimension and is used to characterize the abstract lesion features of the sample. :sample Category The corresponding proxy weight vector, dimension and embedded features Consistency serves as the feature center for this category, guiding the clustering of features among similar samples; :sample The true category label is determined by breast cancer-related categories (such as tumor / non-tumor, pathological complete response (pCR) or incomplete response (non-pCR), tumor grade I / II / III, etc.). The squared Euclidean distance is used to calculate embedded features. With proxy weight vector Spatial distance, the smaller the distance, the higher the feature similarity; Temperature parameter, ranging from 0.1 to 0.5, is used to adjust the smoothness of the probability distribution in the loss function. The smaller the value, the stronger the inter-class differentiation, which is suitable for the characteristic distribution of breast cancer lesion categories; The total number of categories of breast cancer MRI images, determined based on the identification task (e.g., tumor / non-tumor task). In non-pCR tasks In the task of tumor grading wait); : No. The proxy weight vectors for each category, dimensions, and embedded features. Consistency forms the feature center set for all categories, used to calculate the similarity between samples and each category; exp: the exponential function, used to map the negative square of the Euclidean distance to a positive value, serving as an exponential representation of similarity, facilitating subsequent normalization calculations; log: the natural logarithm function, used to map the normalized similarity to a logarithmic space, forming the core component of cross-entropy loss, enabling effective backpropagation of the loss value.

[0096] S140. Based on the lesion embedding features, construct a moment matching game framework with the expert-annotated lesion state distribution as the optimization objective.

[0097] Preferably, the distribution of lesion areas is based on expert annotations (i.e., expert status distribution). Starting from the objective function, the identification strategy is optimized by minimizing the cumulative reward gap between the expert strategy and the model strategy. This cumulative reward gap is calculated using the expected difference in cumulative rewards for each strategy under the reward function, which is generated by mapping the lesion embedding features to the clinical diagnostic labels. Specifically, the objective function is:

[0098]

[0099]

[0100] in, The lesion identification strategy of the model corresponds to the end-to-end identification model composed of a convolutional neural network and a classification head, and outputs the lesion category determination result, which belongs to the strategy space. Elements in; The strategy space contains a set of all possible lesion identification strategies, covering model prediction rules under different network parameter configurations. The reward function, generated by mapping salient features of breast cancer lesions to clinical labels, is used to quantify the quality of the strategy identification results and belongs to the reward function space. Elements in; The reward function space contains the set of all possible reward functions, and its core is to establish a scoring rule for the correlation between features and clinical labels. :Strategy In the reward function The cumulative reward expectation is used to measure the accuracy of the strategy in lesion identification; a higher value indicates better identification performance. Expert strategy, constructed based on lesion area data annotated by clinicians, represents the optimal lesion identification standard and is used to guide the optimization of model strategy; Expert Strategy In the reward function The expected cumulative reward is used as the benchmark for optimizing the model strategy; Policy-based The generated trajectory The mathematical expectation operation is used to calculate the average reward of the policy across all possible recognition trajectories; Lesion identification trajectory, which corresponds to the complete reasoning process of the model from feature extraction to category determination for a single MRI image, including state sequence and action sequence; Image sequence length, for a single MRI image. This represents the number of layers in the feature extraction network; for multi-frame image sequences, Number of frames; : No. The state at time 1, in single MRI scan recognition, is the 1st... The feature map output by the convolutional layer is the first layer in the sequence image. Frame feature representation; : No. In lesion identification, the actions at any given moment are the model's transformation of features or decision-making operations (such as convolution, fully connected mapping, and category determination). : No. Moment State With action The corresponding immediate reward is calculated by the reward function, which quantifies the effectiveness of the identification operation in this step; Expert status distribution is a probability distribution of status based on data of lesion areas annotated by clinicians, which includes common feature patterns and spatial location information of lesions.

[0101] S150. Based on the moment matching game framework, an inverse reinforcement learning strategy without reinforcement learning is adopted, and the parameters of the lesion recognition model containing the convolutional neural network are iteratively optimized through the regretless moment matching algorithm to obtain the optimized lesion recognition model.

[0102] Preferably, according to the moment matching game framework, an experience buffer is first initialized, wherein the experience buffer is used to store the lesion state distribution labeled by experts; during the iteration process, a state and time step are randomly sampled from the lesion state distribution labeled by experts, and after performing a random action on the sampled state, the subsequent identification trajectory is completed using the historical best strategy; the cumulative reward of the subsequent identification trajectory is calculated, and the data triple consisting of the current state, the executed action, and the cumulative reward is stored in the experience buffer; the parameters of the lesion identification model are updated based on all the data triples stored in the experience buffer using a regretless algorithm.

[0103] The parameters of the lesion identification model, updated using a regretless algorithm based on all data triples stored in the experience buffer, include:

[0104] Using all data triples as training samples, construct an optimization objective function with entropy regularization term;

[0105] The objective function is solved using a regretless algorithm, and the parameters of the lesion identification model are iteratively updated.

[0106] Calculate the cumulative expected reward difference of the lesion identification model strategy in two adjacent iterations. If the difference is less than the preset convergence threshold, stop updating the parameters; otherwise, repeat the iteration until the model performance converges.

[0107] Specifically, initialization strategy With experience buffer The buffer stores the state-action pairs of lesion areas annotated by experts. and the corresponding feature mapping results, where, Expert state, i.e., the feature state corresponding to the lesion area marked by the clinician, serves as the optimal state sample for model learning; Expert actions, namely the decisions made by clinicians regarding the labeling of lesions (such as tumor or non-tumor determination), serve as the optimization target for model actions; Experience buffer, used to store expert state-action pairs. and generated during model iteration Data provides sample support for strategy updates.

[0108] During the iteration process, the distribution of expert states in random sampling The state in With time step In state Randomly execute actions Then, the historical best strategy was adopted. Complete the subsequent trajectory This avoids the global exploration required by traditional RL.

[0109] Calculate the cumulative reward of the trajectory suffix ,Will Store in buffer Update policy parameters using the False Rejection Principle (FTRL) algorithm:

[0110]

[0111]

[0112] in, : No. The cumulative reward for the trajectory suffix of time, that is, from Time's up The sum of instantaneous rewards at each moment is used to evaluate the overall value of subsequent recognition operations in that state; : The time index of the trajectory suffix, with a value range of Used for traversing and calculating from Accumulated rewards starting from a specific moment; : No. The updated model strategy, obtained through a regretless optimization algorithm, is superior to the previous strategy. It has superior recognition performance; Based on experience buffer The mathematical expectation operation of the middle sample is used to calculate the average performance of the strategy on the historical samples; Entropy regularization coefficient, used to balance the exploratory and exploitative aspects of a strategy, to prevent the model from overfitting expert samples; its value is usually in the range of 0.01-0.1. :Strategy Entropy is used to measure the randomness of a policy. A higher entropy value indicates a stronger exploratory nature of the policy, while a lower entropy value indicates that the policy is more inclined to take advantage of known effective operations. FTRL: Follow the Regularized Leader algorithm, a regret-free online learning algorithm used to update policy parameters based on experience buffer samples, ensuring that the accumulated regret of the policy is minimized during the iteration process.

[0113] The reward function is optimized through a discriminator during iteration. Distinguish the feature differences between expert trajectories and model trajectories, and iterate until policy performance converges (e.g., the cumulative reward difference between two adjacent iterations is less than a threshold). ).

[0114] S160. Determine the nature of the lesion based on the optimized lesion identification model and output the lesion nature determination result and the corresponding confidence score.

[0115] Preferably, the lesion embedding features in step S130 above are used as input, and the probability distribution of each lesion category is predicted by the classification module of the lesion recognition model.

[0116] The lesion category corresponding to the maximum probability value in the probability distribution is output as the lesion nature determination result, and the confidence score of the determination result is calculated based on the probability distribution.

[0117] Specifically, taking the extracted lesion embedding features as input, the classification head outputs the lesion nature determination result (such as tumor / non-tumor, pathological complete response / incomplete response, tumor grade, etc.), and simultaneously outputs the feature confidence score. The confidence level is calculated as follows:

[0118]

[0119] in, The confidence score for lesion identification results, with a value range of [value missing]. The closer the score is to 1, the higher the certainty of the model's judgment result, which is used to assist clinical decision-making; The first output of the classification header The probability of each category, with values ​​ranging from 1 to 2. This reflects the likelihood that the model determines a sample belongs to that category; The maximum value among all category probabilities output by the classification head corresponds to the final lesion category determination result of the model; The total number of lesion categories identified, compared to step 3.3. Consistency (e.g., oncology / non-oncology tasks) non-pCR task wait).

[0120] The classification module consists of two fully connected layers and one Softmax layer. A Dropout layer and a ReLU activation function are inserted between the two fully connected layers, and the probability distribution of each category is finally output through the Softmax layer.

[0121] like Figure 3 and Figure 4 The image shows MRI images and cell images in a real-world application scenario. Experimental results demonstrate that this invention employs an end-to-end joint optimization architecture for feature extraction and lesion recognition, integrating the feature extraction process of the improved convolutional neural network with the recognition decision-making process. Compared to the traditional step-by-step approach of segmentation followed by classification, this architecture fundamentally avoids the cumulative problem of errors from the segmentation stage propagating to the recognition stage. Experimental verification using over 600 sets of MRI samples at multiple time points shows a 60% reduction in cumulative lesion recognition error, and an improvement in recognition accuracy and prediction stability of 15%-20% compared to traditional methods, fully validating the accurate recognition capability of this invention in real-world clinical scenarios.

[0122] Based on the above embodiments, the present invention achieves the following beneficial effects:

[0123] (1) The random window enhancement and random channel interchangeable enhancement techniques proposed in this invention solve the problems of local occlusion simulation and multimodal channel feature consistency respectively. Combined with random translation, amplitude scaling and other techniques, they fully compensate for the data scarcity defect, so that the model still has robust learning ability in data scarcity scenarios, the overfitting risk is reduced by more than 30%, and the generalization ability is significantly better than traditional enhancement methods.

[0124] (2) This invention guides deep metric learning through proxy NCA loss, and combines spatial attention and channel attention mechanisms to enhance the tumor region feature response and suppress background noise interference, thereby improving the intra-class feature compactness of lesions by more than 40% and the inter-class separation by more than 35%, effectively solving the feature confusion problem caused by blurred lesion edges and large scale differences.

[0125] (3) This invention uses an inverse reinforcement learning optimization strategy without reinforcement learning and a regretless moment matching algorithm to avoid global exploration by utilizing expert state distribution, thereby improving the efficiency of strategy iteration by more than 50%. The integrated architecture optimizes feature extraction and recognition end-to-end. Compared with the traditional step-by-step scheme, the cumulative recognition error is reduced by 60%, and the accuracy of lesion recognition and prediction is improved by 15%-20%. The confidence score can assist clinical decision-making and reduce the risk of misdiagnosis and missed diagnosis.

[0126] (4) This invention supports the processing of MRI samples at multiple time points (before, during and after treatment), and can be used alone for lesion identification or linked with multi-omics feature fusion prediction models; it can be adapted to different identification tasks (tumor / non-tumor, grade, treatment response assessment), without the need for significant adjustments to the network structure, and is easy to integrate into existing clinical diagnostic systems, providing efficient technical support for large-scale clinical sample analysis and individualized treatment.

[0127] Example 2

[0128] Figure 5 This is a schematic diagram of the structure of a breast cancer MRI image enhancement and lesion automatic identification system based on an attention mechanism, as provided in Embodiment 2 of the present invention. Figure 5 As shown, the system includes:

[0129] The data acquisition and preprocessing module 510 is used to acquire breast MRI image samples at multiple time points including multiple treatment stages, and to preprocess the image samples.

[0130] The data augmentation module 520 is used to perform random window augmentation and random channel exchangeable augmentation on the preprocessed image samples in sequence to obtain the augmented image samples.

[0131] The lesion embedding feature extraction module 530 is used to input the enhanced image sample into the improved convolutional neural network and extract lesion embedding features by optimizing the deep metric learning loss function based on surrogate NCA.

[0132] The moment matching game framework construction module 540 is used to construct a moment matching game framework with the expert-annotated lesion state distribution as the optimization objective based on the lesion embedding features.

[0133] The lesion recognition model optimization module 550 is used to perform iterative optimization of the parameters of the lesion recognition model containing the convolutional neural network according to the moment matching game framework, using an inverse reinforcement learning strategy without reinforcement learning, and through a regretless moment matching algorithm, to obtain an optimized lesion recognition model.

[0134] The lesion nature determination module 560 is used to determine the nature of lesions based on the optimized lesion recognition model and output the lesion nature determination result and the corresponding confidence score.

[0135] The attention-based MRI image enhancement and lesion automatic identification system for breast cancer provided in this embodiment can execute the attention-based MRI image enhancement and lesion automatic identification method for breast cancer provided in any of the above embodiments of this invention. It has the corresponding functions and beneficial effects of executing the attention-based MRI image enhancement and lesion automatic identification method for breast cancer. For detailed process, please refer to the relevant operations of the attention-based MRI image enhancement and lesion automatic identification method for breast cancer in the foregoing embodiments.

[0136] Example 3

[0137] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, and may also represent various forms of mobile devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0138] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0139] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0140] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the attention-based MRI image enhancement and lesion automatic identification method for breast cancer described above.

[0141] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0142] The above embodiments are merely illustrative examples and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for breast cancer MRI image enhancement and lesion automatic identification based on attention mechanism, characterized in that, The method comprises the following steps: acquire multi-time point breast MRI image samples containing multiple treatment stages, and pre-process the image samples; the pre-processed image samples are sequentially subjected to random window enhancement and random channel exchangeable enhancement to obtain enhanced image samples; the enhanced image samples are input into an improved convolutional neural network, and a deep metric learning loss function based on proxy NCA is used for optimization to extract lesion embedding features; based on the lesion embedding features, a moment matching game framework is constructed with the expert-annotated lesion state distribution as the optimization target, including: based on the lesion embedding features, defining the recognition strategy of the lesion recognition model and the expert annotation strategy; a moment matching game framework is constructed, and the cumulative reward difference between the recognition strategy and the expert annotation strategy is taken as the objective function, wherein the cumulative reward difference is calculated by the cumulative reward expectation difference of each strategy under the reward function, and the reward function is generated by mapping the lesion embedding features and the clinical diagnosis label; wherein the objective function of the moment matching game framework is: ； ； in, The lesion identification strategy of the model : Strategy space, which contains the set of all possible lesion identification strategies; Reward function; : Reward function space; :Strategy In the reward function The expected cumulative reward; Expert strategy; Policy-based The generated trajectory Mathematical expectation operation; Lesion identification trajectory; Image sequence length; : No. The state at time 1, in single MRI scan recognition, is the 1st... The feature map output by the convolutional layer is the first layer in the sequence image. Frame feature representation; : No. In lesion identification, the actions at any given moment represent the model's transformation of features or decision-making operations. : No. Moment State With action The corresponding instant reward; based on the moment matching game framework, an inverse reinforcement learning strategy without reinforcement learning is adopted, and the parameters of the lesion recognition model containing the convolutional neural network are iteratively optimized by a no-regret moment matching algorithm to obtain an optimized lesion recognition model; based on the optimized lesion recognition model, the lesion property is determined and the lesion property determination result and the corresponding confidence score are output.

2. The method of claim 1, wherein, the pre-processed image samples are sequentially subjected to random window enhancement and random channel exchangeable enhancement to obtain enhanced image samples, including: random occlusion is applied to the boundary region of the pre-processed image sample, and the mean value of the pixels in the occluded region is used for filling to obtain the image sample after random window enhancement; based on the image sample after random window enhancement, any channel in the multi-modal weighted channel contained in the image sample is randomly selected, the pixel value matrix of the selected channel is copied and replaces the pixel value matrix of the remaining channels to generate the image sample after random channel exchangeable enhancement.

3. The method of claim 2, wherein, The improved convolutional neural network takes DenseNet as the backbone network, including convolutional layers, spatial attention modules and channel attention modules, wherein the backbone network is used for feature extraction of the enhanced image sample to generate a feature map; the spatial attention module is used to perform maximum pooling and average pooling on the feature map respectively, fuse the pooling results, and perform convolution processing and Sigmoid function activation to generate a spatial attention weight map, which is used for spatial domain weighting of the feature map; the channel attention module is used to perform global average pooling in the channel dimension on the feature map, and the pooling result is transformed by a multi-layer perceptron and activated by a Sigmoid function to generate a channel attention weight vector, which is used for channel domain weighting of the feature map.

4. The method of claim 3, wherein, the deep metric learning loss function based on proxy NCA is used for optimization to extract lesion embedding features, including: The feature map weighted by the spatial attention module and the channel attention module is encoded by the improved convolutional neural network to output a lesion embedding feature vector corresponding to each image sample; A learnable proxy weight matrix corresponding to all lesion categories is constructed, and a normalized distance between the lesion embedding feature vector corresponding to each image sample and each category proxy vector in the learnable proxy weight matrix is calculated; Based on the normalized distance, a proxy NCA loss value is calculated by cross-entropy loss; The parameters of the convolutional neural network and the proxy weight matrix are simultaneously updated by a gradient backpropagation algorithm to obtain a lesion embedding feature with intra-class compactness and inter-class separation, aiming to minimize the proxy NCA loss value.

5. The method of claim 1, wherein, According to the matrix matching game framework, a non-reinforcement learning inverse reinforcement learning strategy is adopted, and the parameters of the lesion recognition model containing the convolutional neural network are iteratively optimized by a no-regret matrix matching algorithm, including: Initializing an experience buffer, wherein the experience buffer is used to store expert-labeled lesion state distribution; In the iteration process, a state and a time step are randomly sampled from the expert-labeled lesion state distribution, and after a random action is performed on the sampled state, a subsequent recognition trajectory is completed along the history optimal policy; The cumulative reward of the subsequent recognition trajectory is calculated, and a data triple consisting of the current state, the executed action and the cumulative reward is stored in the experience buffer; Based on all data triples stored in the experience buffer, the parameters of the lesion recognition model are updated by a no-regret algorithm.

6. The method of claim 5, wherein, Based on all data triples stored in the experience buffer, the parameters of the lesion recognition model are updated by a no-regret algorithm, including: Taking all data triples as training samples, an optimization objective function containing an entropy regularization term is constructed; The optimization objective function is solved by a no-regret algorithm to iteratively update the parameters of the lesion recognition model; The cumulative reward expectation gap of the lesion recognition model strategy in adjacent two iterations is calculated, if the gap is less than a preset convergence threshold, the parameter updating is stopped, otherwise the iteration is repeated until the model performance converges.

7. The method of claim 1, wherein, According to the optimized lesion recognition model, the lesion property is determined and the lesion property determination result and the corresponding confidence score are output, including: The lesion embedding feature is input into the classification module of the lesion recognition model to obtain a probability distribution belonging to each lesion category; The lesion category corresponding to the maximum value of the probability in the probability distribution is output as the lesion property determination result, and the confidence score of the determination result is calculated based on the probability distribution.

8. An attention mechanism based breast cancer MRI image enhancement and lesion automatic identification system, configured to implement the attention mechanism based breast cancer MRI image enhancement and lesion automatic identification method according to any one of claims 1-7, characterized in that, including: A data acquisition and preprocessing module is configured to acquire multiple time point breast MRI image samples containing multiple treatment stages, and to preprocess the image samples; A data enhancement module is configured to sequentially perform random window enhancement and random channel exchange enhancement on the preprocessed image samples to obtain enhanced image samples; A lesion embedding feature extraction module is configured to input the enhanced image samples into an improved convolutional neural network, and to extract lesion embedding features by proxy NCA-based deep metric learning loss function optimization. A module for constructing a matrix matching game framework according to the lesion embedding features, and the matrix matching game framework is constructed with the expert-labeled lesion state distribution as an optimization objective; A module for optimizing the lesion identification model according to the matrix matching game framework, and the parameters of the lesion identification model containing the convolutional neural network are iteratively optimized by using a no-reinforcement learning inverse reinforcement learning strategy and a no-regret matrix matching algorithm to obtain an optimized lesion identification model; A module for determining the lesion properties according to the optimized lesion identification model and outputting the lesion property determination results and corresponding confidence scores.

Images