Attention-based feature extraction methods, devices, and computer equipment

Abstract

This invention relates to the field of digital medical technology, and in particular to a feature extraction method based on an attention mechanism, which can be applied to image feature extraction and other application scenarios in the medical field. The method includes: preprocessing an original feature map to obtain a first feature map; inputting the first feature map into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module; and concatenating the original feature map and the second feature map to obtain the final feature extraction result. This improves the feature extraction model's ability to extract information about lesions with significant differences in size and shape, thereby improving the accuracy of the detection results.

Description

Technical Field

[0001] This invention relates to the field of digital medical technology, and in particular to a feature extraction method, apparatus, and computer device based on an attention mechanism. Background Technology

[0002] With advancements in medical imaging technology, particularly the continuous upgrading of medical imaging systems and the enhancement of data storage capabilities, pathological image data has experienced explosive growth. Compared to small sample data, a large volume of pathological images can uncover more potential information and patterns. For example, because cervical cancer pathological diagnosis requires observation and diagnosis of cellular-level medical images, a single sample often requires multiple slides. The processes of slide preparation, staining, diagnosis, and reporting are time-consuming, characterized by low automation and long diagnostic times. Deep learning methods are widely used in cervical cancer assisted screening to alleviate the diagnostic burden on pathologists. For instance, deep neural networks are used to classify, segment, and detect targets in cervical cancer pathological images to assist in medical diagnosis. Computer equipment can build target recognition models based on deep neural networks to identify lesions within pathological images. Currently, in pathological image recognition tasks, target recognition models are typically used to extract features from pathological images, obtain corresponding feature maps, and predict the location and category information of lesions in the pathological image based on these feature maps.

[0003] However, pathological images often contain lesions of varying sizes and shapes. Target detection models have poor information extraction capabilities for these lesions, resulting in a high false positive rate in practical applications. This means that normal areas are frequently identified as lesions. Furthermore, some minor lesions, due to their small size, may be missed, leading to low accuracy in the detection results. Summary of the Invention

[0004] The main objective of this application is to provide a feature extraction method, apparatus, and computer device based on an attention mechanism, which can solve the technical problem in existing digital medical technologies where pathological images often contain lesions with large differences in size and shape, and the target recognition model has poor information extraction capabilities for these lesions with large differences in size and shape, resulting in low accuracy of detection results.

[0005] This application provides a feature extraction method based on an attention mechanism, including:

[0006] The original feature map is preprocessed to obtain the first feature map;

[0007] The first feature map is input into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module;

[0008] The original feature map and the second feature map are concatenated to obtain the final feature extraction result.

[0009] Furthermore, before inputting the first feature map into the pre-trained feature extraction model to generate the second feature map, the method includes:

[0010] Acquire training data, which includes training samples and labels corresponding to the training samples;

[0011] The training samples are processed by the feature extraction model to be trained to obtain the prediction results and attention vector of the feature extraction model;

[0012] Based on the prediction results of the training samples and the labels, the inherent loss function is obtained;

[0013] The total loss function is determined based on the inherent loss function and the attention vector;

[0014] The model parameters of the feature extraction model are updated based on the total loss function to obtain the target model parameters of the feature extraction model, thereby obtaining a feature extraction model with the target model parameters as model parameters.

[0015] Furthermore, the step of processing the training samples using the feature extraction model to be trained to obtain the prediction result and attention vector of the feature extraction model includes:

[0016] The training samples are processed by the feature extraction model to obtain the hidden state of the feature extraction model;

[0017] The attention vector of the feature extraction model is obtained based on the hidden state.

[0018] Furthermore, the step of determining the total loss function based on the inherent loss function and the attention vector includes:

[0019] The attention vector entropy is obtained based on the attention vector.

[0020] The attention distribution loss function is obtained based on the attention vector entropy.

[0021] The total loss function is obtained based on the attention distribution loss function and the inherent loss function.

[0022] Furthermore, the step of updating the model parameters of the feature extraction model based on the total loss function to obtain the target model parameters of the feature extraction model includes:

[0023] The gradient of the model parameters is obtained through the feature extraction model based on the total loss function and the training samples;

[0024] The model parameters of the feature extraction model are updated based on the gradient of the model parameters.

[0025] Further, the step of inputting the first feature map into a pre-trained feature extraction model to generate the second feature map includes:

[0026] The first feature map is reshaped to obtain the third feature map;

[0027] The third feature map is input into three different fully connected (FC) layers to obtain a first result corresponding to the FC layer, wherein the input and output dimensions of the FC layer remain unchanged.

[0028] Perform a reshape operation on the first result to obtain a first input sequence, a second input sequence, and a third input sequence;

[0029] Perform matrix multiplication and softmax calculation on the first input sequence and the second input sequence to generate the fourth feature map;

[0030] The second feature map is generated by performing matrix multiplication on the fourth feature map and the third input sequence.

[0031] Further, the step of preprocessing the original feature map to obtain the first feature map includes:

[0032] The original feature map is normalized to unify its size and specifications.

[0033] This application also provides a feature extraction device based on an attention mechanism, comprising:

[0034] The preprocessing module is used to preprocess the original feature map to obtain the first feature map;

[0035] The second feature map acquisition module is used to input the first feature map into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module;

[0036] The final feature extraction result acquisition module is used to concatenate the original feature map and the second feature map to obtain the final feature extraction result.

[0037] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.

[0038] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above methods.

[0039] Compared to existing technologies, this application provides a feature extraction method based on an attention mechanism for use in the digital healthcare field. The method includes: preprocessing an original feature map to obtain a first feature map; inputting the first feature map into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module; and concatenating the original feature map and the second feature map to obtain the final feature extraction result. In the digital healthcare field, this method can improve the ability of the feature extraction model to extract information about lesions with significant differences in size and shape, thereby improving the accuracy of detection results. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the steps of a feature extraction method based on an attention mechanism in one embodiment of this application;

[0041] Figure 2 This is a schematic diagram illustrating the steps of a feature extraction method based on an attention mechanism in another embodiment of this application;

[0042] Figure 3 This is a structural block diagram of an attention-based feature extraction device according to an embodiment of this application;

[0043] Figure 4 This is a schematic block diagram of an embodiment of the computer device of this application;

[0044] Figure 5 This is a flowchart illustrating the specific workflow of this application.

[0045] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0046] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0048] Reference Figure 1 An attention-based feature extraction method according to an embodiment of this application includes:

[0049] S1. Preprocess the original feature map to obtain the first feature map;

[0050] S2. Input the first feature map into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module;

[0051] S3. The original feature map and the second feature map are concatenated to obtain the final feature extraction result.

[0052] In step S1, the preprocessing involves normalizing the original feature map to unify its size and specifications. This application can be applied to the medical field; specifically, the original feature map in this application can be a pathological image. By normalizing the pathological image, the mean is subtracted from each pathological image to remove some common features, highlighting the differences in features between different pathological images. This helps the model learn and identify the main features in the pathological image, reduces the gradient during backpropagation, accelerates the convergence of contrastive learning pre-training and the learning speed of the network, and increases generalization ability. In this embodiment, the original feature map can be a historical medical imaging diagnostic report. Historical medical imaging diagnostic reports include radiological diagnostic reports, ultrasound diagnostic reports, nuclear medicine diagnostic reports, and interventional radiological diagnostic reports; among them, radiological diagnostic reports can include X-ray plain films, CT images, and MR images, etc.; ultrasound diagnostic reports can include traditional surface ultrasound, endoscopic ultrasound, and vaginal ultrasound, etc.; nuclear medicine diagnostic reports include ECT images, PET / CT images, PET / MR images, etc.

[0053] In steps S2 and S3, the planar attention module compensates for the lack of self-attention calculation for a single pathological image. Secondly, the planar attention module proposed in this application can be well embedded in existing models, especially in the current mainstream encoder-decoder model structure, where high-level language feature maps can better learn global information, which is significant for dense-prediction tasks. Finally, the proposed planar self-attention mechanism can be well integrated with various attention mechanisms in the current CV field, exhibiting excellent compatibility.

[0054] In one specific embodiment, when the image being processed is a CT image of the liver, this CT image is used as the original feature map and subjected to normalization and other processing to obtain a first CT feature map. Then, the first CT feature map is input into a pre-trained feature extraction model to generate a second CT feature map, wherein the feature extraction model includes a planar attention module. Finally, the CT image and the second CT feature map are stitched together to obtain the final CT feature extraction result. The planar attention module can compensate for the lack of self-attention calculation on a single CT image, which can improve the feature extraction model's ability to extract information about lesions with large differences in size and shape from CT images, thereby improving the accuracy of the detection results.

[0055] In one feasible implementation, before inputting the first feature map into a pre-trained feature extraction model to generate the second feature map, the method includes:

[0056] S201. Obtain training data, wherein the training data includes training samples and labels corresponding to the training samples;

[0057] S202. The training samples are processed by the feature extraction model to be trained to obtain the prediction results and attention vector of the feature extraction model;

[0058] S203. Based on the prediction results of the training samples and the labels, obtain the inherent loss function;

[0059] S204. Determine the total loss function based on the inherent loss function and the attention vector;

[0060] S205. Update the model parameters of the feature extraction model based on the total loss function to obtain the target model parameters of the feature extraction model, and then obtain the feature extraction model with the target model parameters as model parameters.

[0061] In step S201, the feature extraction model uses mini-batch training data for each training session. Each mini-batch of training data results in very small updates to the model parameters, so a large number of mini-batch data are needed to converge the model parameters.

[0062] In step S202, in an exemplary embodiment, processing the training samples through the feature extraction model to obtain the attention vector of the feature extraction model may include: processing the training samples through the feature extraction model to obtain the hidden state of the feature extraction model; and obtaining the attention vector of the feature extraction model based on the hidden state.

[0063] In step S203, the feature extraction model's network contains multiple parameters, which are updated during model training to steer the model towards outputs that more closely resemble the sample labels. Therefore, when updating parameters, it's necessary to first calculate the deviation between the current model output and the sample labels—the loss function—and then calculate the updated parameters based on this deviation. Commonly used loss functions include 0-1 loss, absolute value loss, squared loss, and so on.

[0064] In step S204, in some embodiments, determining the total loss function based on the inherent loss function and the attention vector may include: obtaining the attention vector entropy based on the attention vector; obtaining the attention distribution loss function based on the attention vector entropy; and obtaining the total loss function based on the attention distribution loss function and the inherent loss function.

[0065] Based on the principles of neural network model training, any term in the loss function that contains network parameters will tend to decrease during training. Therefore, this application adds an attention vector entropy term to the inherent loss function.

[0066] In step S205, the parameter updates of deep learning models, such as feature extraction models, are generally performed using gradient descent. First, the gradient is calculated through backpropagation (following the chain rule for partial derivative calculation), and then the parameters are updated using gradient descent. Specifically, the gradient of the model parameters is obtained through the feature extraction model based on the total loss function and training samples; the model parameters of the feature extraction model are then updated according to the gradient of the model parameters.

[0067] In related technologies, deep learning using attention mechanisms is very slow when driven entirely by training samples, requiring numerous experiments to eventually converge the model. In this embodiment, steps S201-S205 involve a feature extraction model using the target model parameters as model parameters to predict the output from the input training samples. By adding an attention vector term to the inherent loss function, the feature extraction model optimizes the attention vector during training, improving its convergence speed and thus reducing the computational resource consumption when processing training samples.

[0068] In one feasible implementation, step S202, which involves processing the training samples using the feature extraction model to be trained to obtain the prediction results and attention vectors of the feature extraction model, includes:

[0069] S2021. The training samples are processed by the feature extraction model to obtain the hidden state of the feature extraction model;

[0070] S2022. Obtain the attention vector of the feature extraction model based on the hidden state.

[0071] In the above steps S2021 and S2022, processing the training samples through the feature extraction model to obtain the attention vector of the feature extraction model may include: processing the training samples through the feature extraction model to obtain the hidden state of the feature extraction model; and obtaining the attention vector of the feature extraction model based on the hidden state.

[0072] In one feasible implementation, step S204, which determines the total loss function based on the inherent loss function and the attention vector, includes:

[0073] S2041. Obtain the attention vector entropy based on the attention vector;

[0074] S2042. Obtain the attention distribution loss function based on the attention vector entropy;

[0075] S2043. Obtain the total loss function based on the attention distribution loss function and the inherent loss function.

[0076] In some embodiments, steps S2041 to S2043 above, which involve determining the total loss function based on the inherent loss function and the attention vector, may include: obtaining the attention vector entropy based on the attention vector; obtaining the attention distribution loss function based on the attention vector entropy; and obtaining the total loss function based on the attention distribution loss function and the inherent loss function.

[0077] Based on the principles of neural network model training, any term in the loss function that contains network parameters will tend to decrease during training. Therefore, this application adds an attention vector entropy term to the inherent loss function.

[0078] Specifically, the attention distribution vector in step S2041 is a normalized discrete distribution vector. According to the definition of the entropy of a discrete distribution, the entropy is relatively large when the probability distribution is relatively "uniform" and relatively small when the probability distribution is relatively "concentrated". Therefore, the magnitude of the entropy value of the attention vector is obtained through the probability distribution of the attention vector.

[0079] In steps S2042 and S2043, the attention distribution loss function is: Where, θ t The vector C(θ) used to represent the model parameters at time t. t ) represents the inherent loss function, and λ represents the preset regularization strength. T is used to represent the attention distribution loss function. x T is used to represent the number of symbols in the input sequence of the feature extraction model.y α is used to represent the number of symbols in the output sequence of the feature extraction model. ij The attention weight is used to represent the attention weight of the j-th symbol in the input sequence when the i-th symbol in the output sequence of the feature extraction model is obtained, where i is greater than or equal to 1 and less than or equal to T. y A positive integer, j is greater than or equal to 1 and less than or equal to T. x Positive integers. After adding the attention vector entropy term, the model parameters will tend to shift towards a direction that makes the attention distribution α... ij More focused state updates speed up model training, allowing the model to be iteratively updated to a convergent state using less training data.

[0080] In a feasible implementation, step S205, which updates the model parameters of the feature extraction model based on the total loss function to obtain the target model parameters of the feature extraction model, includes:

[0081] S2051. Obtain the gradient of the model parameters based on the total loss function and the training samples through the feature extraction model;

[0082] S2052. Update the model parameters of the feature extraction model according to the gradient of the model parameters.

[0083] In steps S2051 and S2052 above, the parameter updates of deep learning models, such as feature extraction models, are generally performed using gradient descent. First, the gradient is calculated through backpropagation (following the chain rule for partial derivative calculation), and then the parameters are updated using gradient descent. Specifically, the gradient of the model parameters is obtained through the feature extraction model based on the total loss function and training samples; the model parameters of the feature extraction model are then updated according to the gradient of the model parameters.

[0084] like Figure 2 As shown, in a feasible implementation, step S2, which involves inputting the first feature map into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module, includes:

[0085] S21. Perform a reshape operation on the first feature map to obtain the third feature map;

[0086] S22. Input the third feature map into three different fc layers to obtain the first result corresponding to the fc layer, wherein the input dimension and output dimension of the fc layer remain unchanged;

[0087] S23. Perform a reshape operation on the first result to obtain a first input sequence, a second input sequence, and a third input sequence;

[0088] S24. Perform matrix multiplication and softmax calculation on the first input sequence and the second input sequence to generate the fourth feature map;

[0089] S25. Perform matrix multiplication on the fourth feature map and the third input sequence to generate the second feature map.

[0090] In recent years, researchers have conducted in-depth explorations of attention mechanisms, proposing numerous attention mechanism modules. In the field of computer vision (CV), attention mechanisms can be broadly categorized into three types from an implementation perspective: channel attention, spatial attention, and non-local pixel-level attention. The most typical examples of channel attention mechanisms are SE-Net, proposed in 2017, and its successor, ECA-Net; while BAM and CBAM are the most typical representatives of spatial attention; and non-local pixel-level attention is a typical application of self-attention in CV.

[0091] Currently, attention mechanisms have been widely applied in the field of computer vision and have achieved excellent results. However, existing attention mechanism modules mostly learn the attention of the entire feature space. For example, channel attention is used to extract key channels, spatial attention is used to emphasize important regions in the entire space, and non-local pixel-level attention also learns the relationships between pixels from the entire space. Therefore, they all ignore the attention of each feature map.

[0092] To address the shortcomings of existing attention mechanisms mentioned above, this implementation proposes a planar attention mechanism for learning self-attention for each feature map. A flowchart is provided below. Figure 5 .

[0093] In step S21, the dimensions of the first feature map are (N, C, H, W), where N: Batch, the batch size, representing the number of images in a batch; Channel, the number of channels in an image; H: Height, representing the number of pixels in the vertical dimension of the image; W: Width, representing the number of pixels in the horizontal dimension of the image. The reshape operation refers to processing the first feature map using the reshape function. The reshape function is a MATLAB function that transforms a specified matrix into a matrix of a specific dimension, while keeping the number of elements in the matrix unchanged. The function can readjust the number of rows, columns, and dimensions of the matrix. The function syntax is B = reshape(A, size), which returns an n-dimensional array with the same elements as A, but the size of the reconstructed array is determined by the vector size. The dimensions of the first feature map are transformed from (N, C, H, W) into the third feature map, where the dimensions of the third feature map are (N*C, H*W).

[0094] In steps S22 and 23, the FC (Full Connection) layer means that every neuron in the previous layer is connected to every neuron in the next layer. In this step, the third feature map is input into three different FC layers. The input and output dimensions of the FC layers remain unchanged. The output of the FC layers is reshaped to generate the first, second, and third input sequences, respectively. The first input sequence is Q(N*C,H*W,1), the second input sequence is K(N*C,1,H*W), and the third sequence is V(N*C,1,H*W). The first input sequence is obtained through the Query branch, which uses a spatial attention mechanism; the second input sequence is obtained through the Key branch, which uses a channel attention mechanism; and the third input sequence is obtained through the Value branch.

[0095] In steps S24 and S25, specifically, matrix multiplication is performed on Q(N*C,H*W,1) and K(N*C,1,H*W), followed by a softmax calculation to generate the required fourth feature map (N*C,H*W,H*W). The fourth attention map is then multiplied with V(N*C,1,H*W), and the original feature map is added to obtain our final output result.

[0096] Compared to existing attention mechanisms, this implementation differs primarily in that the self-attention calculated is applied to each feature map. Specifically, existing channel attention calculates a weight for each feature map to select key features; spatial attention, on the other hand, calculates a weight map and multiplies all channels by this map. Therefore, spatial attention emphasizes attention across the entire feature space, reinforcing different regions within it. Non-local self-attention calculates the attention between different pixels in each channel, resulting in a weight map that reflects the attention between different pixels across the entire feature space. The proposed planar attention, as demonstrated by the above calculation process, calculates the attention between pixels on each feature map, thus enabling better extraction of key information from each feature map. This attention calculation method, building upon existing methods, addresses the lack of self-attention calculation for individual feature maps. Furthermore, the proposed planar attention method can be well integrated into existing models, especially in mainstream encoder-decoder models, where high-level language feature maps can better learn global information, which is significant for dense-prediction tasks. Finally, the proposed planar self-attention mechanism can be well integrated and combined with various attention mechanisms in the current CV field, and has good compatibility.

[0097] In one feasible implementation, step S1, which preprocesses the original feature map to obtain the first feature map, includes:

[0098] S11. Normalize the original feature map to unify its size and specifications.

[0099] Normalizing the original feature maps removes some common features by subtracting the mean from each original feature map, highlighting the differences in features between different samples. This helps the model learn and identify the main features in the original feature maps, reduces gradients during backpropagation, accelerates the convergence of contrastive learning pre-training and the learning speed of the network, and increases generalization ability.

[0100] Reference Figure 3 This application also provides a feature extraction device based on an attention mechanism, comprising:

[0101] Preprocessing module 1 is used to preprocess the original feature map to obtain the first feature map;

[0102] The second feature map acquisition module 2 is used to input the first feature map into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module;

[0103] The final feature extraction result acquisition module 3 is used to concatenate the original feature map and the second feature map to obtain the final feature extraction result.

[0104] Furthermore, the attention-based feature extraction device includes:

[0105] The first acquisition module is used to acquire training data, which includes training samples and labels corresponding to the training samples.

[0106] The sample processing module is used to process the training samples through the feature extraction model to be trained, and obtain the prediction results and attention vectors of the feature extraction model.

[0107] The intrinsic loss function acquisition module is used to obtain the intrinsic loss function based on the prediction results of the training samples and the labels.

[0108] The total loss function acquisition module is used to determine the total loss function based on the inherent loss function and the attention vector;

[0109] The target model parameter acquisition module is used to update the model parameters of the feature extraction model based on the total loss function, obtain the target model parameters of the feature extraction model, and then obtain the feature extraction model with the target model parameters as model parameters.

[0110] Furthermore, the sample processing module includes:

[0111] The first acquisition submodule is used to process the training samples through the feature extraction model to obtain the hidden state of the feature extraction model;

[0112] The second acquisition submodule is used to obtain the attention vector of the feature extraction model based on the hidden state.

[0113] Furthermore, the total loss function acquisition module includes:

[0114] The third acquisition submodule is used to obtain the attention vector entropy based on the attention vector;

[0115] The fourth acquisition submodule is used to obtain the attention distribution loss function based on the attention vector entropy;

[0116] The fifth acquisition submodule is used to obtain the total loss function based on the attention distribution loss function and the inherent loss function.

[0117] Furthermore, the target model parameter acquisition module includes:

[0118] The sixth acquisition submodule is used to obtain the gradient of the model parameters based on the total loss function and the training samples through the feature extraction model;

[0119] The seventh acquisition submodule is used to update the model parameters of the feature extraction model according to the gradient of the model parameters.

[0120] Furthermore, the second feature map acquisition module includes:

[0121] The eighth acquisition submodule is used to perform a reshape operation on the first feature map to obtain the third feature map;

[0122] The ninth acquisition submodule is used to input the third feature map into three different fully connected (FC) layers to obtain a first result corresponding to the FC layer, wherein the input dimension and output dimension of the FC layer remain unchanged;

[0123] The tenth acquisition submodule is used to perform a reshape operation on the first result to obtain a first input sequence, a second input sequence, and a third input sequence.

[0124] The eleventh acquisition submodule is used to perform matrix multiplication and softmax calculation on the first input sequence and the second input sequence to generate the fourth feature map;

[0125] The twelfth acquisition submodule is used to perform matrix multiplication on the fourth feature map and the third input sequence to generate the second feature map.

[0126] Furthermore, the preprocessing module includes:

[0127] The preprocessing module is used to normalize the original feature map to unify its size and specifications.

[0128] Reference Figure 4 This application also provides a computer device whose internal structure can be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, display device, and input device connected via a system bus. The network interface is used for communication with external terminals via a network connection. The display device is used to display interactive pages. The input device is used to receive user input. The processor is designed to provide computational and control capabilities. The memory includes non-volatile storage media. This non-volatile storage media stores the operating system, computer programs, and a database. The database stores raw data. When executed by the processor, the computer program implements an attention-based feature extraction method.

[0129] The processor executes the attention-based feature extraction method described above, preprocesses the original feature map to obtain a first feature map, inputs the first feature map into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module, and concatenates the original feature map and the second feature map to obtain the final feature extraction result, thereby improving the feature extraction model's ability to extract information about lesions with large differences in size and shape, and thus improving the accuracy of the detection results.

[0130] Furthermore, before inputting the first feature map into the pre-trained feature extraction model to generate the second feature map, the method includes:

[0131] Acquire training data, which includes training samples and labels corresponding to the training samples;

[0132] The training samples are processed by the feature extraction model to be trained to obtain the prediction results and attention vector of the feature extraction model;

[0133] Based on the prediction results of the training samples and the labels, the inherent loss function is obtained;

[0134] The total loss function is determined based on the inherent loss function and the attention vector;

[0135] The model parameters of the feature extraction model are updated based on the total loss function to obtain the target model parameters of the feature extraction model, thereby obtaining a feature extraction model with the target model parameters as model parameters.

[0136] Furthermore, the step of processing the training samples using the feature extraction model to be trained to obtain the prediction result and attention vector of the feature extraction model includes:

[0137] The training samples are processed by the feature extraction model to obtain the hidden state of the feature extraction model;

[0138] The attention vector of the feature extraction model is obtained based on the hidden state.

[0139] Furthermore, the step of determining the total loss function based on the inherent loss function and the attention vector includes:

[0140] The attention vector entropy is obtained based on the attention vector.

[0141] The attention distribution loss function is obtained based on the attention vector entropy.

[0142] The total loss function is obtained based on the attention distribution loss function and the inherent loss function.

[0143] Furthermore, the step of updating the model parameters of the feature extraction model based on the total loss function to obtain the target model parameters of the feature extraction model includes:

[0144] The gradient of the model parameters is obtained through the feature extraction model based on the total loss function and the training samples;

[0145] The model parameters of the feature extraction model are updated based on the gradient of the model parameters.

[0146] Further, the step of inputting the first feature map into a pre-trained feature extraction model to generate the second feature map includes:

[0147] The first feature map is reshaped to obtain the third feature map;

[0148] The third feature map is input into three different fully connected (FC) layers to obtain a first result corresponding to the FC layer, wherein the input and output dimensions of the FC layer remain unchanged.

[0149] Perform a reshape operation on the first result to obtain a first input sequence, a second input sequence, and a third input sequence;

[0150] Perform matrix multiplication and softmax calculation on the first input sequence and the second input sequence to generate the fourth feature map;

[0151] The second feature map is generated by performing matrix multiplication on the fourth feature map and the third input sequence.

[0152] Further, the step of preprocessing the original feature map to obtain the first feature map includes:

[0153] The original feature map is normalized to unify its size and specifications.

[0154] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by the processor, it implements a feature extraction method based on an attention mechanism. The method preprocesses the original feature map to obtain a first feature map; inputs the first feature map into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module; and concatenates the original feature map and the second feature map to obtain the final feature extraction result. This improves the feature extraction model's ability to extract information about lesions with large differences in size and shape, thereby improving the accuracy of the detection results.

[0155] Furthermore, before inputting the first feature map into the pre-trained feature extraction model to generate the second feature map, the method includes:

[0156] Acquire training data, which includes training samples and labels corresponding to the training samples;

[0157] The training samples are processed by the feature extraction model to be trained to obtain the prediction results and attention vector of the feature extraction model;

[0158] Based on the prediction results of the training samples and the labels, the inherent loss function is obtained;

[0159] The total loss function is determined based on the inherent loss function and the attention vector;

[0160] The model parameters of the feature extraction model are updated based on the total loss function to obtain the target model parameters of the feature extraction model, thereby obtaining a feature extraction model with the target model parameters as model parameters.

[0161] Furthermore, the step of processing the training samples using the feature extraction model to be trained to obtain the prediction result and attention vector of the feature extraction model includes:

[0162] The training samples are processed by the feature extraction model to obtain the hidden state of the feature extraction model;

[0163] The attention vector of the feature extraction model is obtained based on the hidden state.

[0164] Furthermore, the step of determining the total loss function based on the inherent loss function and the attention vector includes:

[0165] The attention vector entropy is obtained based on the attention vector.

[0166] The attention distribution loss function is obtained based on the attention vector entropy.

[0167] The total loss function is obtained based on the attention distribution loss function and the inherent loss function.

[0168] Furthermore, the step of updating the model parameters of the feature extraction model based on the total loss function to obtain the target model parameters of the feature extraction model includes:

[0169] The gradient of the model parameters is obtained through the feature extraction model based on the total loss function and the training samples;

[0170] The model parameters of the feature extraction model are updated based on the gradient of the model parameters.

[0171] Further, the step of inputting the first feature map into a pre-trained feature extraction model to generate the second feature map includes:

[0172] The first feature map is reshaped to obtain the third feature map;

[0173] The third feature map is input into three different fully connected (FC) layers to obtain a first result corresponding to the FC layer, wherein the input and output dimensions of the FC layer remain unchanged.

[0174] Perform a reshape operation on the first result to obtain a first input sequence, a second input sequence, and a third input sequence;

[0175] Perform matrix multiplication and softmax calculation on the first input sequence and the second input sequence to generate the fourth feature map;

[0176] The second feature map is generated by performing matrix multiplication on the fourth feature map and the third input sequence.

[0177] Further, the step of preprocessing the original feature map to obtain the first feature map includes:

[0178] The original feature map is normalized to unify its size and specifications.

[0179] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media provided in this application and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0180] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0181] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A feature extraction method based on an attention mechanism, characterized in that, include: The original feature map is preprocessed to obtain the first feature map; The first feature map is input into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module; The original feature map and the second feature map are concatenated to obtain the final feature extraction result; The step of inputting the first feature map into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module, includes: A reshape operation is performed on the first feature map to obtain the third feature map. The dimensions of the first feature map are (N, C, H, W), where N is the batch size (number of images in a batch), Channel is the number of channels in an image, H is the height (number of pixels in the vertical dimension), and W is the width (number of pixels in the horizontal dimension). The reshape operation uses the reshape function to process the first feature map, transforming its dimensions from (N, C, H, W) into the third feature map, where the dimensions of the third feature map are (NC, HW). The third feature map is input into three different fully connected (FC) layers to obtain a first result corresponding to the FC layer, wherein the input and output dimensions of the FC layer remain unchanged. The first result is reshaped to obtain a first input sequence, a second input sequence, and a third input sequence, where the first input sequence is Q(NC, HW, 1), the second input sequence is K(NC, 1, HW), and the third sequence is V(NC, 1, HW). The first input sequence is obtained through the Query branch, which uses a spatial attention mechanism; the second input sequence is obtained through the Key branch, which uses a channel attention mechanism; and the third input sequence is obtained through the Value branch. The first input sequence Q and the second input sequence K are multiplied by a matrix, and then the softmax operation is performed to generate the fourth feature map (NC, HW, H*W). The fourth feature map and the third input sequence V are multiplied by a matrix, and the original feature map is added to generate the second feature map.

2. The feature extraction method based on the attention mechanism according to claim 1, characterized in that, Before generating the second feature map by inputting the first feature map into the pre-trained feature extraction model, the method includes: Acquire training data, which includes training samples and labels corresponding to the training samples; The training samples are processed by the feature extraction model to be trained to obtain the prediction results and attention vector of the feature extraction model; Based on the prediction results of the training samples and the labels, the inherent loss function is obtained; The total loss function is determined based on the inherent loss function and the attention vector; The model parameters of the feature extraction model are updated based on the total loss function to obtain the target model parameters of the feature extraction model, thereby obtaining a feature extraction model with the target model parameters as model parameters.

3. The feature extraction method based on the attention mechanism according to claim 2, characterized in that, The step of processing the training samples using the feature extraction model to be trained to obtain the prediction result and attention vector of the feature extraction model includes: The training samples are processed by the feature extraction model to obtain the hidden state of the feature extraction model; The attention vector of the feature extraction model is obtained based on the hidden state.

4. The feature extraction method based on the attention mechanism according to claim 2, characterized in that, The step of determining the total loss function based on the inherent loss function and the attention vector includes: The attention vector entropy is obtained based on the attention vector. The attention distribution loss function is obtained based on the attention vector entropy. The total loss function is obtained based on the attention distribution loss function and the inherent loss function.

5. The feature extraction method based on the attention mechanism according to claim 2, characterized in that, The step of updating the model parameters of the feature extraction model based on the total loss function to obtain the target model parameters of the feature extraction model includes: The gradient of the model parameters is obtained through the feature extraction model based on the total loss function and the training samples; The model parameters of the feature extraction model are updated based on the gradient of the model parameters.

6. The feature extraction method based on the attention mechanism according to claim 1, characterized in that, The step of preprocessing the original feature map to obtain the first feature map includes: The original feature map is normalized to unify its size and specifications.

7. A feature extraction device based on an attention mechanism, used to implement the method described in any one of claims 1-6, characterized in that, include: The preprocessing module is used to preprocess the original feature map to obtain the first feature map; The second feature map acquisition module is used to input the first feature map into a pre-trained feature extraction model to generate a second feature map, wherein the feature extraction model includes a planar attention module; The final feature extraction result acquisition module is used to concatenate the original feature map and the second feature map to obtain the final feature extraction result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images