Construction and application method of target detection model, model, equipment and medium

Abstract

The invention relates to the field of computer vision, and discloses a construction and application method of a target detection model, a model, equipment and a medium, the construction scheme introduces an attention mechanism module and a residual module into a deep learning model, the attention mechanism module can better capture important features in a mammary gland image, and the residual module can better capture the important features in the mammary gland image. The breast nodules and the background are effectively distinguished, the residual module can relieve the gradient disappearance problem in the training process of the target detection model, and the learning ability of the model is enhanced. Therefore, according to the construction scheme of the application, the target detection model with better performance can be obtained by using the target detection model which introduces the attention mechanism module and the residual module into the mammary gland image training, and a more accurate detection result can be obtained when the model is used for detecting the mammary gland nodules in the mammary gland image.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to methods, models, devices, and media for constructing and applying target detection models. Background Art

[0002] Breast nodules are a common breast disease, and their early detection and diagnosis are of great significance to improving patient survival rates. Traditional breast nodule detection directly relies on experienced radiologists to manually label and analyze breast images (such as breast images), which is not only time-consuming and labor-intensive, but also easily affected by human factors, that is, a large amount of manual work causes doctors to be too tired, making it impossible to guarantee the accuracy and consistency of diagnosis.

[0003] With the development of deep learning technology, computer vision-based automatic target detection technology has been widely used in medical image analysis, that is, the target detection model can be used to perform preliminary detection on medical images, thereby assisting doctors in making more accurate medical diagnoses. However, the existing target detection models used to detect breast images are generally designed based on general convolutional neural networks (such as YOLO (You Only Look Once), Faster R-CNN and other models), and general convolutional neural networks can generally only obtain some local information in breast image data, thereby missing the extraction of key information in breast images. Therefore, the existing target detection models still face the following problems when processing breast images: Due to the presence of a large amount of complex background information in breast images, the existing target detection models are easily interfered by background information when extracting features, resulting in inaccurate final detection results, that is, false detection and missed detection; Due to the different shapes and sizes of breast nodules in breast images, the existing target detection models have limited detection capabilities for breast nodules of different shapes, that is, they cannot detect breast nodules with larger structures well.

[0004] In summary, the performance of existing target detection models is poor (i.e., existing target detection models can generally only obtain some local information in breast image data, thereby missing the extraction of key information in breast images), which may lead to inaccurate detection results when detecting breast nodules in breast images. Therefore, there is an urgent need for a target detection model with better performance to improve the detection accuracy of breast nodules in breast images. Summary of the invention

[0005] The present application provides a target detection model construction, application method and target detection model to solve the problem that the performance of the existing target detection model is poor, which may lead to inaccurate detection results when detecting breast nodules in breast images.

[0006] A first aspect of the present application provides a method for constructing a target detection model, wherein the target detection model is used to detect breast images, and the method comprises:

[0007] Acquire a breast image set, wherein the breast image set includes a plurality of breast images;

[0008] Constructing an initial target detection model, wherein the initial target detection model includes:

[0009] A feature extraction module, used for performing feature extraction on each of the breast images to obtain original feature maps of multiple channels of each of the breast images;

[0010] An attention module, used for performing attention processing on the original feature map of each channel of each breast image to obtain an attention feature map of each channel of each breast image;

[0011] A residual module, used for retaining the multi-channel original feature map corresponding to each breast image;

[0012] A feature fusion module, used for fusing the attention feature map and the original feature map of each channel of each breast image to obtain a fused feature map of each channel of each breast image;

[0013] A target detection module, for obtaining a category prediction value and a bounding box coordinate prediction value corresponding to a breast nodule in each of the breast images based on a fusion feature map of all channels of each of the breast images, and obtaining a detection result corresponding to a breast nodule in each of the breast images based on the category prediction value and the bounding box coordinate prediction value corresponding to the breast nodule in each of the breast images;

[0014] The initial target detection model is trained multiple times iteratively using the breast image set, wherein in each iterative training process, the parameters of the target detection model are updated according to a preset total loss function.

[0015] In some embodiments of the present application, the attention module includes:

[0016] A global attention submodule, used for performing global attention processing on the original feature map of each channel of each breast image to obtain a global feature map of each channel of each breast image;

[0017] A local attention submodule, used for performing local attention processing on the original feature map of each channel of each breast image to obtain a local feature map of each channel of each breast image;

[0018] The feature fusion submodule is used to fuse the global feature map and the local feature map of each channel of each breast image to obtain an attention feature map of each channel of each breast image.

[0019] In some embodiments of the present application, the original feature map of each of the multiple channels of the breast image is divided into two parts according to the number of channels, and the attention module includes:

[0020] A global attention submodule, used for performing global attention processing on the original feature map of each channel in the part of channels of each breast image, so as to obtain a global feature map of each channel in the part of channels of each breast image;

[0021] A local attention submodule, used for performing local attention processing on the original feature map of each channel in another part of the channels of each of the breast images, so as to obtain a local feature map of each channel in another part of the channels of each of the breast images;

[0022] The feature splicing submodule is used to splice the global feature map of a part of the channels of each breast image and the local feature map of another part of the channels to obtain an attention feature map of each channel of each breast image.

[0023] In some embodiments of the present application, the global attention submodule includes:

[0024] A first feature extraction unit is used to perform feature extraction on an original feature map of each channel of each breast image or an original feature map of each channel in a part of the channels to obtain a first intermediate feature map of each channel of each breast image or a first intermediate feature map of each channel in a part of the channels;

[0025] A global attention unit is used to perform global attention processing on the first intermediate feature map of each channel of each breast image or the first intermediate feature map of each channel in a part of the channels to obtain a global feature map of each channel of each breast image or the global feature map of each channel in a part of the channels.

[0026] In some embodiments of the present application, the local attention submodule includes:

[0027] A second feature extraction unit is used to perform feature extraction on an original feature map of each channel of each of the breast images or an original feature map of each channel in a portion of the channels to obtain a second intermediate feature map of each channel of each of the breast images or a second intermediate feature map of each channel in a portion of the channels;

[0028] A local attention unit is used to perform local attention processing on the second intermediate feature map of each channel of each breast image or the second intermediate feature map of each channel in a part of the channels to obtain a local feature map of each channel of each breast image or the local feature map of each channel in a part of the channels.

[0029] In some embodiments of the present application, the breast image set further includes: a category label and a position label corresponding to a breast nodule in each breast image, the category label indicates a category truth value corresponding to the breast nodule, the position label indicates a bounding box coordinate truth value corresponding to the breast nodule, and the total loss function includes:

[0030] A category loss function is determined based on the category true value and predicted value corresponding to each breast nodule in the breast image, and a bounding box loss function is determined based on the bounding box coordinate true value and predicted value corresponding to the breast nodule in the breast image.

[0031] A second aspect of the present application provides a target detection model, the target detection model comprising:

[0032] A feature extraction module, used to extract features from the acquired breast image to obtain original feature maps of multiple channels of the breast image;

[0033] An attention module, used for performing attention processing on the original feature map of each channel of the breast image to obtain an attention feature map of each channel of the breast image;

[0034] A residual module, used for retaining a multi-channel original feature map corresponding to the breast image;

[0035] A feature fusion module, used for fusing the attention feature map and the original feature map of each channel of the breast image to obtain a fused feature map of each channel of the breast image;

[0036] The target detection module is used to obtain the category prediction value and bounding box coordinate prediction value corresponding to the breast nodule in the breast image based on the fusion feature map of all channels of the breast image, and to obtain the detection result corresponding to the breast nodule in the breast image based on the category prediction value and bounding box coordinate prediction value corresponding to the breast nodule in the breast image.

[0037] The third aspect of the present application provides a method for applying a target detection model, the method comprising:

[0038] Acquire a breast image to be detected;

[0039] The target detection model described in the above embodiment is used to obtain the detection result corresponding to the breast nodule in the image to be detected based on the breast image to be detected.

[0040] The fourth aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first and third aspects of the above-mentioned embodiments when executing the computer program.

[0041] The fifth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, and when the program instructions are executed by a processor, the method described in any one of the first aspect and the third aspect of the above embodiments is implemented.

[0042] The present application has the following beneficial effects: the construction scheme of the target detection model of the present application introduces an attention mechanism module and a residual module into the deep learning model. The attention mechanism module can better capture the important features in the breast image and effectively distinguish breast nodules from the background. The residual module can alleviate the gradient vanishing problem in the training process of the target detection model and enhance the learning ability of the model. Therefore, the construction scheme of the present application can obtain a target detection model with better performance by introducing the target detection model of the attention mechanism module and the residual module through breast image training. The model can obtain more accurate detection results when detecting breast nodules in breast images. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0044] Figure 1 This is a flow chart of an embodiment of a method for constructing a target detection model provided by the present application;

[0045] Figure 2 It is a schematic diagram of the structure of the first embodiment of the target detection model provided by the present application;

[0046] Figure 3 It is a schematic diagram of the structure of the second embodiment of the target detection model provided by this application;

[0047] Figure 4 It is a schematic diagram of the structure of the third embodiment of the target detection model provided by this application;

[0048] Figure 5 It is a schematic diagram of the structure of the fourth embodiment of the target detection model provided by the present application;

[0049] Figure 6 It is a schematic diagram of the structure of the fifth embodiment of the target detection model provided by the present application;

[0050] Figure 7 It is a schematic diagram of a framework of an embodiment of an electronic device provided by the present application;

[0051] Figure 8 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0052] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0053] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0054] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0055] As described in the background technology, the performance of existing target detection models is poor (i.e., existing target detection models can generally only obtain some local information in breast image data, thereby missing the extraction of key information in breast images), which may lead to inaccurate detection results when detecting breast nodules in breast images.

[0056] The inventors have found that the attention mechanism can help the model capture key information in the input data more effectively. The residual network (ResNet) is a deep neural network structure that aims to solve the problems of gradient disappearance and gradient explosion in deep networks.

[0057] To this end, the present application proposes a new target detection model construction scheme by combining the attention mechanism and the residual network. In the scheme of the present application, by introducing the attention mechanism module and the residual module in the deep learning model, the attention mechanism module can better capture the important features in the breast image and effectively distinguish the breast nodules from the background. The residual module can alleviate the gradient vanishing problem in the target detection model training process and enhance the learning ability of the model. Therefore, the construction scheme of the present application uses the target detection model that introduces the attention mechanism module and the residual module through breast image training to obtain a target detection model with better performance. The model can obtain more accurate detection results when detecting breast nodules in breast images.

[0058] The present application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] According to one embodiment of the present application, Figure 1 As shown, the present application proposes a method for constructing a target detection model, wherein the target detection model is used to detect breast images, and the method comprises: S1, obtaining a breast image set, wherein the breast image set comprises a plurality of breast images; S2, constructing an initial target detection model, wherein, as Figure 2 As shown, the initial target detection model includes: a feature extraction module, which is used to extract features from each of the breast images to obtain original feature maps of multiple channels of each of the breast images; an attention module, which is used to perform attention processing on the original feature map of each channel of each of the breast images to obtain an attention feature map of each channel of each of the breast images; a residual module, which is used to retain the multi-channel original feature map corresponding to each of the breast images; a feature fusion module, which is used to perform fusion processing on the attention feature map of each channel of each of the breast images and the original feature map to obtain an attention feature map of each of the breast images. A fused feature map of each channel of the breast image; a target detection module, which is used to obtain the category prediction value and bounding box coordinate prediction value corresponding to the breast nodule in each of the breast images based on the fused feature maps of all channels of each of the breast images, and obtain the detection result corresponding to the breast nodule in each of the breast images based on the category prediction value and bounding box coordinate prediction value corresponding to the breast nodule in each of the breast images; S3, using the breast image set to perform multiple iterative training on the initial target detection model, wherein in each iterative training process, the parameters of the target detection model are updated according to the preset total loss function.

[0060] In order to more intuitively understand the technical solution of the present application, the above steps S2 and S3 are described in detail below.

[0061] 1. Step S2

[0062] In step S2, an initial target detection model is constructed.

[0063] In order to further improve the performance of the model, the present application introduces a global local attention mechanism into the target detection model, so that the model can capture features of different scales in breast images (i.e., it can capture long-range dependencies and local details in breast images, where the global attention mechanism can focus on each part of the breast image and calculate the relationship between them, thereby capturing global information; the local attention mechanism focuses on small areas in the breast image and extracts fine spatial information), thereby improving the model's ability to recognize nodule areas. The following is combined with a specific embodiment to illustrate the purpose of introducing a global local attention mechanism into a target detection model.

[0064] 1. Examples of global and local feature fusion

[0065] According to one embodiment of the present application, Figure 3 As shown, the attention module of the target detection model includes: a global attention submodule, which is used to perform global attention processing on the original feature map of each channel of each breast image to obtain a global feature map of each channel of each breast image; a local attention submodule, which is used to perform local attention processing on the original feature map of each channel of each breast image to obtain a local feature map of each channel of each breast image; a feature fusion submodule, which is used to perform fusion processing on the global feature map and the local feature map of each channel of each breast image to obtain an attention feature map of each channel of each breast image. The global attention submodule and the local attention submodule are described in detail below.

[0066] (1) Global Attention Submodule

[0067] According to one embodiment of the present application, Figure 4 As shown, the global attention submodule includes: a first feature extraction unit, which is used to extract features from the original feature map of each channel of each breast image to obtain the first intermediate feature map of each channel of each breast image; a global attention unit, which is used to perform global attention processing on the first intermediate feature map of each channel of each breast image to obtain the global feature map of each channel of each breast image. According to another embodiment of the present application, the global attention unit is configured to: perform global attention processing on the original feature map of each channel of each breast image using a multi-head self-attention mechanism (Multi-Head Self-Attention) to obtain the global feature map of each channel of each breast image. It should be noted that the multi-head self-attention mechanism is one of the embodiments of the present application, and other attention mechanisms such as multi-head attention, relative position coding attention, and scalable dot product attention can also be used to perform global attention processing on the original feature map of each channel of each breast image. The specific selection should take into account the specific requirements of the task and the available computing resources, which are not specifically limited here.

[0068] (2) Local Attention Submodule

[0069] According to one embodiment of the present application, Figure 5As shown, the local attention submodule includes: a second feature extraction unit, used to perform feature extraction on the original feature map of each channel of each breast image to obtain a second intermediate feature map of each channel of each breast image; a local attention unit, used to perform local attention processing on the second intermediate feature map of each channel of each breast image to obtain a local feature map of each channel of each breast image. According to one embodiment of the present application, the local attention unit is configured to: perform local attention processing on the original feature map of each channel in another part of the channel of each breast image using a convolution operation to obtain a local feature map of each channel in another part of the channel of each breast image. It should be noted that the convolution operation is one of the embodiments of the present application, and the specific operation selection should take into account the specific requirements of the task and the available computing resources, which are not specifically limited here.

[0070] It can be seen from the above embodiments that global attention processing and local attention processing are performed on the original feature maps of all channels of each breast image, and the two processing results are fused and then fused with the original feature maps of all channels to obtain the detection result of the breast image based on the fusion result. By combining the residual connection with the global local attention mechanism, the present application can simultaneously utilize the global context, local details and original features to achieve a more powerful feature extraction capability, thereby improving the model's ability to recognize nodule areas.

[0071] 2. Examples of Global and Local Feature Stitching

[0072] Since the global attention submodule and the local attention submodule perform attention processing on the feature maps of all channels in the breast image respectively, the computational complexity is high and feature redundancy may also occur. For this reason, the present application divides the original feature maps of multiple channels of each breast image into two parts according to the number of channels, one part is processed by the global attention submodule, and the other part is processed by the local attention submodule, thereby reducing the computational complexity. At the same time, the channel is divided into two parts to respectively learn global and local features, avoiding redundancy of feature information, thereby affecting the detection results of the breast image.

[0073] According to an embodiment of the present application, the original feature map of each of the multiple channels of the breast image is divided into two parts according to the number of channels, such as Figure 6As shown, the attention module of the target detection model includes: a global attention submodule, which is used to perform global attention processing on the original feature map of each channel in a part of the channels of each breast image to obtain a global feature map of each channel in a part of the channels of each breast image; a local attention submodule, which is used to perform local attention processing on the original feature map of each channel in another part of the channels of each breast image to obtain a local feature map of each channel in another part of the channels of each breast image; a feature splicing submodule, which is used to perform splicing processing on the global feature map of a part of the channels of each breast image and the local feature map of another part of the channels to obtain an attention feature map of each channel of each breast image. The global attention submodule and the local attention submodule are described in detail below.

[0074] (1) Global Attention Submodule

[0075] According to one embodiment of the present application, still referring to Figure 4 , the global attention submodule includes: a first feature extraction unit, which is used to extract features from the original feature map of each channel in a part of the channels of each breast image to obtain a first intermediate feature map of each channel in a part of the channels of each breast image; a global attention unit, which is used to perform global attention processing on the first intermediate feature map of each channel in a part of the channels of each breast image to obtain a global feature map of each channel in a part of the channels of each breast image. According to another embodiment of the present application, the global attention unit is configured to: perform global attention processing on the original feature map of each channel in a part of the channels of each breast image using a multi-head self-attention mechanism (Multi-Head Self-Attention) to obtain a global feature map of each channel in a part of the channels of each breast image. It should be noted that the multi-head self-attention mechanism is one of the embodiments of the present application, and other attention mechanisms such as multi-head attention, relative position coding attention, and scalable dot product attention can also be used to perform global attention processing on the original feature map of each channel of each breast image. The specific selection should take into account the specific requirements of the task and the available computing resources, which are not specifically limited here.

[0076] (2) Local Attention Submodule

[0077] According to one embodiment of the present application, still referring to Figure 5, the local attention submodule includes: a second feature extraction unit, used to perform feature extraction on the original feature map of each channel in a part of the channel of each breast image, so as to obtain a second intermediate feature map of each channel in a part of the channel of each breast image; a local attention unit, used to perform local attention processing on the second intermediate feature map of each channel in a part of the channel of each breast image, so as to obtain a local feature map of each channel in a part of the channel of each breast image. According to another embodiment of the present application, the local attention unit is configured to: perform local attention processing on the original feature map of each channel in another part of the channel of each breast image using a convolution operation, so as to obtain a local feature map of each channel in another part of the channel of each breast image. It should be noted that the convolution operation is one of the embodiments of the present application, and the specific operation selection should take into account the specific requirements of the task and the available computing resources, which are not specifically limited here.

[0078] It can be seen from the above embodiments that by performing global attention processing on the original feature map of a part of the channels of each breast image and local attention processing on the original feature map of another part of the channels of each breast image, and splicing the two processing results and fusing them with the original feature maps of all channels, the detection result of the breast image is obtained based on the fusion result. By combining the residual connection with the global local attention mechanism, the present application can simultaneously utilize the global context, local details and original features, achieve more powerful feature extraction capabilities while reducing computational complexity and avoiding redundancy of feature information, thereby improving the model's ability to recognize nodule areas.

[0079] Step S3

[0080] In step S3, the initial target detection model is trained multiple times by using the breast image set, wherein in each iterative training process, the parameters of the target detection model are updated according to a preset total loss function.

[0081] According to one embodiment of the present application, the breast image set also includes: a category label and a position label corresponding to the breast nodule in each breast image, the category label indicates the category truth value corresponding to the breast nodule, the position label indicates the bounding box coordinate truth value corresponding to the breast nodule, and the total loss function includes: a category loss function determined based on the category truth value and predicted value corresponding to the breast nodule in each breast image, and a bounding box loss function determined based on the bounding box coordinate truth value and predicted value corresponding to the breast nodule in the breast image. According to one embodiment of the present application, the total loss function is:

[0082]

[0083] Among them, w1 and w 2 All are preset hyperparameters. is the category loss function, L GIoU is the bounding box loss function.

[0084] According to another embodiment of the present application, the category loss function is:

[0085]

[0086] The bounding box loss function is:

[0087]

[0088] Wherein, N is the number of breast images in the breast image set, y i represents the true value of the category corresponding to the breast nodule in the i-th breast image, represents the category prediction value corresponding to the breast nodule in the i-th breast image, A i represents the predicted value of the bounding box coordinates corresponding to the breast nodule in the i-th breast image, B i represents the true value of the bounding box coordinates corresponding to the breast nodule in the i-th breast image, C i Represents the true value of the bounding box coordinates that can enclose the breast nodule in the i-th breast image B i And the bounding box coordinate prediction value A i The closed box of , γ is the preset adjustment factor.

[0089] It should be noted that the above-mentioned total loss function, category loss function and bounding box loss function are all embodiments of the present application. With the continuous development of neural network training methods, the total loss function, category loss function and bounding box loss function can adopt other loss functions that can further improve the performance of the target detection model. No specific limitation is made here.

[0090] It can be seen from the above description that by training the target detection model using the training method of the above embodiment, a target detection model with better performance can be obtained.

[0091] In order to verify that the target detection model construction scheme of the above embodiment of the present application can obtain a target detection model with better performance, the inventor conducted a series of experiments. The following is an explanation of the experimental model selection, the setting of relevant parameters of the training model and the experimental results.

[0092] 1. Experimental model selection

[0093] The inventors have found that the existing target detection models for detecting breast images are generally designed based on general convolutional neural networks (such as YOLO (You Only Look Once), Faster R-CNN and other models).

[0094] 1. The target detection model designed based on the YOLO (You Only Look Once) model can only obtain some local information in the breast image data, thus missing the extraction of key information in the breast image, causing the model to be easily interfered by background information when extracting features, resulting in false detection and missed detection. In addition, since the breast nodules in the breast images have different shapes and sizes, the model has limited detection capabilities for breast nodules of different shapes, that is, it cannot detect breast nodules with larger structures well. In addition, in medical images with high-precision detection requirements, the detection accuracy of the YOLO model is difficult to meet clinical requirements.

[0095] 2. The target detection model designed based on the Faster R-CNN model can only obtain some local information in the breast image data, thus missing the extraction of key information in the breast image. As a result, the model is easily disturbed by background information when extracting features, resulting in false detection and missed detection. In addition, since the breast nodules in the breast images have different shapes and sizes, the model has limited detection capabilities for breast nodules of different shapes, that is, it cannot detect breast nodules with larger structures well. In addition, since the target detection model designed based on the Faster R-CNN model uses a two-stage detection method when performing breast nodule detection, that is, first detecting the bounding box of the breast nodule, and then classifying the breast nodules based on the bounding box, this detection method results in a slow detection speed for the model, which is not suitable for real-time detection needs.

[0096] Therefore, in order to solve the problems existing in the above-mentioned existing target detection models, the inventors found through further research that the RTMDet model is a high-performance target detection model, which is suitable for application scenarios that require real-time processing capabilities. To this end, the present application constructs a new target detection model based on the RTMDet model, as shown in Table 1, which shows the model structure parameters designed by the present application based on the RTMDet model, where 1-7 correspond to the feature extraction module, 9 corresponds to the attention module, 8 and 10 correspond to the residual module, and 11-15 correspond to the feature fusion module and the target detection module. It should be noted that constructing a target detection model based on the RTMDet model is only one of the embodiments of the present application. With the development of deep learning models, it is also possible to improve the technical concept of the present application based on a model with better performance to obtain a target detection model with better performance for breast nodule detection, which is not specifically limited here.

[0097] Table 1

[0098] 1 Stem ConvModule Convolutional Layer 2 Stem ConvModule 3 Stem ConvModule 4 Stage 1 ConvModule 5 Stage 1 CSPLayer Convolutional Residual Layer 6 Stage 2 ConvModule 7 Stage 2 CSPLayer 8 Stage 2 Residual Connection Residual Connection Layer 9 Stage 2 GLSA Global-local attention layer 10 Stage 2 Residual Connection 11 Stage 3 ConvModule 12 Stage 3 CSPLayer 13 Stage 4 ConvModule 14 Stage 4 CSPLayer 15 Stage 4 SPPBottleneck Spatial Pyramid Pooling Layer

[0099] 2. Training model related parameter settings

[0100] As shown in Table 2, it shows the main hyperparameter setting information when training the model. These parameter information are some basic setting parameters of the deep learning model, which will not be described in detail here. The model training of this application adopts the method of dividing the data set by patient ID and the data enhancement cycle to improve accuracy and prevent overfitting. It should be noted that the hyperparameter information of the training model in Table 2 is only an example in the experimental process. All hyperparameters can be adjusted according to the requirements of the task or the final experimental results, and are not specifically limited here.

[0101] Table 2

[0102]

[0103]

[0104] 3. Experimental Results

[0105] As shown in Table 3, it shows the detection results corresponding to different models. Among them, the three versions of RTMDet tsl represent models with different parameters. RTMDet-GLSA is an improved model based on RTMDet-s using the technical concept of this application. The experimental results show that compared with the existing target detection model, the nodule detection accuracy is improved after the residual and attention modules (i.e. GLSA) are introduced into the model.

[0106] Table 3

[0107]

[0108] In summary, compared with the existing target detection models designed based on general convolutional neural networks (such as YOLO (You Only LookOnce), Faster R-CNN and other models), the target detection model obtained by this application is optimized on the structure of the Rtmdet model, which improves the computational efficiency, achieves high precision with a model with low parameters, and realizes fast and efficient breast nodule detection, which is suitable for real-time applications.

[0109] In addition, using the target detection model construction method described in any of the above embodiments, according to an embodiment of the present application, a target detection model is provided, which includes: a feature extraction module, used to extract features from the acquired breast image to obtain original feature maps of multiple channels of the breast image; an attention module, used to perform attention processing on the original feature map of each channel of the breast image to obtain an attention feature map of each channel of the breast image; a residual module, used to retain the multi-channel original feature map corresponding to the breast image; a feature fusion module, used to fuse the attention feature map and the original feature map of each channel of the breast image to obtain a fused feature map of each channel of the breast image; a target detection module, used to obtain a category prediction value and a bounding box coordinate prediction value corresponding to the breast nodule in the breast image based on the fused feature map of all channels of the breast image, and obtain a detection result corresponding to the breast nodule in the breast image based on the category prediction value and the bounding box coordinate prediction value corresponding to the breast nodule in the breast image.

[0110] In practical applications, according to the target detection model of the above-mentioned embodiment, the present application provides an application method of the target detection model, the method comprising: obtaining a breast image to be detected; using the target detection model described in the above-mentioned embodiment, based on the breast image to be detected, obtaining the detection result corresponding to the breast nodule in the image to be detected.

[0111] It can be seen from the above embodiments that the target detection model constructed by the present application can improve the detection accuracy of breast nodules in breast images.

[0112] Based on the inventive concept of the above embodiments, the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the above embodiments when executing the computer program. Figure 7 Provide detailed explanation.

[0113] like Figure 7 As shown, it shows the electronic device of the present application, which may specifically include a processor 110 and a memory 120. The memory 120 is coupled to the processor 110.

[0114] The processor 110 is used to control the operation of the electronic device. The processor 110 may also be referred to as a CPU (Central Processing Unit). The processor 110 may be an integrated circuit chip having the ability to process signals. The processor 110 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 110 may also be any conventional processor, etc.

[0115] The memory 120 is used to store computer programs, and may be a RAM, a ROM, or other types of storage terminals. Specifically, the memory 120 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 120 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage terminals, flash memory storage terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 120 is used to store at least one program code.

[0116] The processor 110 is used to execute the computer program stored in the memory 120 to implement the methods described in the various method embodiments of the present application.

[0117] In some embodiments, the electronic device may further include: a peripheral terminal interface 130 and at least one peripheral terminal. The processor 110, the memory 120 and the peripheral terminal interface 130 may be connected via a bus or a signal line. Each peripheral terminal may be connected to the peripheral terminal interface 130 via a bus, a signal line or a circuit board. Specifically, the peripheral terminal includes: at least one of a radio frequency circuit 140, a display screen 150, an audio circuit 160 and a power supply 170.

[0118] The peripheral terminal interface 130 may be used to connect at least one peripheral terminal related to I / O (Input / output) to the processor 110 and the memory 120. In some embodiments, the processor 110, the memory 120, and the peripheral terminal interface 130 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 110, the memory 120, and the peripheral terminal interface 130 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0119] The radio frequency circuit 140 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 140 communicates with the communication network and other IoT devices through electromagnetic signals, and the radio frequency circuit 140 is the communication circuit of the electronic device. The radio frequency circuit 140 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 140 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identity module card, and the like. The radio frequency circuit 140 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 140 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0120] The display screen 150 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 150 is a touch display screen, the display screen 150 also has the ability to collect touch signals on the surface or above the surface of the display screen 150. The touch signal can be input to the processor 110 as a control signal for processing. At this time, the display screen 150 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 150 can be one, set on the front panel of the electronic device; in other embodiments, the display screen 150 can be at least two, respectively set on different surfaces of the electronic device or in a folding design; in other embodiments, the display screen 150 can be a flexible display screen, set on a curved surface or a folding surface of the electronic device. Even, the display screen 150 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 150 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0121] The audio circuit 160 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, and convert the sound waves into electrical signals and input them into the processor 110 for processing, or input them into the radio frequency circuit 140 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there can be multiple microphones, which are respectively arranged in different parts of the electronic device. The microphone can also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 110 or the radio frequency circuit 140 into sound waves. The speaker can be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 160 may also include a headphone jack.

[0122] The power supply 170 is used to power various components in the electronic device. The power supply 170 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 170 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0123] For a detailed description of the functions and execution processes of each functional module or component in the electronic device embodiment of the present application, reference can be made to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0124] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the various embodiments of the electronic devices described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0125] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0126] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0127] Based on the inventive concept of the above embodiment, the present application also provides a computer-readable storage medium storing a computer program, which can be executed by a processor to implement the above projection area calculation method. Figure 8 The execution process of the above-mentioned embodiment in a computer-readable storage medium is described.

[0128] like Figure 8 As shown, it shows the computer-readable storage medium of the present application. If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 200. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions / computer programs to enable an IoT device (which can be a personal computer, server, or network terminal, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and electronic terminals such as computers, mobile phones, laptops, tablet computers, cameras, etc. having the above-mentioned storage medium.

[0129] The description of the execution process of the program data in the computer-readable storage medium can refer to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0130] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

[0131] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

Claims

1. A method for constructing a target detection model, wherein the target detection model is used to detect breast images, characterized in that: The method comprises: Acquire a breast image set, wherein the breast image set includes a plurality of breast images; Constructing an initial target detection model, wherein the initial target detection model includes: A feature extraction module, used for performing feature extraction on each of the breast images to obtain original feature maps of multiple channels of each of the breast images; An attention module, used for performing attention processing on the original feature map of each channel of each breast image to obtain an attention feature map of each channel of each breast image; A residual module, used for retaining the multi-channel original feature map corresponding to each breast image; A feature fusion module, used for fusing the attention feature map and the original feature map of each channel of each breast image to obtain a fused feature map of each channel of each breast image; A target detection module, for obtaining a category prediction value and a bounding box coordinate prediction value corresponding to a breast nodule in each of the breast images based on a fusion feature map of all channels of each of the breast images, and obtaining a detection result corresponding to a breast nodule in each of the breast images based on the category prediction value and the bounding box coordinate prediction value corresponding to the breast nodule in each of the breast images; The initial target detection model is trained multiple times iteratively using the breast image set, wherein in each iterative training process, the parameters of the target detection model are updated according to a preset total loss function.

2. The method according to claim 1, characterized in that The attention module comprises: A global attention submodule, used for performing global attention processing on the original feature map of each channel of each breast image to obtain a global feature map of each channel of each breast image; A local attention submodule, used for performing local attention processing on the original feature map of each channel of each breast image to obtain a local feature map of each channel of each breast image; The feature fusion submodule is used to fuse the global feature map and the local feature map of each channel of each breast image to obtain an attention feature map of each channel of each breast image.

3. The method according to claim 1, characterized in that The original feature map of each of the multiple channels of the breast image is divided into two parts according to the number of channels, and the attention module includes: A global attention submodule, used for performing global attention processing on the original feature map of each channel in the part of channels of each breast image, so as to obtain a global feature map of each channel in the part of channels of each breast image; A local attention submodule, used for performing local attention processing on the original feature map of each channel in another part of the channels of each of the breast images, so as to obtain a local feature map of each channel in another part of the channels of each of the breast images; The feature splicing submodule is used to splice the global feature map of a part of the channels of each breast image and the local feature map of another part of the channels to obtain an attention feature map of each channel of each breast image.

4. The method according to claim 2 or 3, characterized in that: The global attention submodule includes: A first feature extraction unit is used to perform feature extraction on an original feature map of each channel of each breast image or an original feature map of each channel in a part of the channels to obtain a first intermediate feature map of each channel of each breast image or a first intermediate feature map of each channel in a part of the channels; A global attention unit is used to perform global attention processing on the first intermediate feature map of each channel of each breast image or the first intermediate feature map of each channel in a part of the channels to obtain a global feature map of each channel of each breast image or the global feature map of each channel in a part of the channels.

5. The method according to claim 2 or 3, characterized in that: The local attention submodule includes: A second feature extraction unit is used to perform feature extraction on an original feature map of each channel of each of the breast images or an original feature map of each channel in a portion of the channels to obtain a second intermediate feature map of each channel of each of the breast images or a second intermediate feature map of each channel in a portion of the channels; A local attention unit is used to perform local attention processing on the second intermediate feature map of each channel of each breast image or the second intermediate feature map of each channel in a part of the channels to obtain a local feature map of each channel of each breast image or the local feature map of each channel in a part of the channels.

6. The method according to claim 1, characterized in that The breast image set also includes: a category label and a position label corresponding to the breast nodule in each breast image, the category label indicates the category truth value corresponding to the breast nodule, the position label indicates the bounding box coordinate truth value corresponding to the breast nodule, and the total loss function includes: A category loss function is determined based on the category true value and predicted value corresponding to each breast nodule in the breast image, and a bounding box loss function is determined based on the bounding box coordinate true value and predicted value corresponding to the breast nodule in the breast image.

7. A target detection model, characterized in that: The target detection model includes: A feature extraction module, used to extract features from the acquired breast image to obtain original feature maps of multiple channels of the breast image; An attention module, used for performing attention processing on the original feature map of each channel of the breast image to obtain an attention feature map of each channel of the breast image; A residual module, used for retaining a multi-channel original feature map corresponding to the breast image; A feature fusion module, used for fusing the attention feature map and the original feature map of each channel of the breast image to obtain a fused feature map of each channel of the breast image; The target detection module is used to obtain the category prediction value and bounding box coordinate prediction value corresponding to the breast nodule in the breast image based on the fusion feature map of all channels of the breast image, and to obtain the detection result corresponding to the breast nodule in the breast image based on the category prediction value and bounding box coordinate prediction value corresponding to the breast nodule in the breast image.

8. A method for applying a target detection model, characterized in that: The method comprises: Acquire a breast image to be detected; Utilizing the target detection model as described in claim 7, a detection result corresponding to the breast nodule in the image to be detected is obtained based on the breast image to be detected.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 6 and 8 when executing the computer program.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method described in any one of claims 1-6 and 8 is implemented.

Images