Training method and detection method of detection model

By extracting and enhancing depth feature maps of different scales, combining multi-view and three-dimensional segmentation modules, the lung nodule detection model is optimized, the false positive problem is solved, the detection accuracy and efficiency are improved, and the model training complexity is reduced.

CN117710317BActive Publication Date: 2024-11-08THE CHINESE UNIV OF HONG KONG (SHENZHEN) +1
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202311726223.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-13
Publication Date
2024-11-08
Estimated Expiration
2043-12-13

AI Technical Summary

Technical Problem

In the prior art, the deep learning-based lung nodule detection algorithm has the problem of excessive false positives, which leads to a decrease in model accuracy, and requires relying on multi-stage model training and manual definition of hyperparameters, which increases the complexity of model training.

Method used

By extracting depth feature maps of different scales and performing feature enhancement processing, augmented feature map is generated, combining multi-view prediction and three-dimensional segmentation modules, the initial detection model is optimized, and the dependence on multi-stage models and artificial hyperparameters is reduced.

Benefits of technology

It improves the accuracy of lung nodule detection, reduces the complexity of model training and the efficiency of detection process, and reduces the dependence on false positive models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117710317B_ABST
    Figure CN117710317B_ABST
Patent Text Reader

Abstract

The present invention provides a detection model training method and a detection method, which relates to the field of medical image processing technology, including: obtaining a number of training image samples, and preprocessing the training image samples; for any training image sample: inputting the preprocessed training image sample into the encoding module in the initial detection model to extract a plurality of depth feature maps of different scales; performing feature enhancement processing on the depth feature maps of different scales respectively to obtain enhanced feature maps of different scales; inputting the depth feature maps and enhanced feature maps of different scales into the decoding module in the initial detection model to obtain the detection results corresponding to the training image samples; based on the detection results of each of the training image samples, iteratively training the initial detection model to obtain a target detection model. The present invention achieves the goal of reducing the complexity of model training while ensuring the accuracy of detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a detection model training method and a detection method. Background Art

[0002] The detection of lung nodules is an important task in lung cancer analysis and can help doctors screen for lung cancer. Common deep learning-based lung nodule detection algorithms usually face the problem of too many false positives, that is, the lung nodule detection results detected by the model contain many non-nodule tissues such as blood vessels. This will reduce the accuracy of the model, causing doctors to spend more time screening out erroneous detection results. In order to solve this problem, a two-stage detection method is usually adopted, that is, firstly, suspected lung nodules are detected by a detection model, and then a false positive removal model is used to perform secondary classification on the detected suspected lung nodules, thereby filtering out erroneous detection results. However, this model training method that relies on multi-stage model training and more manually defined hyperparameters significantly increases the complexity of model training. Summary of the invention

[0003] The present invention provides a detection model training method and a detection method, aiming to solve the technical problem that the model training method relies on multi-stage model training and more manually defined hyperparameters, which significantly increases the complexity of model training.

[0004] The present invention provides a method for training a detection model, comprising:

[0005] Acquire a plurality of training image samples, and preprocess the training image samples;

[0006] For any of the training image samples: inputting the preprocessed training image sample into the encoding module in the initial detection model to extract a plurality of depth feature maps of different scales;

[0007] Performing feature enhancement processing on the depth feature maps of different scales respectively to obtain enhanced feature maps of different scales;

[0008] Inputting the depth feature maps and enhanced feature maps of different scales into the decoding module in the initial detection model to obtain the detection results corresponding to the training image samples;

[0009] Based on the detection results of each of the training image samples, the initial detection model is iteratively trained to obtain a target detection model

[0010] According to a detection model training method provided by the present invention, the feature enhancement processing is performed on the depth feature maps of different scales respectively to obtain enhanced feature maps of different scales, including:

[0011] For a deep feature map of any scale:

[0012] Dividing the depth feature map into two-dimensional feature sequences of different views;

[0013] Inputting the two-dimensional feature sequences of different views into the multi-view prediction module in the initial detection model for prediction respectively, and obtaining prediction results corresponding to the two-dimensional feature sequences of different views;

[0014] Based on the prediction results of different views, first weight maps of different views are constructed;

[0015] The enhanced feature map is generated based on the depth feature map and the first weight map of the different views.

[0016] According to a detection model training method provided by the present invention, the generating of the enhanced feature map based on the depth feature map and the first weight map of the different views includes:

[0017] Performing an average operation on the first weight maps of different views to obtain a target weight map;

[0018] The target weight map is subjected to feature fusion with the depth feature map to obtain the enhanced feature map.

[0019] According to a detection model training method provided by the present invention, after respectively inputting the two-dimensional feature sequences of different views into the multi-view prediction module in the initial detection model for prediction and obtaining the prediction results corresponding to the two-dimensional feature sequences of different views, the method further includes:

[0020] For any prediction result corresponding to the two-dimensional feature sequence: based on the prediction result and preset position annotation information, a first loss value is calculated, wherein the first loss value is used to optimize the model parameters of the initial detection model.

[0021] According to a detection model training method provided by the present invention, the feature enhancement processing is performed on the depth feature maps of different scales respectively to obtain enhanced feature maps of different scales, including:

[0022] For a deep feature map of any scale:

[0023] Performing convolution processing on the depth feature map to obtain a segmentation result;

[0024] Based on the segmentation result, construct a second weight map;

[0025] Based on the second weight map and the depth feature map, the enhanced feature map is generated.

[0026] According to a detection model training method provided by the present invention, after the convolution layer is used to perform convolution processing on the depth feature map to obtain a segmentation result, the method further includes:

[0027] Generate three-dimensional mask information based on preset position annotation information;

[0028] Determining a second loss value based on the three-dimensional mask information and the segmentation result;

[0029] Based on the position annotation information and the segmentation result, a third loss value is calculated using a contrast loss function;

[0030] The second loss value and the third loss value are used to optimize the model parameters of the initial detection model.

[0031] According to a detection model training method provided by the present invention, the depth feature maps and enhanced feature maps of different scales include a depth feature map of a first scale, a depth feature map of a second scale, a depth feature map of a third scale, and an enhanced feature map of the first scale, an enhanced feature map of the second scale, and an enhanced feature map of the third scale;

[0032] The first scale is smaller than the second scale, and the second scale is smaller than the third scale.

[0033] According to a detection model training method provided by the present invention, the depth feature maps and enhanced feature maps of different scales are input into a decoding module in the initial detection model to obtain the detection results corresponding to the training image samples, including:

[0034] Upsampling the depth feature map and the enhanced feature map of the first scale to obtain a first upsampled feature map;

[0035] Performing convolution processing on the first up-sampled feature map to obtain a first convolution feature map;

[0036] Performing feature fusion processing on the first convolution feature map, the depth feature map of the second scale, and the enhanced feature map to obtain a first fused feature map;

[0037] Upsampling the fused feature map to obtain a second upsampled feature map;

[0038] Performing convolution processing on the second up-sampled feature map to obtain a second convolution feature map;

[0039] Performing feature fusion processing on the second convolution feature map, the depth feature map of the third scale, and the enhanced feature map to obtain a second fused feature map;

[0040] The second fused feature map is convolved to obtain the detection result.

[0041] According to a detection model training method provided by the present invention, the preprocessed training image samples are input into the encoding module in the initial detection model to extract a plurality of depth feature maps of different scales, including:

[0042] Using a preset sliding window to intercept the training image samples to obtain a plurality of three-dimensional image blocks;

[0043] Feature extraction is performed on each of the three-dimensional image blocks to obtain depth feature maps of different scales.

[0044] The present invention also provides a detection method, comprising:

[0045] Acquire an image to be detected, and preprocess the image to be detected;

[0046] The preprocessed image to be detected is input into the encoding module in the target detection model to extract target depth feature maps of multiple different scales;

[0047] Performing feature enhancement processing on the target depth feature maps of different scales respectively to obtain target enhanced feature maps of different scales;

[0048] The target depth feature maps and target enhancement feature maps of different scales are input into the decoding module in the target detection model to obtain the lung nodule detection result output by the detection model.

[0049] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a training method for a detection model as described above is implemented.

[0050] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method for a detection model as described in any one of the above.

[0051] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the training method of the detection model as described in any one of the above is implemented.

[0052] The training method and detection method of the detection model provided by the present invention include: obtaining a plurality of training image samples and preprocessing the training image samples; for any of the training image samples: inputting the preprocessed training image samples into the encoding module in the initial detection model to extract a plurality of depth feature maps of different scales; performing feature enhancement processing on the depth feature maps of different scales respectively to obtain enhanced feature maps of different scales; inputting the depth feature maps of different scales and the enhanced feature maps into the decoding module in the initial detection model to obtain the detection results corresponding to the training image samples; based on the detection results of each of the training image samples, iteratively training the initial detection model to obtain a target detection model. The present invention extracts depth feature maps of different scales through the encoding module, and then performs feature enhancement processing on the depth feature maps of different scales to extract more image features, improve the accuracy of the model for lung nodule detection, thereby detecting suspected lung nodules without first training the obtained detection model, and then performing secondary classification on the detected suspected lung nodules by training a false positive removal model, which greatly reduces the complexity of model training and effectively improves the efficiency of the detection model for image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 It is a flowchart of the training method of the detection model provided by the present invention;

[0055] Figure 2 It is a schematic diagram of a process of generating an enhanced feature map through a multi-view prediction module provided by the present invention;

[0056] Figure 3 It is a schematic diagram of a process of generating an enhanced feature map through a three-dimensional segmentation module provided by the present invention;

[0057] Figure 4 It is a schematic diagram of the process of the detection method provided by the present invention;

[0058] Figure 5 It is a structural schematic diagram of a training device for a detection model provided by the present invention;

[0059] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] The terms used in one or more embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present invention. The singular forms of "a", "said" and "the" used in one or more embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present invention refers to and includes any or all possible combinations of one or more associated listed items.

[0062] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present invention, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present invention, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when..." or "when...".

[0063] Figure 1 FIG. 1 is a flow chart of the training method of the detection model provided by the present invention. Figure 1 As shown, the training method of the detection model includes:

[0064] Step S11, obtaining a plurality of training image samples, and preprocessing the training image samples;

[0065] It should be noted that the training image samples are publicly available CT image data of lung nodules and clinically collected image data of lung nodules. Optionally, the training image samples are associated with manually annotated information, such as the location and size information of lung nodules in the image data.

[0066] Specifically, the training image samples are first resampled, and optionally, a specific resolution is required to eliminate the resolution difference of scanning by different devices, and then the distribution range is determined according to the HU value of the training image samples to normalize the data. It should be noted that in medical image processing, the HU value (Hounsfield Unit) is usually used to measure the relative density of tissues in CT (computed tomography) images. Further, the training image samples are segmented using an image segmentation algorithm to segment the image lung area, and the image segmentation algorithm includes algorithms such as a watershed segmentation algorithm, a threshold segmentation algorithm, and an edge-based segmentation algorithm. Further, the training image samples are subjected to data enhancement processing, and optionally, the data enhancement includes methods such as random flipping and rotation.

[0067] Step S12, for any of the training image samples: input the preprocessed training image sample into the encoding module in the initial detection model to extract a plurality of depth feature maps of different scales;

[0068] Specifically, for any of the training image samples, the following steps are performed: the training image sample is intercepted using a preset sliding window to obtain a plurality of three-dimensional image blocks, and then the depth feature maps of different scales of each image block are extracted. For example, the step size of the sliding window is 48, the size of the intercepted three-dimensional image block is 96*96*96, and the sizes of the depth feature maps of different scales extracted by the three-dimensional feature pyramid network are 48*48*48, 24*24*24, and 12*12*12, respectively.

[0069] Step S13, performing feature enhancement processing on the depth feature maps of different scales respectively to obtain enhanced feature maps of different scales;

[0070] Specifically, in one embodiment, the initial detection model includes a multi-view prediction module. The following steps are performed for the depth feature map of any scale: the depth feature map is divided into multiple groups of two-dimensional feature sequences of different views, and then the two-dimensional feature sequences of different views are respectively input into the multi-view prediction module in the initial detection model for prediction, and the prediction results corresponding to the two-dimensional feature sequences of different views are obtained. Furthermore, based on the prediction results of different views, the first weight maps of different views are constructed; the construction process of the first weight map is specifically described in the following embodiments and will not be repeated here. Then, the first weight maps of different views are averaged to obtain a target weight map, and then the depth feature map and the target weight map are feature multiplied, and then the multiplied feature map and the depth feature map are feature added to obtain the enhanced feature map.

[0071] In one embodiment, the initial detection model also includes a three-dimensional shape estimation module, and the three-dimensional shape estimation module includes a three-dimensional convolution layer; the following steps are performed for the depth feature map of any scale: the depth feature map is convolved using a three-dimensional convolution layer to obtain a segmentation result, and then based on the segmentation result, a second weight map is constructed. The construction process of the weight map is specifically described in the following embodiment and is not repeated here. Then, the second weight map and the depth feature map are fused, and optionally, the second weight map and the depth feature map are feature multiplied, and then the multiplied feature map and the depth feature map are feature added to obtain the enhanced feature map.

[0072] Step S14, inputting the depth feature maps and enhanced feature maps of different scales into the decoding module in the initial detection model to obtain the detection results corresponding to the training image samples;

[0073] It should be noted that the depth feature maps and enhanced feature maps of different scales include a depth feature map of a first scale, a depth feature map of a second scale, a depth feature map of a third scale and an enhanced feature map of a first scale, an enhanced feature map of a second scale and an enhanced feature map of a third scale; wherein the first scale is smaller than the second scale, and the second scale is smaller than the third scale.

[0074] Specifically, the depth feature map and the enhanced feature map of the first scale are first upsampled to obtain a first upsampled feature map, and then the first upsampled feature map is convolved to obtain a first convolutional feature map, and further, the first convolutional feature map is fused with the depth feature map and the enhanced feature map of the second scale to obtain a first fused feature map. The fused feature map is then upsampled to obtain a second upsampled feature map; the second upsampled feature map is then convolved to obtain a second convolutional feature map, and the second convolutional feature map is fused with the depth feature map and the enhanced feature map of the third scale to obtain a second fused feature map, and finally the second fused feature map is convolved to obtain the detection result.

[0075] Optionally, the number of upsampling, convolution and feature fusion processing is related to the scale level. For example, if there are only feature maps of the first scale and the second scale, after obtaining the first fused feature map, the first fused feature map is convolved to obtain the detection result.

[0076] Step S15, based on the detection results of each of the training image samples, iteratively train the initial detection model to generate an actual classification prediction model.

[0077] Specifically, based on the detection results of each of the training image samples and the position annotation information corresponding to each of the training image samples, a model loss value is calculated, and then based on the model loss value, the model parameters of the initial detection model are iteratively optimized to obtain a well-trained target detection model.

[0078] In one embodiment, after the two-dimensional feature sequences of the different views are respectively input into the multi-view prediction module in the initial detection model for prediction, and the prediction results corresponding to the two-dimensional feature sequences of the different views are obtained, the first loss value can be calculated based on the prediction results and the preset position annotation information. For example, the loss function can be set according to the actual situation, and no specific limitation is made here. Thus, the model parameters of the initial detection model can be iteratively optimized by combining the first loss value and the above-mentioned model loss value to obtain a well-trained target detection model.

[0079] In one embodiment, the convolution layer in the three-dimensional segmentation module performs convolution processing on the depth feature map, and after obtaining the segmentation result, generates three-dimensional mask information based on preset position annotation information; determines a second loss value based on the three-dimensional mask information and the segmentation result; and calculates a third loss value using a contrast loss function based on the position annotation information and the segmentation result. The calculation process of the second loss value and the third loss value is specifically described in the following embodiments and will not be repeated here. The second loss value, the third loss value and the above-mentioned model loss value can then be combined to iteratively optimize the model parameters of the initial detection model to obtain a well-trained target detection model.

[0080] In one embodiment, the first loss value, the second loss value, the third loss value and the above-mentioned model loss value can also be combined to iteratively optimize the model parameters of the initial detection model to obtain a well-trained target detection model.

[0081] In the process of combining multiple loss values ​​for model parameter optimization, multiple loss values ​​can be weighted to obtain a total loss value, so as to optimize the model parameters of the initial detection model based on the total loss value to obtain a well-trained target detection model.

[0082] The embodiment of the present invention, through the above scheme, includes: obtaining a number of training image samples, and preprocessing the training image samples; for any of the training image samples: inputting the preprocessed training image samples into the encoding module in the initial detection model to extract a plurality of depth feature maps of different scales; performing feature enhancement processing on the depth feature maps of different scales respectively to obtain enhanced feature maps of different scales; inputting the depth feature maps of different scales and the enhanced feature maps into the decoding module in the initial detection model to obtain the detection results corresponding to the training image samples; based on the detection results of each of the training image samples, iteratively training the initial detection model to obtain the target detection model. The embodiment of the present invention extracts depth feature maps of different scales through the encoding module, and then performs feature enhancement processing on the depth feature maps of different scales to extract more image features, thereby improving the accuracy of the model for lung nodule detection, thereby eliminating the need to first detect suspected lung nodules through a detection model, and then using a false positive removal model to perform secondary classification on the detected suspected lung nodules, greatly reducing the complexity of model training, and effectively improving the efficiency of the detection model for image detection.

[0083] In one embodiment of the present invention, the performing feature enhancement processing on the depth feature maps of different scales to obtain enhanced feature maps of different scales includes:

[0084] For a deep feature map of any scale:

[0085] Dividing the depth feature map into two-dimensional feature sequences of different views;

[0086] Inputting the two-dimensional feature sequences of different views into the multi-view prediction module in the initial detection model for prediction respectively, and obtaining prediction results corresponding to the two-dimensional feature sequences of different views;

[0087] Based on the prediction results of different views, first weight maps of different views are constructed;

[0088] The enhanced feature map is generated based on the depth feature map and the first weight map of the different views.

[0089] Specifically, the following steps are performed for the depth feature map of any scale: first, the depth feature map is divided into two-dimensional feature sequences of different views, and then the multi-view prediction module in the initial detection model is used to detect and predict the two-dimensional feature sequences of the different views respectively, and the prediction results corresponding to the two-dimensional feature sequence of each view are obtained. Optionally, the prediction results include the prediction probability of the lung nodule bounding box at each position, and then based on the prediction probability of each position corresponding to the two-dimensional feature sequence of any view, the first weight map of any view is constructed. Furthermore, the first weight maps of different views are averaged to obtain the final target weight map, and feature fusion is performed based on the depth feature map and the target weight map to obtain the enhanced feature map.

[0090] Understandably, referring to Figure 2 , Figure 2It is a flow chart of generating an enhanced feature map through a multi-view prediction module provided by the present invention. For an input three-dimensional depth feature map of size z*h*w*c, where z, h, w represent the depth, height and width of the depth feature map respectively, and c is the number of channels of the feature map, the depth feature map is first divided into three groups of two-dimensional feature sequences of multiple views, and the two-dimensional feature sequences of multiple views include axial plane feature sequences, coronal plane feature sequences, and sagittal plane feature sequences, with sizes of z*(h*w*c), h*(z*w*c), and w*(z*h*c), respectively. Optionally, CT (computed tomography) images usually display human body structures through three main anatomical planes, namely, the axial plane, the coronal plane, and the sagittal plane. These three planes provide slices in different directions to more fully understand the anatomical structure. Axial plane: This is a plane along the longitudinal axis of the body (from head to toe). In a CT scan, the axial plane is usually a transverse slice parallel to the ground. This plane provides cross-sectional information about the body structure and is suitable for viewing internal organs, lung nodules, etc. Coronal Plane: This is a plane along the front-to-back direction of the body. In CT scans, the coronal plane is usually a longitudinal slice, perpendicular to the axial plane. Through the coronal plane, the structure of the body from front to back can be observed, which is suitable for viewing the cross-sections of the head and neck, chest and abdomen. Sagittal Plane: This is a plane along the left-to-right direction of the body. In CT scans, the sagittal plane is also usually a longitudinal slice, perpendicular to the axial plane. Through the sagittal plane, the structure of the body from left to right can be observed, which is suitable for viewing the cross-sections of the head and neck, spine and joints. Furthermore, each group of two-dimensional feature sequences predicts the lung nodules of each two-dimensional feature sequence through a two-dimensional detection module with shared parameters, and obtains the prediction results corresponding to the three groups of two-dimensional feature sequences. For example, for a two-dimensional feature sequence of size z*(h*w*c), the prediction result is z*(h*w*n*7), which means that for each of the z sections, the feature map size is h*w, the number of channels of the feature map is c, and 7 represents the predicted probability of the lung nodule bounding box at each position and the regression value of the lung nodule center and size. The maximum predicted probability is taken at each position to construct a weight map of size h*w. The three sets of feature sequences finally obtain three weight maps of size z*h*w.

[0091] The embodiment of the present invention divides the depth feature map of any scale into two-dimensional feature sequences of different views to extract feature information of different views, and then constructs a weight map based on the prediction results corresponding to the two-dimensional feature sequences of different views, so as to generate the enhanced feature map based on the depth feature map and the weight maps of different views to extract more feature information of lung nodules and improve the accuracy of model detection.

[0092] In one embodiment of the present invention, generating the enhanced feature map based on the depth feature map and the first weight map of the different views includes:

[0093] Performing an average operation on the first weight maps of different views to obtain a target weight map;

[0094] The target weight map is subjected to feature fusion with the depth feature map to obtain the enhanced feature map.

[0095] Specifically, for a depth feature map of any scale: average the first weight maps of the different views to obtain a target weight map, and then perform feature multiplication on the target weight map and the depth feature map to obtain a target feature map, and further perform feature addition on the target feature map and the depth feature map to obtain the enhanced feature map. For example: three groups of feature sequences ultimately obtain three first weight maps of size z*h*w, and average the three first weight maps to obtain a target weight map of a final size z*h*w. Multiply the target weight map with the original input depth feature map of size z*h*w*c to obtain a target feature map, and then add the target feature map to the input depth feature map to obtain the final enhanced feature map. Enhanced feature maps of different scales are thereby obtained.

[0096] The embodiment of the present invention obtains an enhanced feature map by fusing the first weight map and the depth feature map of the different views, so as to extract more feature information of the lung nodules and improve the accuracy of model detection.

[0097] In one embodiment of the present invention, the performing feature enhancement processing on the depth feature maps of different scales to obtain enhanced feature maps of different scales includes:

[0098] For a deep feature map of any scale:

[0099] Performing convolution processing on the depth feature map to obtain a segmentation result;

[0100] Based on the segmentation result, construct a second weight map;

[0101] Based on the second weight map and the depth feature map, the enhanced feature map is generated.

[0102] It should be noted that, refer to Figure 3 , Figure 3It is a flow chart of generating an enhanced feature map through a three-dimensional segmentation module provided by the present invention. The initial detection model includes a three-dimensional segmentation module, and the three-dimensional segmentation module includes a three-dimensional convolution layer. Specifically, the three-dimensional segmentation module is used to perform convolution processing on the depth feature map to obtain a segmentation result, and the segmentation result includes the probability that each pixel belongs to a lung nodule, and then based on the probability of each pixel in the segmentation result, a second weight map is constructed, and further, the target weight map is feature multiplied with the depth feature map to obtain a target feature map, and then the target feature map and the depth feature map are feature added to obtain the enhanced feature map.

[0103] The embodiment of the present invention performs convolution processing on the depth feature map by utilizing a three-dimensional segmentation module, and then constructs a weight map based on the segmentation result, and then generates the enhanced feature map based on the weight map and the depth feature map, thereby extracting more feature information of lung nodules and improving the accuracy of model detection.

[0104] In one embodiment of the present invention, after the convolution layer is used to perform convolution processing on the depth feature map to obtain a segmentation result, the method further includes:

[0105] Generate three-dimensional mask information based on preset position annotation information;

[0106] Determining a second loss value based on the three-dimensional mask information and the segmentation result;

[0107] Based on the position annotation information and the segmentation result, a third loss value is calculated using a contrast loss function;

[0108] The second loss value and the third loss value are used to optimize the model parameters of the initial detection model.

[0109] Specifically, in one embodiment, based on the lung nodule bounding box annotated in the position annotation information, a three-dimensional mask information is generated, and the three-dimensional mask information is used to distinguish the foreground (i.e., the lung nodule) and the background in the image. Then, the segmentation result of the predicted lung nodule is compared with the three-dimensional mask information, for example, the intersection over union (IoU) or binary cross entropy loss function can be used to calculate the second loss value.

[0110] In another embodiment, in order to improve the model's ability to distinguish between background and foreground features. Feature extraction is performed on different areas in the image (i.e., the background within the bounding box, the background outside the bounding box, and the foreground within the bounding box) to obtain feature representations of these areas. Then, based on the position annotation information and the segmentation result, the distance between the positive sample pairs (the feature representations of the background within the bounding box and the background outside the bounding box) and the distance between the negative sample pairs (the feature representations of the background within the bounding box and the foreground) are calculated. Usually this distance is calculated using some metric methods, such as Euclidean distance or cosine similarity. Further, the third loss value is calculated using a contrast loss function. The contrast loss function usually has multiple forms, but the basic principle is to make the distance between the positive sample pairs as small as possible and the distance between the negative sample pairs as large as possible.

[0111] Furthermore, after calculating the second loss value and the third loss value, the parameters of the model can be optimized in combination with the second loss value and the third loss value during the training process, thereby improving the model's ability to understand and distinguish different areas in the image and improving the accuracy of model detection.

[0112] In one embodiment of the present invention, the step of inputting the depth feature maps and enhanced feature maps of different scales into a decoding module in the initial detection model to obtain the detection results corresponding to the training image samples includes:

[0113] Upsampling the depth feature map and the enhanced feature map of the first scale to obtain a first upsampled feature map;

[0114] Performing convolution processing on the first up-sampled feature map to obtain a first convolution feature map;

[0115] Performing feature fusion processing on the first convolution feature map, the depth feature map of the second scale, and the enhanced feature map to obtain a first fused feature map;

[0116] Upsampling the fused feature map to obtain a second upsampled feature map;

[0117] Performing convolution processing on the second up-sampled feature map to obtain a second convolution feature map;

[0118] Performing feature fusion processing on the second convolution feature map, the depth feature map of the third scale, and the enhanced feature map to obtain a second fused feature map;

[0119] The second fused feature map is convolved to obtain the detection result.

[0120] Specifically, for feature maps of different scales corresponding to any three-dimensional image block in each of the training image samples, the following steps are performed: upsampling the depth feature map and the enhanced feature map of the first scale to obtain a first upsampled feature map, and optionally, the scale of the first upsampled feature map is the same as the second scale. Then, convolution processing is performed on the first upsampled feature map to obtain a first convolution feature map. Further, the first convolution feature map is feature added with the depth feature map and the enhanced feature map of the second scale to obtain a first fused feature map. Then, upsampling is performed on the fused feature map to obtain a second upsampled feature map; optionally, the scale of the second upsampled feature map is the same as the third scale, and then convolution processing is performed on the second upsampled feature map to obtain a second convolution feature map, further, feature addition is performed on the second convolution feature map with the depth feature map and the enhanced feature map of the third scale to obtain a second fused feature map, and finally, convolution processing is performed on the second fused feature map to obtain the detection result.

[0121] Understandably: Assume that the sizes of multi-scale feature maps are 48*48*48, 24*24*24, and 12*12*12 respectively. In the decoding module, the feature map of size 12*12*12 is first upsampled to obtain a feature map of size 24*24*24, and the feature map of size 24*24*24 is convolved through a three-dimensional convolution layer and then added to the feature map of size 24*24*24 to obtain a fused feature map of 24*24*24, and the fused feature map is upsampled to obtain a feature map of size 48*48*48, and the feature map of size 48*48*48 is convolved and then added to the feature map of size 48*48*48 to obtain a fused feature map of size 48*48*48, and the final detection result is output after a convolution block.

[0122] The embodiment of the present invention uses the above scheme to determine the detection results of the training image samples by combining the depth feature maps of different scales and the enhanced feature maps of different scales after feature enhancement processing, thereby improving the accuracy of model detection.

[0123] Figure 4 Schematic diagram of the detection method provided by the present invention. Figure 4 As shown, the detection method includes:

[0124] Step S21, obtaining an image to be detected, and preprocessing the image to be detected;

[0125] Step S22, inputting the preprocessed image to be detected into the encoding module in the target detection model to extract target depth feature maps of multiple different scales;

[0126] Step S23, performing feature enhancement processing on the target depth feature maps of different scales respectively to obtain target enhanced feature maps of different scales;

[0127] Step S24, inputting the target depth feature maps and target enhancement feature maps of different scales into the decoding module in the target detection model to obtain the lung nodule detection result output by the detection model.

[0128] Specifically, the complete detection process of this embodiment is as follows: first, the image to be detected is obtained, and the image to be detected is preprocessed and data enhanced. The preprocessing and data enhancement processing are the same as the processing methods in the model training stage, which will not be repeated here. Then, the preprocessed image to be detected is input into the encoding module in the target detection model, so as to extract target depth feature maps of multiple different scales. Further, for the depth feature map of any scale: the target depth feature map is divided into two-dimensional feature sequences of different views; the two-dimensional feature sequences of different views are respectively input into the multi-view prediction module in the target detection model for prediction, and the prediction results corresponding to the two-dimensional feature sequences of different views are obtained; based on the prediction results of different views, weight maps of different views are constructed; based on the depth feature map and the weight map of different views, the target enhanced feature map is generated. And the target depth feature map can be convolved to obtain a segmentation result; based on the segmentation result, a weight map is constructed; based on the weight map and the target depth feature map, the target enhanced feature map is generated. Furthermore, the target depth feature maps and target enhancement feature maps of different scales are input into the decoding module in the target detection model, so as to obtain the lung nodule detection result output by the detection model.

[0129] The embodiment of the present invention uses a target detection model to detect lung nodules, and can extract target depth feature maps and target enhancement feature maps of different scales, thereby improving the accuracy of the model in detecting lung nodules. There is no need to first detect suspected lung nodules through a detection model and then use a false positive removal model to perform secondary classification on the detected suspected lung nodules, thereby effectively improving the efficiency of the detection model image detection.

[0130] The following is a description of the training device for the detection model provided by the present invention. The training device for the detection model described below and the training method for the detection model described above can refer to each other.

[0131] Figure 5 Schematic diagram of the structure of the training device of the detection model provided by the present invention, such as Figure 5 As shown, a detection model training device according to an embodiment of the present invention includes:

[0132] An acquisition module 21 acquires a plurality of training image samples and pre-processes the training image samples;

[0133] The feature extraction module 22 is used for: for any of the training image samples: inputting the preprocessed training image sample into the encoding module in the initial detection model to extract a plurality of depth feature maps of different scales;

[0134] A feature enhancement module 23 is used to perform feature enhancement processing on the depth feature maps of different scales to obtain enhanced feature maps of different scales;

[0135] A detection module 24, used for inputting the depth feature maps and enhanced feature maps of different scales into a decoding module in the initial detection model to obtain a detection result corresponding to the training image sample;

[0136] The training module 25 is used to iteratively train the initial detection model based on the detection results of each of the training image samples to obtain a target detection model.

[0137] The training device of the detection model also includes:

[0138] For a deep feature map of any scale:

[0139] Dividing the depth feature map into two-dimensional feature sequences of different views;

[0140] Inputting the two-dimensional feature sequences of different views into the multi-view prediction module in the initial detection model for prediction respectively, and obtaining prediction results corresponding to the two-dimensional feature sequences of different views;

[0141] Based on the prediction results of different views, first weight maps of different views are constructed;

[0142] The enhanced feature map is generated based on the depth feature map and the first weight map of the different views.

[0143] The training device of the detection model also includes:

[0144] Performing an average operation on the first weight maps of different views to obtain a target weight map;

[0145] The target weight map is subjected to feature fusion with the depth feature map to obtain the enhanced feature map.

[0146] The training device of the detection model also includes:

[0147] For any prediction result corresponding to the two-dimensional feature sequence: based on the prediction result and preset position annotation information, a first loss value is calculated, wherein the first loss value is used to optimize the model parameters of the initial detection model.

[0148] The training device of the detection model also includes:

[0149] For a deep feature map of any scale:

[0150] Performing convolution processing on the depth feature map to obtain a segmentation result;

[0151] Based on the segmentation result, construct a second weight map;

[0152] Based on the second weight map and the depth feature map, the enhanced feature map is generated.

[0153] The training device of the detection model also includes:

[0154] Generate three-dimensional mask information based on preset position annotation information;

[0155] Determining a second loss value based on the three-dimensional mask information and the segmentation result;

[0156] Based on the position annotation information and the segmentation result, a third loss value is calculated using a contrast loss function;

[0157] The second loss value and the third loss value are used to optimize the model parameters of the initial detection model.

[0158] The training device of the detection model also includes:

[0159] The depth feature maps and enhanced feature maps of different scales include a depth feature map of a first scale, a depth feature map of a second scale, a depth feature map of a third scale, and an enhanced feature map of the first scale, an enhanced feature map of the second scale, and an enhanced feature map of the third scale;

[0160] The training device of the detection model also includes:

[0161] Upsampling the depth feature map and the enhanced feature map of the first scale to obtain a first upsampled feature map;

[0162] Performing convolution processing on the first up-sampled feature map to obtain a first convolution feature map;

[0163] Performing feature fusion processing on the first convolution feature map, the depth feature map of the second scale, and the enhanced feature map to obtain a first fused feature map;

[0164] Upsampling the fused feature map to obtain a second upsampled feature map;

[0165] Performing convolution processing on the second up-sampled feature map to obtain a second convolution feature map;

[0166] Performing feature fusion processing on the second convolution feature map, the depth feature map of the third scale, and the enhanced feature map to obtain a second fused feature map;

[0167] The second fused feature map is convolved to obtain the detection result.

[0168] The training device of the detection model also includes:

[0169] Using a preset sliding window to intercept the training image samples to obtain a plurality of three-dimensional image blocks;

[0170] Feature extraction is performed on each of the three-dimensional image blocks to obtain depth feature maps of different scales.

[0171] It should be noted here that the above-mentioned device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0172] Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor 310, a memory 320, a communication interface 330 and a communication bus 340, wherein the processor 310, the memory 320 and the communication interface 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 320 to execute the training method of the detection model.

[0173] In addition, the logic instructions in the above-mentioned memory 320 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0174] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the training method of the detection model provided by the above-mentioned methods.

[0175] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the training methods of the detection models provided by the above methods.

[0176] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0177] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training a detection model, characterized in that: include: Acquire a plurality of training image samples, and preprocess the training image samples; For any of the training image samples: inputting the preprocessed training image sample into the encoding module in the initial detection model to extract a plurality of depth feature maps of different scales; Performing feature enhancement processing on the depth feature maps of different scales respectively to obtain enhanced feature maps of different scales; Inputting the depth feature maps and enhanced feature maps of different scales into the decoding module in the initial detection model to obtain the detection results corresponding to the training image samples; Based on the detection results of each of the training image samples, iteratively training the initial detection model to obtain a target detection model; The performing feature enhancement processing on the depth feature maps of different scales respectively to obtain enhanced feature maps of different scales includes: For a deep feature map of any scale: Dividing the depth feature map into two-dimensional feature sequences of different views; Inputting the two-dimensional feature sequences of different views into the multi-view prediction module in the initial detection model for prediction respectively, and obtaining prediction results corresponding to the two-dimensional feature sequences of different views; Based on the prediction results of different views, first weight maps of different views are constructed; The enhanced feature map is generated based on the depth feature map and the first weight map of the different views.

2. The method for training a detection model according to claim 1, characterized in that: The step of generating the enhanced feature map based on the depth feature map and the first weight map of the different views comprises: Performing an average operation on the first weight maps of different views to obtain a target weight map; The target weight map is subjected to feature fusion with the depth feature map to obtain the enhanced feature map.

3. The method for training a detection model according to claim 1, characterized in that: After respectively inputting the two-dimensional feature sequences of different views into the multi-view prediction module in the initial detection model for prediction and obtaining the prediction results corresponding to the two-dimensional feature sequences of different views, the method further includes: For any prediction result corresponding to the two-dimensional feature sequence: based on the prediction result and preset position annotation information, a first loss value is calculated, wherein the first loss value is used to optimize the model parameters of the initial detection model.

4. The method for training a detection model according to claim 1, characterized in that: The performing feature enhancement processing on the depth feature maps of different scales respectively to obtain enhanced feature maps of different scales includes: For a deep feature map of any scale: Performing convolution processing on the depth feature map to obtain a segmentation result; Based on the segmentation result, construct a second weight map; Based on the second weight map and the depth feature map, the enhanced feature map is generated.

5. The method for training a detection model according to claim 4, characterized in that: After performing convolution processing on the depth feature map to obtain a segmentation result, the method further includes: Generate three-dimensional mask information based on preset position annotation information; Determining a second loss value based on the three-dimensional mask information and the segmentation result; Based on the position annotation information and the segmentation result, a third loss value is calculated using a contrast loss function; The second loss value and the third loss value are used to optimize the model parameters of the initial detection model.

6. The method for training a detection model according to claim 1, characterized in that: The depth feature maps and enhanced feature maps of different scales include a depth feature map of a first scale, a depth feature map of a second scale, a depth feature map of a third scale, and an enhanced feature map of the first scale, an enhanced feature map of the second scale, and an enhanced feature map of the third scale; The first scale is smaller than the second scale, and the second scale is smaller than the third scale.

7. The method for training a detection model according to claim 6, characterized in that: The step of inputting the depth feature maps and enhanced feature maps of different scales into a decoding module in the initial detection model to obtain a detection result corresponding to the training image sample includes: Upsampling the depth feature map and the enhanced feature map of the first scale to obtain a first upsampled feature map; Performing convolution processing on the first up-sampled feature map to obtain a first convolution feature map; Performing feature fusion processing on the first convolution feature map, the depth feature map of the second scale, and the enhanced feature map to obtain a first fused feature map; Upsampling the fused feature map to obtain a second upsampled feature map; Performing convolution processing on the second up-sampled feature map to obtain a second convolution feature map; Performing feature fusion processing on the second convolution feature map, the depth feature map of the third scale, and the enhanced feature map to obtain a second fused feature map; The second fused feature map is convolved to obtain the detection result.

8. The method for training a detection model according to claim 1, characterized in that: The preprocessed training image samples are input into the encoding module in the initial detection model to extract multiple depth feature maps of different scales, including: Using a preset sliding window to intercept the training image samples to obtain a plurality of three-dimensional image blocks; Feature extraction is performed on each of the three-dimensional image blocks to obtain depth feature maps of different scales.

9. A detection method, characterized in that: include: Acquire an image to be detected, and preprocess the image to be detected; The preprocessed image to be detected is input into the encoding module in the target detection model to extract target depth feature maps of multiple different scales; Performing feature enhancement processing on the target depth feature maps of different scales respectively to obtain target enhanced feature maps of different scales; Inputting the target depth feature maps and target enhancement feature maps of different scales into a decoding module in the target detection model to obtain a lung nodule detection result output by the detection model; Wherein, the target detection model is obtained based on the training method of the detection model described in any one of claims 1 to 8.

Citation Information

Patent Citations

  • Double-attention generative adversarial network for noise reduction and artifact removal of low-dose CT image

    CN111968195A

  • Method for generating gadolinium contrast agent enhanced magnetic resonance image

    CN113379616A

  • Coding and decoding network glioma fusion segmentation system and method based on attention enhancement

    CN115661165A

  • Pulmonary nodule CT image segmentation algorithm based on REMU-Net

    CN115690126A

  • Image feature enhancement method and related method and device

    CN116563152A