A target detection model training method, a detection method and a system

Abstract

The application discloses a kind of training method, detection method and system of target detection model, wherein the training method obtains the sample image of target sample;The sample image is extracted in multiple dimensions, and a sample feature set is obtained, the sample feature set includes several sample feature maps obtained based on the extraction order of the multiple dimension feature extraction, the feature dimension of the subsequent sample feature map is less than the feature dimension of the previous sample feature map;The sample feature set is sampled and adaptively fused on the feature pyramid, and several fusion feature maps are obtained;According to all the fusion feature maps, the initialized target detection model is updated, and the trained target detection model is obtained.The training method can make the model fully utilize the small target features of the image, thereby facilitating to improve the accuracy of the model in detecting small targets.The application relates to the field of image processing technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a target detection model training method, a detection method and a system. BACKGROUND

[0002] Target detection is a core technology in the field of computer vision, aiming to automatically identify and locate specific targets from images or videos. Specifically, in natural light scenes, the detection network used by the target detection technology generally includes YOLO series-based detection network, Faster RCNN-based detection network and SSD (Single Shot MultiBox Detector) -based detection network.

[0003] Currently, existing SSD detection technology generally performs down-sampling operation on the multi-scale feature maps output by the neural network after the image is input into the corresponding neural network, and fuses and detects the features of different levels. This method cannot fully utilize the small target features of the image, and the accuracy of small target detection is not satisfactory.

[0004] Therefore, the existing problems need to be solved and optimized. SUMMARY

[0005] The present application aims to at least partially solve one of the technical problems in the related art.

[0006] To this end, one object of the present application is to provide a target detection model training method, a detection method and a system, wherein the training method can make the model fully utilize the small target features of the image, thereby facilitating the improvement of the accuracy of the model in detecting small targets.

[0007] In order to achieve the above technical purpose, the technical solutions adopted by the embodiments of the present application include:

[0008] In a first aspect, the present application provides a target detection model training method, comprising:

[0009] obtaining a sample image of a target sample;

[0010] performing multi-dimensional feature extraction on the sample image to obtain a sample feature set, the sample feature set including a plurality of sample feature maps obtained based on the extraction order of the multi-dimensional feature extraction, the feature dimension of a subsequent sample feature map being smaller than that of a previous sample feature map;

[0011] The sample feature set is subjected to feature pyramid upsampling and adaptive fusion to obtain a plurality of fused feature maps, each of which is used to represent feature fusion information of a first feature map and a second feature map subjected to upsampling, the first feature map being a sample feature map with the same feature dimension as the fused feature map, and the second feature map being a subsequent sample feature map of the first feature map.

[0012] According to all the fused feature maps, the initialized target detection model is subjected to parameter updating to obtain a trained target detection model.

[0013] In addition, the method according to the above-mentioned embodiments of the present application can have the following additional technical features:

[0014] Further, in an embodiment of the present application, the multi-dimensional feature extraction on the sample image to obtain a sample feature set comprises:

[0015] The sample image is subjected to feature dimension increasing processing to obtain a dimension-increased feature map.

[0016] The dimension-increased feature map is subjected to feature extraction by a plurality of cascaded dimension-reduced feature extraction units to obtain a sample feature map output by each of the dimension-reduced feature extraction units.

[0017] Further, in an embodiment of the present application, the feature pyramid upsampling and adaptive fusion on the sample feature set to obtain a fused feature map comprises:

[0018] The second feature map is subjected to feature upsampling to obtain a third feature map.

[0019] The first feature map and the third feature map are subjected to adaptive feature fusion processing to obtain the fused feature map.

[0020] Further, in an embodiment of the present application, the adaptive feature fusion processing on the first feature map and the third feature map to obtain the fused feature map comprises:

[0021] According to the first feature map and the third feature map, dynamic weight calculation is performed to obtain a first dynamic weight of the first feature map and a second dynamic weight of the third feature map.

[0022] According to the first dynamic weight and the first feature map, feature weighted fusion is performed on the second dynamic weight and the third feature map to obtain a fourth feature map.

[0023] The fourth feature map is subjected to feature enhancement to obtain the fused feature map.

[0024] Further, in an embodiment of the present application, the dynamic weight calculation according to the first feature map and the third feature map comprises:

[0025] performing similarity measurement calculation on the first feature map and the third feature map to obtain a feature similarity;

[0026] performing feature complementarity evaluation on the first feature map according to the feature similarity to obtain a first complementarity value of the first feature map, the first complementarity value being negatively correlated with the feature similarity;

[0027] performing feature complementarity evaluation on the third feature map according to the feature similarity to obtain a second complementarity value of the third feature map, the second complementarity value being negatively correlated with the feature similarity;

[0028] obtaining the first dynamic weight of the first feature map and the second dynamic weight of the third feature map according to the first complementarity value and the second complementarity value.

[0029] Further, in an embodiment of the present application, the method further comprises:

[0030] performing global normalization on the fusion feature map to obtain a normalized feature map;

[0031] performing feature enhancement processing on the normalized feature map to obtain an enhanced feature map;

[0032] performing attention extraction on the enhanced feature map to obtain a feature attention map;

[0033] performing attention weighted fusion update on the fusion feature map according to the feature attention map to obtain a fusion feature map after fusion update.

[0034] Further, in an embodiment of the present application, the feature enhancement processing on the normalized feature map to obtain an enhanced feature map comprises:

[0035] performing region segmentation on the normalized feature map to obtain a plurality of first feature regions;

[0036] performing region statistics on all the first feature regions to obtain a region mean value and a region standard deviation of each first feature region;

[0037] performing conditional feature enhancement on the corresponding first feature region according to the region mean value and the region standard deviation to obtain a second feature region;

[0038] obtaining the enhanced feature map according to all the second feature regions.

[0039] In a second aspect, an embodiment of the present application provides a detection method of a target detection model, comprising:

[0040] obtaining a target image of a target to be detected;

[0041] inputting the target image into the trained target detection model to perform detection and recognition, and obtaining a target detection result of the target to be detected.

[0042] In a third aspect, an embodiment of the present application provides a training system of a target detection model, comprising:

[0043] a first processing unit configured to obtain a sample image of a target sample;

[0044] a second processing unit configured to perform multi-dimensional feature extraction on the sample image to obtain a sample feature set, the sample feature set comprising a plurality of sample feature maps obtained based on an extraction order of the multi-dimensional feature extraction, a feature dimension of a subsequent sample feature map being smaller than a feature dimension of a previous sample feature map;

[0045] a third processing unit configured to perform feature pyramid upsampling and adaptive fusion on the sample feature set to obtain a plurality of fusion feature maps, each fusion feature map being used to represent feature fusion information of a first feature map and a second feature map after upsampling, the first feature map being a sample feature map with a same feature dimension as the fusion feature map, and the second feature map being a subsequent sample feature map of the first feature map;

[0046] a fourth processing unit configured to perform parameter updating on an initialized target detection model according to all the fusion feature maps to obtain a trained target detection model.

[0047] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising:

[0048] at least one processor;

[0049] at least one memory configured to store at least one program;

[0050] when the at least one program is executed by the at least one processor, the at least one processor implements the method described above.

[0051] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a program executable by a processor, and the program executable by the processor is used to implement the method described above when executed by the processor.

[0052] The advantages and beneficial effects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be learned through the practice of the present application:

[0053] The training method of the target detection model, the detection method and the system disclosed by the embodiments of the present application, wherein the training method obtains a sample image of a target sample; multi-dimensional feature extraction is performed on the sample image to obtain a sample feature set, the sample feature set including a plurality of sample feature maps obtained based on the extraction order of the multi-dimensional feature extraction, the feature dimension of a subsequent sample feature map being smaller than that of a previous sample feature map; feature pyramid upsampling and adaptive fusion are performed on the sample feature set to obtain a plurality of fusion feature maps, each fusion feature map being used to represent the feature fusion information of a first feature map and a second feature map after upsampling, the first feature map being a sample feature map with the same feature dimension as the fusion feature map, and the second feature map being a subsequent sample feature map of the first feature map; and the initialized target detection model is updated in parameters according to all the fusion feature maps to obtain a trained target detection model. The training method can make the model fully utilize the small target features in the image by performing adaptive fusion on the upsampled second feature map and the first feature map, thereby being conducive to improving the accuracy of the model in detecting small targets. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the present application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of clearly expressing part of the embodiments of the technical solutions of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0055] Figure 1 A flowchart of a training method of a target detection model provided by the embodiments of the present application;

[0056] Figure 2 A schematic diagram of the overall network structure of a target detection model provided by the embodiments of the present application;

[0057] Figure 3 A structural framework schematic diagram of a training system of a target detection model provided by the embodiments of the present application;

[0058] Figure 4 A structural schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0059] Embodiments of the present application are described below in detail with reference to the accompanying drawings, examples of which are shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary only, are for the purpose of explanation only, and are not to be taken as limiting of the present application. For the steps in the following embodiments, only for the convenience of setting out the description, the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification herein is for describing the embodiments of the present application only and is not intended to limit the present application.

[0061] At present, the existing SSD detection technology usually performs down-sampling operation on the multi-scale feature maps output by the convolutional neural network after the image is input into the convolutional neural network, and fuses and detects the features at different levels. Specifically, since the small target features in the feature map are quickly lost after the input feature map undergoes several times of down-sampling operation, this method cannot fully utilize the small target features of the image, and the accuracy of small target detection is not satisfactory.

[0062] In addition, the existing SSD detection technology usually simply adds the features when fusing the features at different levels, which may bring certain redundant and conflicting feature information, requires more computing resources, and generates poor quality fused features, and the accuracy and efficiency of small target detection are not satisfactory. In addition, there are some SSD detection technologies that can improve the detection performance of small targets of the SSD network model to a certain extent by constructing a multi-level feature detection model, reducing the screening threshold of small target samples, etc. However, since these methods cannot improve the attention of the SSD network model to small target information, the accuracy of small target detection is relatively low.

[0063] Therefore, the embodiments of the present application provide a training method of a target detection model, a detection method and a system, wherein the training method performs self-adaptive fusion on the second feature map with the first feature map after upsampling the second feature map, which can make the model fully utilize the small target features in the image, and suppress the redundant and conflicting feature information in the fused feature map, thereby improving the accuracy and efficiency of small target detection. In addition, the training method further performs feature normalization on the fused feature map, which can effectively reduce the influence of background noise on the small target features in the fused feature map, and performs feature enhancement on the normalized feature map (specifically, conditional feature enhancement is performed on each first feature region), which can effectively improve the attention of the model to the small target features, thereby improving the accuracy of small target detection.

[0064] With reference to Figure 1 and Figure 2 In the embodiments of the present application, a training method of a target detection model comprises:

[0065] Step 110: obtaining a sample image of a target sample;

[0066] In the embodiments of the present application, the target sample can be a sample to be detected by the target detection model, which can be a person, a vehicle, an airplane, etc. The sample image can be an image containing the target sample and used for training the target detection model.

[0067] Step 120: performing multi-dimensional feature extraction on the sample image to obtain a sample feature set, the sample feature set comprising a plurality of sample feature maps obtained in an extraction order based on the multi-dimensional feature extraction, a feature dimension of a subsequent sample feature map being smaller than a feature dimension of a previous sample feature map;

[0068] In the embodiments of the present application, the input feature map of the sample image can be extracted in each dimension based on the extraction order from large to small feature dimension, and each dimension feature extraction can be performed to obtain a sample feature map, which is sorted after the sample feature map of the previous feature dimension, thereby obtaining a plurality of sample feature maps with gradually decreasing feature dimensions.

[0069] In some embodiments, the multi-dimensional feature extraction on the sample image to obtain the sample feature set comprises:

[0070] performing feature dimension increasing processing on the sample image to obtain a dimension-increased feature map;

[0071] performing feature extraction on the dimension-increased feature map by a plurality of cascaded dimension-reduced feature extraction units to obtain a sample feature map output by each dimension-reduced feature extraction unit.

[0072] In the embodiments of the present application, the multi-dimensional feature extraction of the sample image can be realized by the MobileNet-V3 backbone network in the target detection model. Specifically, the feature dimension increasing processing can be to input the input feature map of the sample image into a 2d convolution layer (Conv2d) and three bottleneck layers (Bottleneck, referred to as Bneck) with different convolution kernel sizes layer by layer to increase the dimension, so as to obtain the dimension-increased feature map output by the last Bneck layer, and the feature dimension of the dimension-increased feature map is greater than that of the input feature map.

[0073] It can be understood that the MobileNet-V3 backbone network can include a plurality of dimension-reduced feature extraction units in cascade. Continuing to refer to Figure 2 In the embodiments of the present application, the number of the dimension-reduced feature extraction units in cascade is taken as an example of 6, the first dimension-reduced feature extraction unit can include a Bneck layer with a convolution kernel size of 5x5; the second dimension-reduced feature extraction unit can include a Bneck layer with a convolution kernel size of 5x5 and a 2d convolution layer; and the remaining dimension-reduced feature extraction units can respectively include a depthwise convolution layer (Depthwise Conv) and a 2d convolution layer; wherein the feature dimension corresponding to the latter dimension-reduced feature extraction unit is less than the feature dimension corresponding to the former dimension-reduced feature extraction unit.

[0074] Specifically, the feature extraction of the dimension-increased feature map can be to input the dimension-increased feature map as input into the first dimension-reduced feature extraction unit in all the dimension-reduced feature extraction units in cascade, so as to obtain the sample feature map output by each dimension-reduced feature extraction unit, and the feature dimension of the latter sample feature map is less than that of the former sample feature map. For example, for the sample feature map of the third dimension-reduced feature extraction unit, denoted as the third sample feature map, and the sample feature map of the fourth dimension-reduced feature extraction unit, denoted as the fourth sample feature map, the feature dimension of the third sample feature map is less than that of the fourth sample feature map, and the same applies to the remaining sample feature maps, which can be simply analogized.

[0075] Step 130, performing feature pyramid upsampling and adaptive fusion on the sample feature set to obtain a plurality of fusion feature maps, each fusion feature map being used to represent the feature fusion information of a first feature map and a second feature map after upsampling, the first feature map being a sample feature map with the same feature dimension as the fusion feature map, and the second feature map being a sample feature map after the first feature map;

[0076] In the embodiments of the present application, each sample feature map in the sample feature set is respectively denoted as a first sample feature map, a second sample feature map, a third sample feature map, a fourth sample feature map, a fifth sample feature map and a sixth sample feature map based on the corresponding dimension reduction feature extraction unit, for example, the first sample feature map is the sample feature map of the first dimension reduction feature extraction unit, and the fifth sample feature map is the sample feature map of the fifth dimension reduction feature extraction unit.

[0077] It can be understood that, in step 130, the remaining sample feature maps except the first sample feature map can be up-sampled based on the feature pyramid structure, and the remaining sample feature maps except the sixth sample feature map can be adaptively fused, so as to obtain the fusion feature map corresponding to each feature dimension.

[0078] In some embodiments, the feature pyramid up-sampling and adaptive fusion are performed on the sample feature set to obtain the fusion feature map, including:

[0079] The second feature map is up-sampled to obtain a third feature map.

[0080] In the embodiments of the present application, for any one sample feature map in the sample feature set except the sixth sample feature map, the sample feature map can be determined as a first feature map, and a second feature map of the first feature map is obtained. Specifically, for the third sample feature map in the sample feature set, the third sample feature map can be denoted as the first feature map, and the fourth sample feature map with a smaller feature dimension than the third sample feature map can be denoted as the second feature map.

[0081] It can be understood that, the feature up-sampling of the second feature map can be that the second feature map is sequentially input into a convolution layer with a kernel size of 3x3, a Batch Normalization Layer (BN) and a ReLU activation function layer, then the activated second feature map is up-sampled, and the specific up-sampling operation can be any one of Nearest Neighbor Interpolation, Bilinear Interpolation and Bicubic Interpolation, so as to obtain the third feature map with the same feature dimension as the first feature map.

[0082] The first feature map and the third feature map are adaptively fused to obtain the fusion feature map.

[0083] Further, the adaptive feature fusion processing of the first feature map and the third feature map to obtain the fusion feature map includes:

[0084] According to the first feature map and the third feature map, dynamic weight calculation is performed to obtain a first dynamic weight of the first feature map and a second dynamic weight of the third feature map;

[0085] Further, the dynamic weight calculation according to the first feature map and the third feature map to obtain the first dynamic weight of the first feature map and the second dynamic weight of the third feature map comprises:

[0086] Similarity measurement calculation is performed on the first feature map and the third feature map to obtain a feature similarity;

[0087] According to the feature similarity, feature complementarity evaluation is performed on the first feature map to obtain a first complementarity value of the first feature map, and the first complementarity value is negatively correlated with the feature similarity;

[0088] According to the feature similarity, feature complementarity evaluation is performed on the third feature map to obtain a second complementarity value of the third feature map, and the second complementarity value is negatively correlated with the feature similarity;

[0089] According to the first complementarity value and the second complementarity value, the first dynamic weight of the first feature map and the second dynamic weight of the third feature map are obtained.

[0090] In the embodiments of the present application, the adaptive feature fusion of the first feature map and the third feature map can be realized based on the dynamic weight between the first feature map and the third feature map, and for a certain first feature map and a corresponding third feature map, the relationship between the first feature map and the third feature map can be obtained through feature similarity measurement and feature complementarity evaluation. Specifically, the similarity measurement calculation between the first feature map and the third feature map can be based on any one of cosine similarity, Euclidean distance, KL divergence, etc. The embodiments of the present application take the KL divergence as an example, and the feature similarity between the first feature map and the third feature map can be represented as:

[0091]

[0092] Wherein, D KL (F1||F2) is the first feature similarity between the first feature map and the third feature map; F1 is the first feature map; F2 is the third feature map; C is the number of channels of the feature map, H and W are the height and width of the feature map respectively; is the feature value of the first feature map at position (i, j) and channel c; is the feature value of the second feature map at position (i, j) and channel c.

[0093] And the first complementarity value of the first feature map after feature complementarity evaluation can be represented as:

[0094]

[0095] wherein, W comp (F1,F2) is the first complementary value of the first feature map.

[0096] It can be understood that, since the KL divergence is asymmetric, the first feature similarity D KL (F1∥F2) and the second feature similarity D KL (F2∥F1) are usually different. Therefore, in the embodiments of the present application, if the first feature map and the third feature map are calculated by the way of the KL divergence, the first feature similarity D KL (F1∥F2) and the second feature similarity D KL (F2∥F1) can be obtained, and then the first complementary value of the first feature map and the second complementary value of the third feature map are determined based on the first feature similarity D KL (F1∥F2) and the second feature similarity D KL (F2∥F1), such as determining the first complementary value of the first feature map based on the first feature similarity, and determining the second complementary value of the third feature map based on the second feature similarity; or, the second complementary value of the third feature map can be determined based on the first feature similarity, and the first complementary value of the first feature map can be determined based on the second feature similarity, which is only for illustration, and does not limit the present application.

[0097] It should be noted that the first complementary value and the second complementary value can reflect the complementarity between the two feature maps, and the lower the similarity between the two feature maps, the higher the complementary value, that is, the first complementary value or the second complementary value is negatively correlated with the feature similarity. In addition, for similarity measurement calculation based on cosine similarity or Euclidean distance, the obtained feature similarity is one, at this time, the first complementary value of the first feature map and the second complementary value of the third feature map are equal, which will not be described here.

[0098] It is worth mentioning that after obtaining the first complementary value and the second complementary value, dynamic weights for feature fusion can be respectively allocated to the first feature map and the third feature map, wherein the first dynamic weight of the first feature map can be represented as:

[0099]

[0100] wherein, is the first dynamic weight of the first feature map; W comp (F1,F2) is the first complementary value of the first feature map; W comp (F2,F1) is the third complementary value of the third feature map.

[0101] It is worth noting that the second dynamic weight of the third feature map is similar to the content of the first dynamic weight of the first feature map, which can be simply analogized, and the present application will not be repeated here.

[0102] According to the first dynamic weight and the first feature map, the second dynamic weight and the third feature map are weighted and fused to obtain a fourth feature map;

[0103] The fourth feature map is enhanced to obtain the fusion feature map.

[0104] In the embodiments of the present application, the first dynamic weight and the corresponding first feature map can be used to weight and fuse the second dynamic weight and the corresponding third feature map based on weighted summation, weighted average and the like, so as to obtain the fourth feature map; then the fourth feature map is input into a lightweight feature enhancement module composed of a convolution layer and a ReLU activation function layer for enhancement, so as to obtain the fusion feature map output by the lightweight feature enhancement module, which can be represented as:

[0105]

[0106] Wherein, F enhanced is the fusion feature map; ReLU(·) is the activation function; Conv(·) is the convolution function.

[0107] Step 140, updating the parameters of the initialized target detection model according to all the fusion feature maps to obtain a trained target detection model.

[0108] In the embodiments of the present application, each fusion feature map can be detected and classified to obtain the detection probability value corresponding to each fusion feature map, and then the detection result corresponding to the sample image of the target sample is generated based on the maximum detection probability value; then, based on the Scale-aware Focal Loss loss function optimized based on the scale perception mechanism, denoted as Scale-aware Focal Loss loss function, and the Focal EIoU Loss loss function optimized based on the scale perception mechanism and the Focal loss idea, denoted as Focal EIoU Loss loss function, the target loss value is determined by the detection probability value, and the parameters of the model are updated by using the back propagation algorithm, and the trained target detection model can be obtained after several iterations. The specific number of iterations can be pre-set, or the training can be considered to be completed when the test set reaches the accuracy requirement.

[0109] Exemplarily, the Scale-aware Focal Loss loss function can be represented as:

[0110] L class= -a_pos * (1 - p_pos) γ * log(p_pos) * ω s

[0111] wherein, L class is a loss value of the Scale-aware Focal Loss loss function, denoted as a first sub-loss value; a_pos is a weight parameter of a positive sample; γ is an adjustable parameter; p_pos is a probability of being predicted as a positive sample; ω s is a scale-aware weight under a scale-aware mechanism.

[0112] The Focal EIoU Loss loss function can be expressed as:

[0113]

[0114] wherein, L reg is a loss value of the Focal EIoU Loss loss function, denoted as a second sub-loss value; IoU is an intersection over union of the predicted box and the real box; ρ is an Euclidean distance between the center points of the predicted box and the real box; ρ w and ρ h are the differences in width and height of the predicted box and the real box, respectively; r is a diagonal length of a minimum closure, which is used to represent a minimum rectangular box containing the predicted box and the real box; R w and R h are the width and height of the minimum closure, respectively; β is a focusing factor, used to enhance the punishment for easy classification boxes; ω s is a scale-aware weight.

[0115] It can be understood that after the first sub-loss value and the second sub-loss value are calculated, a simple summation operation can be performed on the first sub-loss value and the second sub-loss value to obtain the target loss value, which will not be described herein.

[0116] In some embodiments, the method further comprises:

[0117] performing global normalization on the fusion feature map to obtain a normalized feature map;

[0118] In the embodiments of the present application, in order to make the fusion feature map more uniform respectively, and reduce the influence of background noise on small target features in the fusion feature map, a global normalization operation can be performed on the fusion feature map. The specific global normalization operation can be any one of Min-Max normalization, Z-Score normalization, L2 norm normalization, etc., so as to obtain a normalized fusion feature map, denoted as a normalized feature map.

[0119] performing feature enhancement processing on the normalized feature map to obtain an enhanced feature map;

[0120] Further, the feature enhancement processing on the normalized feature map comprises:

[0121] regionally segmenting the normalized feature map to obtain a plurality of first feature regions;

[0122] statistically processing all the first feature regions to obtain a region mean value and a region standard deviation of each first feature region;

[0123] performing conditional feature enhancement on a corresponding first feature region according to the region mean value and the region standard deviation to obtain a second feature region;

[0124] obtaining the enhanced feature map according to all the second feature regions.

[0125] In the embodiments of the present application, the feature enhancement processing is used to enhance small target features in the normalized feature map. Specifically, the region segmentation can divide the normalized feature map into a plurality of local regions, and each local region is recorded as a first feature region. For example, the normalized feature map is divided into a plurality of local regions with a size of 5x5.

[0126] It can be understood that, in the first feasible implementation, the region statistics can be to calculate the mean value and the standard deviation of all the first feature regions, and the calculated mean value and standard deviation are respectively determined as the region mean value and the region standard deviation of each first feature region; or, in the second feasible implementation, for a first feature region, the region statistics can be to calculate the mean value and the standard deviation of all feature values of the first feature region, and the calculated mean value and standard deviation are respectively recorded as the region mean value and the region standard deviation, and the remaining first feature regions are the same, which will not be repeated here.

[0127] It should be noted that, for a feature value of a first feature region, the conditional feature enhancement can be to judge the size relationship between the feature value and the region mean value, and based on the size relationship and the region standard deviation, the feature value is selectively enhanced, and the remaining feature values of the first feature region are the same, which can be simply analogized. Exemplarily, the function expression of the conditional feature enhancement in the embodiments of the present application can be:

[0128]

[0129] wherein, E(·) is an expression function of the conditional feature enhancement; x is a feature value of a first feature region; a is a learnable parameter; σ is a region standard deviation of the first feature region; μ is a region mean value of the first feature region.

[0130] It is worth mentioning that after all the second feature regions are obtained, all the second feature regions can be reconstructed to obtain an enhanced normalized feature map, denoted as an enhanced feature map.

[0131] Attention extraction is performed on the enhanced feature map to obtain a feature attention map.

[0132] According to the feature attention map, attention weighted fusion update is performed on the fusion feature map to obtain a fusion updated fusion feature map.

[0133] In the embodiments of the present application, attention feature extraction and weighted fusion can be performed on the enhanced feature map based on the attention mechanism to obtain the fusion updated fusion feature map. Specifically, the enhanced feature map can be sequentially input into a 1x1 convolution layer, a ReLU activation function layer and a Sigmoid activation function layer to obtain a feature attention map of the enhanced feature map; then, the enhanced feature map and the feature attention map are weighted fused by element-wise multiplication operation to obtain the fusion updated fusion feature map.

[0134] In the embodiments of the present application, a detection method of a target detection model includes:

[0135] Step 210, obtaining a target image of a target to be detected;

[0136] Step 220, inputting the target image into the trained target detection model as described above for detection and recognition to obtain a target detection result of the target to be detected.

[0137] In the embodiments of the present application, the target image can be input into the trained target detection model for detection and recognition to obtain the target detection result output by the target detection model. For example, a target image containing a small airplane as a target to be detected can be input into the trained target detection model to obtain a target detection result about the small airplane as the target to be detected.

[0138] A training system of a target detection model according to the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0139] Referring to Figure 3 The training system of a target detection model according to the embodiments of the present application includes:

[0140] The first processing unit 101 is configured to obtain a sample image of a target sample.

[0141] The second processing unit 102 is configured to perform multi-dimensional feature extraction on the sample image to obtain a sample feature set, the sample feature set including a plurality of sample feature maps obtained based on an extraction sequence of the multi-dimensional feature extraction, a feature dimension of a subsequent sample feature map being smaller than a feature dimension of a previous sample feature map.

[0142] The third processing unit 103 is configured to perform feature pyramid upsampling and adaptive fusion on the sample feature set to obtain a plurality of fusion feature maps, each of the fusion feature maps being used to represent feature fusion information of a first feature map and a second feature map after upsampling, the first feature map being a sample feature map with a same feature dimension as the fusion feature map, and the second feature map being a subsequent sample feature map of the first feature map.

[0143] The fourth processing unit 104 is configured to perform parameter updating on an initialized target detection model according to all the fusion feature maps to obtain a trained target detection model.

[0144] It can be understood that the content in the above method embodiments is applicable to the present system embodiments, the present system embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0145] With reference to Figure 4 The present application embodiment further provides an electronic device, which comprises:

[0146] at least one processor 201;

[0147] at least one memory 202 configured to store at least one program;

[0148] When the at least one program is executed by the at least one processor 201, the at least one processor 201 implements the above method embodiments.

[0149] Similarly, it can be understood that the content in the above method embodiments is applicable to the present device embodiments, the present device embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0150] The present application embodiment further provides a computer readable storage medium, which stores a program executable by a processor 201, the program executable by the processor 201 being used to implement the above method embodiments when executed by the processor 201.

[0151] Similarly, the contents of the above method embodiments are applicable to the computer readable storage medium embodiments, the computer readable storage medium embodiments specifically implement the functions of the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0152] In some alternative embodiments, the functions / operations mentioned in the block diagrams can not occur in the order mentioned in the operation diagrams. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially concurrently with each other, or sometimes the blocks can be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts in this application are provided by way of example, and are intended to provide a more thorough and complete understanding of the technology. The disclosed methods are not limited to the operations and logical flows presented in this application. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of a larger operation are independently executed.

[0153] Furthermore, although the present application is described in the context of functional modules, it is to be understood that one or more of the functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules, unless otherwise specified. It is also to be understood that detailed discussion of the actual implementation of each module is unnecessary to an understanding of the present application. Rather, the actual implementation is within the routine skill of engineers familiar with the properties, functions and internal relationships of the various functional modules in the devices disclosed herein. Accordingly, the skilled artisan is capable of implementing the present application as set forth in the claims using ordinary skill in the art without need for further experimentation. It is also to be understood that the particular concepts disclosed are illustrative only and not intended to limit the scope of the present application, which is defined by the full scope of the appended claims and equivalents thereof.

[0154] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0155] The logic and / or steps represented in the flow diagrams or otherwise described herein, for example, can be embodied in non-transitory computer- readable media, executed by an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or in conjunction with which the instructions can be executed. In the context of this specification, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium.

[0156] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via an optical scanner, then compiled, interpreted, or otherwise processed, and stored in a computer memory in a form that is then employable by a computer. In some embodiments, the computer-readable medium can be a computer program product that can be implemented by way of, for example, software, hardware, firmware, or any combination of these elements.

[0157] It should be understood that aspects of the present application can be implemented in hardware, software, firmware, or combinations thereof. In the above-described embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, or combinations thereof, can be used with the necessary design changes to support the specific logic functions of the present application: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and so forth.

[0158] In the above-described description of the present specification, the description referring to the terms "one embodiment / implementation", "another embodiment / implementation", or "some embodiments / implementations" and the like means that a specific feature, structure, material or characteristic described in connection with the embodiments or examples is included in at least one embodiment or example of the present application. The illustrative appearances of the above-described terms in the description are not necessarily referred to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0159] While the embodiments of the present application have been shown and described, it is to be understood that the embodiments can be varied, modified, substituted and changed by those skilled in the art without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

[0160] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and these equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for training a target detection model, the method comprising: The method comprises the following steps: obtaining a sample image of a target sample; performing multi-dimensional feature extraction on the sample image to obtain a sample feature set, the sample feature set comprising a plurality of sample feature maps obtained based on the order of the multi-dimensional feature extraction, the feature dimension of a subsequent sample feature map being smaller than that of a previous sample feature map; performing feature pyramid upsampling and adaptive fusion on the sample feature set to obtain a plurality of fused feature maps, each fused feature map being used to represent feature fusion information of a first feature map and a second feature map after upsampling, the first feature map being a sample feature map with the same feature dimension as the fused feature map, and the second feature map being a subsequent sample feature map of the first feature map; updating the parameters of an initialized target detection model according to all the fused feature maps to obtain a trained target detection model; performing feature pyramid upsampling and adaptive fusion on the sample feature set to obtain a fused feature map, comprising: performing feature upsampling on the second feature map to obtain a third feature map; performing adaptive feature fusion processing on the first feature map and the third feature map to obtain the fused feature map; the adaptive feature fusion processing on the first feature map and the third feature map to obtain the fused feature map, comprising: performing dynamic weight calculation on the first feature map and the third feature map to obtain a first dynamic weight of the first feature map and a second dynamic weight of the third feature map; performing feature weighted fusion on the second dynamic weight and the third feature map according to the first dynamic weight and the first feature map to obtain a fourth feature map; performing feature enhancement on the fourth feature map to obtain the fused feature map; the dynamic weight calculation on the first feature map and the third feature map to obtain the first dynamic weight of the first feature map and the second dynamic weight of the third feature map, comprising: performing similarity measurement calculation on the first feature map and the third feature map to obtain a feature similarity; performing feature complementarity evaluation on the first feature map according to the feature similarity to obtain a first complementarity value of the first feature map, the first complementarity value being negatively correlated with the feature similarity; the feature complementarity evaluation is used to evaluate the feature complementarity between the first feature map and the second feature map; performing feature complementarity evaluation on the third feature map according to the feature similarity to obtain a second complementarity value of the third feature map, the second complementarity value being negatively correlated with the feature similarity; obtaining the first dynamic weight of the first feature map and the second dynamic weight of the third feature map according to the first complementarity value and the second complementarity value; the first dynamic weight is equal to the ratio of the first complementarity value to the sum of the first complementarity value and the second complementarity value; the second dynamic weight is equal to the ratio of the second complementarity value to the sum of the first complementarity value and the second complementarity value.

2. The method of claim 1, wherein, the multi-dimensional feature extraction on the sample image to obtain the sample feature set, comprising: performing feature dimension increasing processing on the sample image to obtain a dimension-increased feature map; The feature extraction unit is cascaded with several dimension reduction feature extraction units to perform feature extraction on the high-dimensional feature map, and sample feature maps output by each dimension reduction feature extraction unit are obtained.

3. The method of claim 1, wherein, The method further comprises: performing global normalization on the fusion feature map to obtain a normalized feature map; performing feature enhancement processing on the normalized feature map to obtain an enhanced feature map; performing attention extraction on the enhanced feature map to obtain a feature attention map; performing attention weighted fusion update on the fusion feature map according to the feature attention map to obtain a fusion feature map after fusion update.

4. The method of claim 3, wherein, The feature enhancement processing on the normalized feature map to obtain an enhanced feature map comprises: performing region segmentation on the normalized feature map to obtain a plurality of first feature regions; performing region statistics on all the first feature regions to obtain a region mean and a region standard deviation of each first feature region; performing conditional feature enhancement on the corresponding first feature region according to the region mean and the region standard deviation to obtain a second feature region; obtaining the enhanced feature map according to all the second feature regions.

5. A detection method of a target detection model, characterized by, It comprises: obtaining a target image of a target to be detected; inputting the target image into the trained target detection model of any one of claims 1-4 for detection and recognition to obtain a target detection result of the target to be detected. 6.A training system of an object detection model, characterized in that, It comprises: a first processing unit configured to obtain a sample image of a target sample; a second processing unit configured to perform multi-dimensional feature extraction on the sample image to obtain a sample feature set, the sample feature set comprising a plurality of sample feature maps obtained in an extraction order based on the multi-dimensional feature extraction, a feature dimension of a subsequent sample feature map being smaller than a feature dimension of a previous sample feature map; a third processing unit configured to perform feature pyramid upsampling and adaptive fusion on the sample feature set to obtain a plurality of fusion feature maps, each fusion feature map being used to represent feature fusion information of a first feature map and a second feature map after upsampling, the first feature map being a sample feature map having the same feature dimension as the fusion feature map, and the second feature map being a subsequent sample feature map of the first feature map; performing feature pyramid upsampling and adaptive fusion on the sample feature set to obtain a fusion feature map comprises: performing feature upsampling on the second feature map to obtain a third feature map; performing adaptive feature fusion processing on the first feature map and the third feature map to obtain the fusion feature map; the adaptive feature fusion processing on the first feature map and the third feature map to obtain the fusion feature map comprises: performing dynamic weight calculation according to the first feature map and the third feature map to obtain a first dynamic weight of the first feature map and a second dynamic weight of the third feature map; performing feature weighted fusion on the second dynamic weight and the third feature map according to the first dynamic weight and the first feature map to obtain a fourth feature map; performing feature enhancement on the fourth feature map to obtain the fusion feature map; The dynamic weight calculation according to the first feature map and the third feature map comprises: performing similarity measurement calculation on the first feature map and the third feature map to obtain a feature similarity; performing feature complementarity evaluation on the first feature map according to the feature similarity to obtain a first complementarity value of the first feature map, the first complementarity value being negatively correlated with the feature similarity; the feature complementarity evaluation is used to evaluate the feature complementarity between the first feature map and the second feature map; performing feature complementarity evaluation on the third feature map according to the feature similarity to obtain a second complementarity value of the third feature map, the second complementarity value being negatively correlated with the feature similarity; obtaining the first dynamic weight of the first feature map and the second dynamic weight of the third feature map according to the first complementarity value and the second complementarity value; the first dynamic weight is equal to the ratio of the first complementarity value to the sum of the first complementarity value and the second complementarity value; the second dynamic weight is equal to the ratio of the second complementarity value to the sum of the first complementarity value and the second complementarity value; a fourth processing unit, configured to perform parameter update on the initialized target detection model according to all the fusion feature maps to obtain a trained target detection model.

7. An electronic device, comprising: comprise: at least one processor; at least one memory for storing at least one program; when the at least one program is executed by the at least one processor, the at least one processor implements the method in any one of claims 1-4.

Images