Target detection method, device, computer equipment and storage medium

Abstract

The present application relates to a target detection method, device, computer equipment and storage medium. The method includes: constructing a deep neural network model based on at least two multi-scale feature fusions; the at least two multi-scale feature fusions are performed in parallel; the multi-scale feature fusion refers to fusing low-level feature maps with high-level feature maps according to a preset feature fusion method; training the constructed deep neural network model based on a labeled training data set to obtain a trained target detection model; when a target image to be detected is obtained, the target image is detected by the target detection model to obtain a detection result; and the target in the target image is determined according to the detection result. The use of this method can improve the detection accuracy of target detection.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a target detection method, device, computer equipment and storage medium. Background Art

[0002] With the development of computer technology, computer vision-based object detection technology has gradually developed and is widely used in various scenarios, such as work badge detection, face detection, vehicle detection, and vehicle damage detection, etc., which has brought a lot of convenience to people's lives. Therefore, how to perform object detection is a problem worthy of attention.

[0003] At present, target detection methods such as YOLO and SSD are usually used to detect targets in target images. However, the existing target detection methods have low detection accuracy for small targets, resulting in the problem of low detection accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a target detection method, device, computer equipment and storage medium that can improve detection accuracy in response to the above technical problems.

[0005] A target detection method, the method comprising:

[0006] Constructing a deep neural network model based on at least two multi-scale feature fusions; the at least two multi-scale feature fusions are performed in parallel; the multi-scale feature fusion refers to fusing a low-level feature map with a high-level feature map according to a preset feature fusion method;

[0007] The constructed deep neural network model is trained based on the labeled training data set to obtain a trained target detection model;

[0008] When a target image to be detected is obtained, the target image is detected by the target detection model to obtain a detection result;

[0009] The target in the target image is determined according to the detection result.

[0010] In one embodiment, the step of multi-scale feature fusion includes:

[0011] Convolving the low-level feature map to obtain a first feature map with a preset number of channels;

[0012] Deconvolving the high-level feature map to obtain a second feature map having the same scale as the low-level feature map;

[0013] Perform channel superposition on the first feature map and the second feature map to obtain a third feature map;

[0014] Convolution is performed on the third feature map to obtain a fused feature map after feature fusion.

[0015] In one embodiment, the step of training the constructed deep neural network model based on the labeled training data set to obtain a trained target detection model includes:

[0016] Get the labeled training data set;

[0017] Selecting target training data from a training data set that has not been used for model training, and predicting the target training data using the deep neural network model to be trained to obtain a prediction result;

[0018] Determine a comprehensive loss value according to the prediction result and the labeling result in the target training data and a preset positioning loss function and a category loss function;

[0019] Reversely adjust the parameters of the deep neural network model according to the comprehensive loss value, and return to the step of selecting target training data from the training data set that has not been used for model training to continue executing until the iteration stop condition is met to obtain a trained target detection model.

[0020] In one embodiment, detecting the target image by using the target detection model to obtain a detection result includes:

[0021] Detecting the target image using the target detection model to obtain target feature maps of multiple target scales;

[0022] Through the target detection model, each target feature map is convolved with the corresponding convolution kernel to obtain a detection frame corresponding to each target feature map; the scale of the detection frame is dynamically determined based on the corresponding target feature map;

[0023] Through the target detection model, the detection result corresponding to the target image is determined according to the detection box corresponding to each target feature map.

[0024] In one of the embodiments, the detection result includes at least one target detection frame corresponding to the target image; and determining the target in the target image according to the detection result includes:

[0025] Extracting a sub-image corresponding to each target detection frame from the target image;

[0026] Each sub-image is identified through a pre-configured classifier to obtain the category of the corresponding target detection box;

[0027] The target in the target image is determined according to the category of the target detection frame.

[0028] In one embodiment, before detecting the target image by the target detection model to obtain a detection result, the method further includes:

[0029] The scale of the target image is adjusted to a preset scale; the preset scale is larger than a standard scale corresponding to the deep neural network model.

[0030] A target detection device, comprising:

[0031] A construction module is used to construct a deep neural network model based on at least two multi-scale feature fusions; the at least two multi-scale feature fusions are performed in parallel; the multi-scale feature fusion refers to fusing a low-level feature map with a high-level feature map according to a preset feature fusion method;

[0032] A training module is used to train the constructed deep neural network model based on the labeled training data set to obtain a trained target detection model;

[0033] A detection module is used to detect the target image to be detected by using the target detection model to obtain a detection result;

[0034] A determination module is used to determine the target in the target image according to the detection result.

[0035] In one embodiment, the construction module is also used to convolve the low-level feature map to obtain a first feature map with a preset number of channels; deconvolve the high-level feature map to obtain a second feature map with the same scale as the low-level feature map; superimpose the channels of the first feature map and the second feature map to obtain a third feature map; and convolve the third feature map to obtain a fused feature map after feature fusion.

[0036] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the target detection method described in each of the above embodiments when executing the computer program.

[0037] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the target detection method described in each of the above embodiments.

[0038] The above-mentioned target detection method, device, computer equipment and storage medium pre-construct a deep neural network model based on at least two multi-scale feature fusions, the at least two multi-scale feature fusions are performed in parallel, and each multi-scale feature fusion is to fuse the low-level feature map with the high-level feature map according to a preset feature fusion method. The constructed deep neural network model is trained based on the training data set to obtain a trained target detection model. In this way, when the target image is detected by the target detection model, more than one low-level feature map can be fused with the corresponding high-level feature map respectively, and the detection result corresponding to the target image is determined based on the obtained fused feature map, and then the target in the target image is determined according to the detection result. The low-level features have a higher resolution, and the high-level features have richer semantic information. Therefore, the low-level feature map and the high-level feature map are fused to improve the detection accuracy of the target detection. And the parallel execution of at least two multi-scale feature fusion operations can improve the detection accuracy while improving the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 An application scenario diagram of a target detection method in an embodiment;

[0040] Figure 2 Schematic diagram of a target detection method in one embodiment;

[0041] Figure 3 is a flow chart of a target detection method in another embodiment;

[0042] Figure 4 is a schematic diagram of the structure of a deep neural network model based on multi-scale feature fusion constructed in one embodiment;

[0043] Figure 5 A schematic diagram of the principle of multi-scale feature fusion in one embodiment

[0044] Figure 6 is a structural block diagram of a target detection device in one embodiment;

[0045] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] The target detection method provided in this application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The server 104 constructs a deep neural network of multi-scale feature fusion at least twice, and the at least two multi-scale feature fusions are performed in parallel, and each multi-scale feature fusion is to fuse the low-level feature map with the high-level feature map according to a preset feature fusion method. The server 104 trains the constructed deep neural network model based on the labeled training data set to obtain a trained target detection model, and when the target image to be detected is obtained, the target image is detected by the target detection model to obtain the detection result, and then the target in the target image is determined according to the detection result. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0048] In one embodiment, Figure 2 As shown, a target detection method is provided, which is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0049] S202, constructing a deep neural network model based on at least two multi-scale feature fusions; at least two multi-scale feature fusions are performed in parallel; multi-scale feature fusion refers to fusing low-level feature maps with high-level feature maps according to a preset feature fusion method.

[0050] Among them, scale can be understood as dimension, and multi-scale refers to different dimensions, such as 32*32 and 1*1 are different scales or dimensions. Multi-scale feature fusion refers to the fusion of features in feature maps of different scales or dimensions. The preset feature fusion method refers to a pre-set method of fusing low-level feature maps with high-level feature maps, such as adjusting the low-level feature map and the high-level feature map to feature maps of the same scale, and performing feature fusion on the feature maps after adjusting the scale. It can be understood that low-level feature maps and high-level feature maps are relative concepts. Taking the high-level feature map as the benchmark, feature maps with a lower level than the benchmark are all low-level feature maps corresponding to the high-level feature map.

[0051] Specifically, the server constructs a deep neural network model based on at least two multi-scale feature fusions. In the constructed deep neural network model, the at least two multi-scale feature fusions are executed in parallel. Each multi-scale feature fusion refers to fusing a preset low-level feature map with a corresponding high-level feature map in accordance with a preset feature fusion method. It is understandable that the feature maps involved in any two multi-scale feature fusions are not completely consistent.

[0052] In one embodiment, the closer the corresponding levels of the low-level feature map and the high-level feature map used for multi-scale feature fusion are, the higher the redundancy between the low-level feature map and the high-level feature map; accordingly, the farther the corresponding levels of the low-level feature map and the high-level feature map are, the lower the redundancy between the low-level feature map and the high-level feature map. Therefore, the low-level feature map and the high-level feature map used for multi-scale feature fusion can be selected according to actual needs, and no specific limitation is made here. For example, in the entire deep neural network model, a feature map with a relatively high level can be selected as a high-level feature map, and a feature map with a relatively low level can be selected as a low-level feature map. For example, for at least two multi-scale feature fusions, if the high-level feature maps in each multi-scale feature fusion are the same, such as taking the feature map of the highest level as the high-level feature map, the low-level feature map in each multi-scale feature fusion can be selected in a manner that the level difference between any two adjacent low-level feature maps is equal. It can be understood that the higher the number of times the multi-scale feature fusion is performed, the more computer resources and time it takes. As the number of times the multi-scale feature fusion is performed increases, the detection effect tends to be stable, that is, the detection effect will not increase significantly with the increase in the number of times the multi-scale feature fusion is performed. Therefore, the number of times the multi-scale feature fusion is performed will not be too large. In one embodiment, the high-level features involved in the at least two multi-scale feature fusions are Figure 1 The low-level feature maps involved are different.

[0053] In one embodiment, the step of multi-scale feature fusion includes: convolving a low-level feature map to obtain a first feature map with a preset number of channels; deconvolving a high-level feature map to obtain a second feature map with the same scale as the low-level feature map; superimposing the first feature map and the second feature map through channels to obtain a third feature map; and convolving the third feature map to obtain a fused feature map after feature fusion.

[0054] The preset number of channels refers to a preset number of channels, which is determined by a convolution kernel convolved with a low-level feature map. For example, if the convolution kernel is 1*1*128, the preset number of channels determined by the convolution kernel is 128. The preset number of channels may be consistent with the number of channels corresponding to a high-level feature map participating in multi-scale feature fusion.

[0055] Specifically, for each multi-scale feature fusion, the server obtains the low-level feature map and the high-level feature map involved in the multi-scale feature fusion, and determines a first convolution kernel for convolving the low-level feature map, and a second convolution kernel for deconvolving the high-level feature map. The server convolves the low-level feature map with the first convolution kernel to obtain a first feature map with a preset number of channels, and deconvolves the high-level feature map according to the corresponding second convolution kernel to obtain a second feature map with the same scale as the low-level feature map. The server performs a superposition operation on the first feature map and the second feature map on the channel to obtain a third feature map. The server determines a third convolution kernel for feature fusion of the first feature map and the second feature map, and convolves the third convolution kernel with the third feature map to obtain a fused feature map after feature fusion.

[0056] In one embodiment, the server determines more than one set of feature maps to be fused, each set of feature maps to be fused includes a low-level feature map and a high-level feature map. The server performs feature fusion on each set of feature maps to be fused according to the multi-scale feature fusion steps to obtain more than one fused feature map.

[0057] In one embodiment, a feature fusion layer is added to the deep neural network model through multi-scale feature fusion, and the fused feature map obtained by multi-scale feature fusion is used as the feature map corresponding to the feature fusion layer.

[0058] S204: Train the constructed deep neural network model based on the labeled training data set to obtain a trained target detection model.

[0059] The training data set is a sample set used to train the constructed deep neural network model. The training data set includes pictures as input features and the annotation results obtained by annotating the objects in each picture.

[0060] Specifically, the server obtains a labeled training data set, uses the images in the training data set as input features of the constructed deep neural network model, uses the corresponding annotation results in the training data set as expected output features, trains the constructed deep neural network model, and obtains a trained target detection model, so as to detect targets in target images based on the target detection model.

[0061] In one embodiment, the server inputs each image in the training data set as an input feature into the deep neural network model for prediction, obtains a prediction result, and compares the prediction result with the corresponding annotation result in the training data set to dynamically adjust the parameters of the deep neural network model according to the comparison result until the iteration stop condition is met, stops the iterative training process, and uses the deep neural network model obtained through iterative training as the trained target detection model.

[0062] In one embodiment, taking the target as a work badge and / or a work badge rope as an example, the labeled training data set includes pictures and labeling results in which the work badge and / or the work badge rope are labeled in each picture.

[0063] In one embodiment, step S204 includes: obtaining a labeled training data set; selecting target training data from a training data set that has not been used for model training, and predicting the target training data through the deep neural network model to be trained to obtain a prediction result; determining a comprehensive loss value according to the prediction result and the labeling result in the target training data, according to a preset positioning loss function and a category loss function; reversely adjusting the parameters of the deep neural network model according to the comprehensive loss value, and returning to the step of selecting target training data from a training data set that has not been used for model training to continue executing until the iteration stop condition is met to obtain a trained target detection model.

[0064] Specifically, the server obtains a labeled training data set, and trains the constructed deep neural network model based on the obtained training data set to obtain a trained target detection model. The specific training process is as follows: the server selects target training data from the training data set that has not been used for model training, and inputs the selected target training data as input features into the deep neural network model to be trained for prediction, and obtains corresponding prediction results; the server substitutes the prediction results and the corresponding annotation results in the training data set into the preset positioning loss function to obtain the positioning loss value, substitutes the prediction results and the annotation results into the preset category loss function to obtain the category loss value, and obtains the corresponding comprehensive loss value according to the positioning loss value and the category loss value; the server reversely adjusts the parameters of the deep neural network model according to the obtained comprehensive loss value, and returns to the step of selecting target training data from the training data set that has not been used for model training to continue execution until the iteration stop condition is met, stops the iterative training process, and obtains a trained target detection model.

[0065] The iteration stop condition is the judgment basis or condition for stopping the iterative training process, for example, every training data in the training data set is used for model training, or the number of iterations is greater than or equal to the iteration threshold, or the comprehensive loss value is less than or equal to the preset loss value threshold, etc.

[0066] In one embodiment, when the model training is initially performed, the constructed deep neural network model is used as the deep neural network model to be trained, and the target training data is directly selected from the acquired training data set. In the subsequent model training process, the deep neural network model after dynamically adjusting the parameters is used as the deep neural network model to be trained. In the iterative training process of the deep neural network model, the training data that has not been used for model training is iteratively selected from the training data set as the target training data, and the deep neural network model to be trained is trained according to the above training method based on the selected target training data.

[0067] In one embodiment, the comprehensive loss function is determined by the preset positioning loss function and the category loss function. The comprehensive loss function can be specifically as follows:

[0068]

[0069] The positioning loss function can be specifically shown as follows:

[0070]

[0071]

[0072]

[0073] The category loss function can be specifically described as follows:

[0074] in,

[0075] Among them, N represents the number of prediction boxes. If N is 0, the comprehensive loss value is 0, and L conf (x,c) represents the category loss, L loc (x,l,g) represents the positioning loss, It is an indicator for matching the i-th prediction box to the j-th real box of category p, which is used to indicate whether the i-th prediction box matches the j-th real box of category p. 0 indicates no match and 1 indicates match. When the i-th prediction box matches the j-th real box of category p, the category of the i-th prediction box is determined to be p. c indicates the category confidence of the prediction box, l indicates the prediction box, and g indicates the real box. Pos indicates the set of prediction boxes that match the real box. Represents the parameter value of the i-th prediction box, where the value of m is (cx, cy), w, and h represent the center coordinates of the prediction box, and the width and height of the prediction box, respectively. Represents the parameter value of the i-th true box, where the value of m is (cx, cy), w, and h represent the center coordinates of the true box, and the width and height of the true box, respectively. Represents the parameter value of the i-th default box. Indicates the confidence value that the i-th prediction box belongs to category p.

[0076] In one embodiment, the server uses a stochastic gradient method to iteratively train the constructed deep neural network model, wherein the learning rate is preset to 0.0001 and the number of iterations is preset to 100,000 times.

[0077] S206, when the target image to be detected is obtained, the target image is detected by the target detection model to obtain a detection result.

[0078] Specifically, the server obtains a target image to be detected. When the target image to be detected is obtained, the obtained target image is input into a trained target detection model, and the target image is detected by the target detection model to obtain a corresponding detection result.

[0079] In one embodiment, a target image is detected by a target detection model to obtain a detection result, including: detecting the target image by a target detection model to obtain target feature maps of multiple target scales; convolving each target feature map with a corresponding convolution kernel by the target detection model to obtain a detection frame corresponding to each target feature map; the scale of the detection frame is dynamically determined based on the corresponding target feature map; and determining the detection result corresponding to the target image according to the detection frame corresponding to each target feature map by the target detection model.

[0080] Specifically, the server performs target detection on the target image to be detected through the trained target detection model, which may specifically include: extracting features of the target image in turn through each convolution layer in the target detection model to obtain feature maps corresponding to each convolution layer, selecting feature maps corresponding to more than one convolution layer, using the more than one feature maps as low-level feature maps, and selecting feature maps corresponding to convolution layers with a level higher than the selected convolution layer as high-level feature maps, performing multi-scale feature fusion on each low-level feature map with the corresponding high-level feature map to obtain a corresponding fused feature map, thereby obtaining more than one fused feature map of the target scale, and determining the more than one fused feature map of the target scale as the target feature map; determining the scale of the convolution kernel and the detection frame corresponding to each target feature map through the target detection model, and convolving each target feature map with the corresponding convolution kernel to obtain a detection frame corresponding to each target feature map, comparing the detection frames corresponding to each target feature map, and determining the detection result corresponding to the target image according to the comparison result.

[0081] In one embodiment, the server uses the target detection model to use the low-level feature maps and high-level feature maps selected in the above manner, as well as the fused feature map obtained by at least two multi-scale feature fusions as the target feature map.

[0082] In one embodiment, during the process of detecting the target image by the target detection model, the convolution layers corresponding to the low-level feature maps and the high-level feature maps used for at least two multi-scale feature fusions, and the target feature layers corresponding to each target feature map, are set when the deep neural network model is constructed. The scale of the detection box corresponding to each target feature map is also optimized during the training process of the deep neural network model.

[0083] In one embodiment, during the training of a deep neural network model, the server predetermines multiple target feature layers based on the constructed deep neural network model, and the scale of the feature map corresponding to each target feature layer, and determines the scale of the default box corresponding to the feature map corresponding to each target feature layer according to the multiple target feature layers, and then continuously optimizes the scale of the default box corresponding to each target feature layer through iterative training, and determines the scale of the default box optimized when the iteration stops as the scale of the detection box corresponding to the target feature map corresponding to the corresponding target feature layer. It can be understood that the scale of the default box corresponding to the feature map of the same scale is the same. Therefore, for feature maps of the same scale, the scale of the default box can be dynamically calculated once, without calculating it separately for each feature map. The server can specifically determine the scale of the default box corresponding to each feature map according to the following mapping relationship.

[0084]

[0085]

[0086]

[0087] Where m represents the number of feature maps with different scales, k represents the kth feature map among the m feature maps, Smin and Smax represent the scale of the default box at the lowest level and the scale of the default box at the highest level, respectively, and a r Indicates the default frame width-to-height ratio. For example, m is 6, Smin is 0.1, and Smax is 0.5.

[0088] In one embodiment, the scale of the detection frame corresponding to the target feature map is dynamically adjusted according to the target feature layer where the target feature map is located, so that the detection frame can cover a sufficiently small target, thereby improving the accuracy of target detection.

[0089] In one embodiment, the convolution kernel used to convolve the target feature map can be expressed as 3*3*(number of detection frames*(number of categories+4)), where the number of detection frames refers to the number of detection frames corresponding to each target feature map, and the number of categories refers to the number of possible categories of each detection frame. Among them, 3*3 represents the scale of the convolution kernel, (number of detection frames*(number of categories+4)) represents the number of channels of the convolution kernel, the number of detection frames refers to the number of detection frames required to be predicted for each pixel in the target feature map, and each detection frame corresponds to four coordinate points and a corresponding number of categories. A detection frame can be determined based on the four coordinate points and the corresponding number of categories. Therefore, (number of detection frames*(number of categories+4)) can be understood as the total number of values ​​required to be calculated for each pixel in the target feature map. The detection frames corresponding to each pixel in the target feature map are merged and deduplicated to obtain the number of detection frames corresponding to the target feature map. It is worth noting that the symbol "*" involved in this application represents a multiplication operation.

[0090] When the number of categories in the convolution kernel is 0, the convolution kernel is convolved with the corresponding target feature map to obtain the detection box corresponding to the target feature map and the position confidence corresponding to each detection box. When the number of categories in the convolution kernel is greater than 0, the convolution kernel is convolved with the corresponding target feature map to obtain the detection box corresponding to the target feature map and the position confidence corresponding to each detection box, as well as the category of each detection box and the corresponding category confidence.

[0091] In one embodiment, the server determines the detection frame corresponding to the target image from the detection frame corresponding to each target feature map, and obtains multiple detection frames corresponding to the target image. The server selects at least one target detection frame from the multiple detection frames according to the position confidence corresponding to each detection frame and the non-maximum suppression method, and obtains the detection result corresponding to the target image according to the selected target detection frame. It can be understood that the detection result includes the target detection frame, but does not include the category of the target detection frame.

[0092] The step of selecting the target detection frame by non-maximum suppression may specifically include: the server selects the candidate detection frame with the largest position confidence from multiple detection frames corresponding to the target image, removes the detection frame whose overlap with the currently selected candidate detection frame is greater than or equal to the overlap threshold from the multiple detection frames, selects the candidate detection frame with the largest position confidence from the remaining detection frames that have not been selected, and continues to perform the above-mentioned related steps of removing the detection frame for the currently selected candidate detection frame until all candidate detection frames are selected, determines the selected candidate detection frame as the target detection frame corresponding to the target image, and determines the detection results including each target detection frame. The overlap threshold is, for example, 0.8.

[0093] In one embodiment, the server determines the candidate detection frame whose position confidence is greater than or equal to the confidence threshold among the selected candidate detection frames as the target detection frame corresponding to the target image, and determines the corresponding detection result based on the target detection frame. It can be understood that the detection result includes the center coordinates, height, width and other parameter values ​​corresponding to each target detection frame, and the corresponding target detection frame can be determined in the target image based on the parameter values. The confidence threshold is, for example, 0.6.

[0094] In one embodiment, after the server determines at least one candidate detection frame from multiple detection frames corresponding to the target image through the target detection model in the above manner, it selects a target detection frame from the determined candidate detection frames according to the category of each candidate detection frame and the corresponding category confidence, and obtains the corresponding detection result according to the selected target detection frame. Specifically, the server can select a target detection frame whose category is consistent with the target to be detected from the candidate detection frame through the target detection model, and obtain a detection result including the target detection frame. It can be understood that the server can also select a target detection frame whose category is consistent with the target to be detected and whose category confidence is greater than or equal to the confidence threshold from the candidate detection frame.

[0095] In one embodiment, the detection result includes at least one target detection frame corresponding to the target image; step S208 includes: extracting a sub-image corresponding to each target detection frame from the target image; identifying each sub-image through a preconfigured classifier to obtain a category of the corresponding target detection frame; and determining the target in the target image according to the category of the target detection frame.

[0096] The category of the target detection frame refers to the category of the target in the target image that is within the target detection frame, such as a work badge or a work badge rope.

[0097] Specifically, the server determines at least one target detection frame corresponding to the target image according to the detection result, and compares each target detection frame with the target image to extract the sub-image corresponding to the target detection frame from the target image according to the comparison result. The server inputs each extracted sub-image into a preconfigured classifier, and uses the classifier to identify the category of the corresponding target detection frame according to the sub-image, and determines whether there is a target to be detected in the target image according to the category of each target detection frame. If so, the target to be detected is marked in the target image. In one embodiment, the classifier is such as Mobilenet, which is not specifically limited here.

[0098] In the above embodiment, the target in the target image is detected by the target detection model, and the detected target is classified by the classifier, so as to avoid filtering out the targets with low category confidence when detecting and classifying the targets in the target image by the target detection model, thereby improving the accuracy of target detection.

[0099] S208, determining the target in the target image according to the detection result.

[0100] Specifically, the server determines whether the target image includes the target to be detected according to the detection result. In one embodiment, when it is determined that the target image contains the target to be detected, the server marks the target in the target image according to the detection result. In one embodiment, the server feeds back prompt information corresponding to the target image to the terminal according to the detection result.

[0101] For example, taking the target to be detected as an ID card and / or an ID card rope, when it is determined based on the detection results that an ID card and / or an ID card rope exists in the target image, the server feeds back a prompt message indicating that the ID card is present to the terminal; otherwise, the server feeds back a prompt message indicating that the ID card is not present.

[0102] In one embodiment, when the detection result includes a target detection frame and a corresponding category, the server determines whether there is a target to be detected in the target image according to the category. When the detection result includes a target detection frame but does not include a category corresponding to the target detection frame, the server performs category recognition on the target detection frame through a preconfigured classifier to determine whether there is a target to be detected in the target image according to the recognition result.

[0103] The above-mentioned target detection method pre-constructs a deep neural network model based on at least two multi-scale feature fusions, and the at least two multi-scale feature fusions are performed in parallel, and each multi-scale feature fusion is to fuse the low-level feature map with the high-level feature map according to a preset feature fusion method. The constructed deep neural network model is trained based on the training data set to obtain a trained target detection model. In this way, when the target image is detected by the target detection model, more than one low-level feature map can be fused with the corresponding high-level feature map respectively, and the detection result corresponding to the target image is determined based on the obtained fused feature map, and then the target in the target image is determined according to the detection result. The low-level features have a higher resolution, and the high-level features have richer semantic information. Therefore, the low-level feature map and the high-level feature map are fused to improve the detection accuracy of the target detection. And the parallel execution of at least two multi-scale feature fusion operations can improve the detection accuracy while improving the detection efficiency.

[0104] In one embodiment, before detecting the target image through the target detection model to obtain the detection result, the above-mentioned target detection method also includes: adjusting the scale of the target image to a preset scale; the preset scale is larger than the standard scale corresponding to the deep neural network model.

[0105] The preset scale is a pre-set scale, such as 512*512. The standard scale refers to the scale of the target image input to the deep neural network model when performing target detection based on the existing deep neural network model, such as 300*300.

[0106] Specifically, the server preconfigures a preset scale locally, and the preconfigured preset scale is larger than the standard scale of the deep neural network model. After the server obtains the target image to be detected, it determines the scale of the target image and compares the determined scale with the preset scale. If the scale of the target image is larger than the preset scale, the scale of the target image is adjusted to the preset scale.

[0107] In one embodiment, the standard scale corresponding to the deep neural network model can be understood as the input image scale of the existing SSD (Single Shot Multibox Detector).

[0108] In the above embodiment, the scale of the target image input to the target detection model is adjusted to a preset scale that is larger than the standard scale, so that the resolution of shallow features / low-level features can be improved, thereby improving the detection effect of small targets.

[0109] like Figure 3 As shown, a target detection method is provided, which specifically includes the following steps:

[0110] S302, constructing a deep neural network model based on at least two multi-scale feature fusions; at least two multi-scale feature fusions are performed in parallel; multi-scale feature fusion refers to fusing low-level feature maps with high-level feature maps according to a preset feature fusion method.

[0111] S304: Train the constructed deep neural network model based on the labeled training data set to obtain a trained target detection model.

[0112] S306, when the target image to be detected is obtained, the target image is detected by the target detection model to obtain target feature maps of multiple target scales.

[0113] S308, through the target detection model, convolve each target feature map with the corresponding convolution kernel to obtain a detection frame corresponding to each target feature map; the scale of the detection frame is dynamically determined based on the corresponding target feature map.

[0114] S310, determining a detection result corresponding to the target image according to the detection frame corresponding to each target feature map through the target detection model; the detection result includes at least one target detection frame corresponding to the target image.

[0115] S312, extracting a sub-image corresponding to each target detection frame from the target image.

[0116] S314: Identify each sub-image through a pre-configured classifier to obtain the category of the corresponding target detection box.

[0117] S316, determining the target in the target image according to the category of the target detection frame.

[0118] Figure 4 FIG. 1 is a schematic diagram of a structure of a deep neural network model based on multi-scale feature fusion constructed in an embodiment. The deep neural network model includes multiple convolutional layers and at least one fusion feature layer obtained by fusion of multi-scale features. Figure 4 As shown, the constructed deep neural network model includes feature maps Conv0 corresponding to convolution layer 0, ..., feature maps Conv11 corresponding to convolution layer 11, ..., feature maps Conv16_1 corresponding to convolution layer 16_1, feature maps Conv16_2 corresponding to convolution layer 16_2, feature maps Conv17_1 corresponding to convolution layer 17_1, and feature maps Conv17_2 corresponding to convolution layer 17_2. Among them, the feature maps Conv0, Conv11, Conv16_1, Conv16_2, Conv17_1 and Conv17_2 have corresponding scales of 256*256*32, 32*32*512, 4*4*128, 2*2*256, 2*2*64 and 1*1*128 respectively.

[0119] like Figure 4 As shown, the feature map Conv11 corresponding to the convolution layer 11 and the feature map Conv17_2 corresponding to the convolution layer 17_2 are subjected to multi-scale feature fusion to obtain the fused feature map Conv18_1 corresponding to the fused feature layer 18_1, and the scale corresponding to the fused feature map Conv18_1 is 16*16*128. The feature map Conv16_2 corresponding to the convolution layer 16_2 and the feature map Conv17_2 corresponding to the convolution layer 17_2 are subjected to multi-scale feature fusion to obtain the fused feature map Conv18_2 corresponding to the fused feature layer 18_2, and the scale corresponding to the fused feature map Conv18_2 is 1*1*128. The fused feature map obtained by multi-scale feature fusion is used as the target feature map corresponding to the target image in the target detection process. Figure 4 As shown, the feature map Conv11 corresponding to the convolution layer 11, the feature map Conv16_2 corresponding to the convolution layer 16_2, and the feature map Conv17_2 corresponding to the convolution layer 17_2 can also be determined as the target feature maps corresponding to the target image.

[0120] Furthermore, each target feature map is convolved with the corresponding convolution kernel to obtain the corresponding detection frame. Figure 4As shown, the fused feature map Conv18_1 is convolved with the convolution kernel 3*3*(4*(3+4)), the fused feature map Conv18_2 is convolved with the convolution kernel 3*3*(6*(3+4)), the feature map Conv11 is convolved with the convolution kernel 3*3*(4*(3+4)), the feature map Conv16_2 is convolved with the convolution kernel 3*3*(6*(3+4)), and the feature map Conv17_2 is convolved with the convolution kernel 3*3*(6*(3+4)), to obtain their respective corresponding detection frames. The detection frames corresponding to each target feature map are passed to the target detection layer, the detection frames corresponding to each target feature map are integrated in the target detection layer, and the integrated multiple detection frames are passed to the non-maximum suppression layer.

[0121] In the target detection process, the target image is input into the target detection model obtained by training the above-mentioned deep neural network model for detection. The structure of the target detection model is the same as Figure 4 The structure of the deep neural network model shown in FIG. 1 is consistent with that of the deep neural network model shown in FIG. 1 . When the target image is detected by the target detection model, the fused feature map obtained by multi-scale feature fusion is determined as the target feature map. Based on the more than one target feature map, multiple detection frames corresponding to the target image can be obtained, and the corresponding detection result is determined according to the multiple detection frames corresponding to the target image in the non-maximum suppression layer. Figure 4 As shown, the scale of the target image can be specifically 512*512*3. It can be understood that the feature layer corresponding to the target feature map is determined as the target feature layer, then Figure 4 The target feature layers shown are for example only and are not intended to be limiting.

[0122] Understandably, Figure 4 The two times of multi-scale feature fusion shown are only examples and are not used to specifically limit the number of multi-scale feature fusions. For example, the feature map Conv13 corresponding to the convolution layer 13 and the feature map Conv17_2 corresponding to the convolution layer 17_2 can also be multi-scale feature fused, and the feature map Conv14_2 corresponding to the convolution layer 14_2 and the feature map Conv17_2 corresponding to the convolution layer 17_2, and / or the feature map Conv15_2 corresponding to the convolution layer 15_2 and the feature map Conv17_2 corresponding to the convolution layer 17_2 can be multi-scale feature fused to obtain their respective corresponding fused feature maps.

[0123] Figure 5 FIG. 1 is a schematic diagram of the principle of multi-scale feature fusion in one embodiment. Figure 5As shown, the low-level feature map is convolved according to the first convolution kernel to obtain the first feature map, the high-level feature map is deconvolved according to the second convolution kernel to obtain the second feature map, the first feature map and the second feature map are iteratively operated on the channel to obtain the third feature map, and the third feature map is convolved according to the third convolution kernel to obtain a fused feature map.

[0124] For example, for the feature map Conv11 corresponding to the convolution layer 11, a convolution operation is performed according to the first convolution kernel 1*1*128 to obtain the first feature map C11, and for the feature map Conv17_2 corresponding to the convolution layer 17_2, a deconvolution operation is performed according to the second convolution kernel 3*3*128 to obtain the second feature map U17_2, the first feature map C11 and the second feature map U17_2 are superimposed on the channel to obtain the third feature map A11_17, and the third feature map A11_17 is convolved according to the third convolution kernel 1*1*128 to obtain the fused feature map Conv18_1.

[0125] It should be understood that although Figure 2-3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-3 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0126] In one embodiment, Figure 6 As shown, a target detection device 600 is provided, comprising: a construction module 602, a training module 604, a detection module 606 and a determination module 608, wherein:

[0127] A construction module 602 is used to construct a deep neural network model based on at least two multi-scale feature fusions; the at least two multi-scale feature fusions are performed in parallel; the multi-scale feature fusion refers to fusing a low-level feature map with a high-level feature map according to a preset feature fusion method;

[0128] A training module 604 is used to train the constructed deep neural network model based on the labeled training data set to obtain a trained object detection model;

[0129] The detection module 606 is used to detect the target image to be detected by using the target detection model to obtain a detection result;

[0130] The determination module 608 is configured to determine the target in the target image according to the detection result.

[0131] In one embodiment, the construction module 602 is also used to convolve the low-level feature map to obtain a first feature map with a preset number of channels; deconvolve the high-level feature map to obtain a second feature map with the same scale as the low-level feature map; superimpose the channels of the first feature map and the second feature map to obtain a third feature map; and convolve the third feature map to obtain a fused feature map after feature fusion.

[0132] In one embodiment, the training module 604 is also used to obtain a labeled training data set; select target training data from the training data set that has not been used for model training, and predict the target training data through the deep neural network model to be trained to obtain a prediction result; determine a comprehensive loss value according to the prediction result and the labeling result in the target training data according to a preset positioning loss function and a category loss function; reversely adjust the parameters of the deep neural network model according to the comprehensive loss value, and return to the step of selecting target training data from the training data set that has not been used for model training to continue executing until the iteration stop condition is met to obtain a trained target detection model.

[0133] In one embodiment, the detection module 606 is further used to detect the target image through the target detection model to obtain target feature maps of multiple target scales; through the target detection model, each target feature map is convolved with the corresponding convolution kernel to obtain a detection frame corresponding to each target feature map; the scale of the detection frame is dynamically determined based on the corresponding target feature map; through the target detection model, the detection result corresponding to the target image is determined according to the detection frame corresponding to each target feature map.

[0134] In one embodiment, the detection result includes at least one target detection frame corresponding to the target image; the determination module 608 is further used to extract sub-images corresponding to each target detection frame from the target image; identify each sub-image through a preconfigured classifier to obtain the category of the corresponding target detection frame; and determine the target in the target image according to the category of the target detection frame.

[0135] In one embodiment, the detection module 606 is further used to adjust the scale of the target image to a preset scale; the preset scale is larger than the standard scale corresponding to the deep neural network model.

[0136] For the specific definition of the target detection device, please refer to the definition of the target detection method above, which will not be repeated here. Each module in the above target detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0137] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store a trained target detection model. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a target detection method is implemented.

[0138] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the target detection method in each of the above embodiments when executing the computer program.

[0140] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the target detection method in each of the above embodiments are implemented.

[0141] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0142] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0143] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A target detection method, the method comprising: Constructing a deep neural network model based on at least two multi-scale feature fusions; wherein the at least two multi-scale feature fusions are performed in parallel; The multi-scale feature fusion refers to fusing the low-level feature map with the high-level feature map according to a preset feature fusion method; the low-level feature map of each multi-scale feature fusion is different, and the high-level feature map is the same; The constructed deep neural network model is trained based on the labeled training data set to obtain a trained target detection model; When a target image to be detected is obtained, the target image is detected by the target detection model to obtain a detection result; the detection result includes at least one target detection frame corresponding to the target image; The target detection frame is obtained by screening from multiple detection frames in a non-maximum suppression manner; Extracting a sub-image corresponding to each of the target detection frames from the target image; Identify each of the sub-images using a preconfigured classifier to obtain a category of a corresponding target detection frame; The category of the target detection frame refers to the category of the target in the target image that is within the target detection frame; The target in the target image is determined according to the category of the target detection frame.

2. The method according to claim 1, characterized in that The step of multi-scale feature fusion includes: Convolving the low-level feature map to obtain a first feature map with a preset number of channels; Deconvolving the high-level feature map to obtain a second feature map having the same scale as the low-level feature map; Perform channel superposition on the first feature map and the second feature map to obtain a third feature map; Convolution is performed on the third feature map to obtain a fused feature map after feature fusion.

3. The method according to claim 1, characterized in that The constructed deep neural network model is trained based on the labeled training data set to obtain a trained target detection model, including: Get the labeled training data set; Selecting target training data from a training data set that has not been used for model training, and predicting the target training data using the deep neural network model to be trained to obtain a prediction result; Determine a comprehensive loss value according to the prediction result and the labeling result in the target training data and a preset positioning loss function and a category loss function; Reversely adjust the parameters of the deep neural network model according to the comprehensive loss value, and return to the step of selecting target training data from the training data set that has not been used for model training to continue executing until the iteration stop condition is met to obtain a trained target detection model.

4. The method according to claim 1, characterized in that: The detecting the target image by using the target detection model to obtain a detection result includes: Detecting the target image using the target detection model to obtain target feature maps of multiple target scales; Through the target detection model, each target feature map is convolved with the corresponding convolution kernel to obtain a detection frame corresponding to each target feature map; the scale of the detection frame is dynamically determined based on the corresponding target feature map; Through the target detection model, the detection result corresponding to the target image is determined according to the detection box corresponding to each target feature map.

5. The method according to any one of claims 1 to 4, characterized in that: Before detecting the target image by the target detection model to obtain a detection result, the method further includes: The scale of the target image is adjusted to a preset scale; the preset scale is larger than a standard scale corresponding to the deep neural network model.

6. A target detection device, characterized in that: The device comprises: A construction module is used to construct a deep neural network model based on at least two multi-scale feature fusions; the at least two multi-scale feature fusions are performed in parallel; the multi-scale feature fusion refers to fusing the low-level feature map with the high-level feature map according to a preset feature fusion method; the low-level feature map of each multi-scale feature fusion is different, and the high-level feature map is the same; A training module is used to train the constructed deep neural network model based on the labeled training data set to obtain a trained target detection model; A detection module, configured to, when a target image to be detected is acquired, detect the target image through the target detection model to obtain a detection result; the detection result includes at least one target detection frame corresponding to the target image; the target detection frame is selected from multiple detection frames in a non-maximum suppression manner; A determination module is used to extract a sub-image corresponding to each target detection frame from the target image; identify each sub-image through a preconfigured classifier to obtain the category of the corresponding target detection frame; the category of the target detection frame refers to the category of the target in the target image that is within the target detection frame; and determine the target in the target image according to the category of the target detection frame.

7. The device according to claim 6, characterized in that The construction module is also used to convolve the low-level feature map to obtain a first feature map with a preset number of channels; deconvolve the high-level feature map to obtain a second feature map with the same scale as the low-level feature map; superimpose the channels of the first feature map and the second feature map to obtain a third feature map; and convolve the third feature map to obtain a fused feature map after feature fusion.

8. The device according to claim 6, characterized in that The training module is also used to obtain a labeled training data set; select target training data from a training data set that has not been used for model training, and predict the target training data through the deep neural network model to be trained to obtain a prediction result; determine a comprehensive loss value according to the prediction result and the labeling result in the target training data according to a preset positioning loss function and a category loss function; reversely adjust the parameters of the deep neural network model according to the comprehensive loss value, and return to the step of selecting target training data from a training data set that has not been used for model training to continue executing until the iteration stop condition is met to obtain a trained target detection model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images