A target image detection method and device, electronic equipment and storage medium

Abstract

The application provides a target image detection method and device, electronic equipment and storage medium, applied to the image detection technical field, the detection method comprises: acquiring a target image to be detected; inputting the target image to be detected into the backbone network of a pre-trained target detection model, performing feature extraction on the target image to be detected, determining the shallow feature map and the deep feature map of the target image to be detected; inputting the shallow feature map and the deep feature map into the feature pyramid network of the target detection model, respectively weighting the shallow feature map and the deep feature map with different weights, and determining the fusion feature map; inputting the fusion feature map into the head network of the target detection model, and determining the target object in the target image to be detected. By weighting the shallow feature map and the deep feature map with different weights in the feature pyramid network, the fusion feature map is obtained, and the detection efficiency and accuracy of the target image are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image detection, in particular to a target image detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the image auditing process, the auditing personnel faces a huge amount of data every day, usually relying on manpower for auditing, which is time-consuming and laborious, or using a deep learning neural network to audit images. The deep learning neural network includes a backbone network, a neck network and a head network, wherein the neck network plays a connecting role, so a good neck network can better pass the features.

[0003] At present, in the current image detection method, the neck network generally uses a feature pyramid (Feature Pyramid Networks, FPN for short), which mainly fuses local features in the shallow layer and semantic features in the deep layer. However, in the feature fusion process, the importance of different features is often different, which will lead to the fused features not being optimal, thereby affecting the detection result of the head network on the image. Therefore, how to improve the accuracy of image detection has become a technical problem that cannot be ignored. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a target image detection method and device, electronic equipment and storage medium, by inputting the shallow feature map and deep feature map of the target image into the feature pyramid network, and performing weighted processing on the shallow feature map and deep feature map in the feature pyramid network with different weights to obtain a fused feature map, thereby improving the detection efficiency and accuracy of the target image.

[0005] The present application provides a target image detection method, which comprises:

[0006] obtaining a target image to be detected;

[0007] inputting the target image to be detected into a backbone network of a pre-trained target detection model, extracting features of the target image to be detected, and determining a shallow feature map and a deep feature map of the target image to be detected;

[0008] inputting the shallow feature map and the deep feature map into a feature pyramid network of the target detection model, respectively weighting the shallow feature map and the deep feature map with different weights, and determining a fused feature map;

[0009] inputting the fused feature map into a head network of the target detection model, and determining a target object in the target image to be detected.

[0010] In a possible implementation, the feature pyramid network inputs the shallow feature map and the deep feature map into the target detection model, respectively processes the shallow feature map and the deep feature map with different weights, and determines a fusion feature map, including:

[0011] The feature pyramid network inputs the shallow feature map and the deep feature map to perform feature splicing, and determines a spliced feature map;

[0012] The feature pyramid network inputs the spliced feature map to a convolution layer, performs convolution processing on the spliced feature map, and determines a first target feature map;

[0013] The feature pyramid network inputs the first target feature map to a pooling layer, performs global average pooling processing on the first target feature map, and determines a second target feature map;

[0014] The feature pyramid network inputs the second target feature map to an activation layer, performs activation processing on the second target feature map, and determines a first target weight and a second target weight;

[0015] The feature pyramid network inputs the first target weight and the second target weight to a fusion layer, performs weighted processing on the shallow feature map by using the first target weight and performs weighted processing on the deep feature map by using the second target weight, and determines the fusion feature map.

[0016] In a possible implementation, the feature pyramid network inputs the spliced feature map to a convolution layer, performs convolution processing on the spliced feature map, and determines a first target feature map, including:

[0017] An initialized two-dimensional matrix is obtained, where a dimension of the initialized two-dimensional matrix is determined according to a number of convolution kernels and a number of feature vectors of the spliced feature map in different dimensions;

[0018] Based on feature information corresponding to the feature vectors of the spliced feature map in different dimensions, a target value of the feature vectors of the spliced feature map in different dimensions is determined;

[0019] The target value of the feature vectors of the spliced feature map in different dimensions is added to the initialized two-dimensional matrix to generate a target two-dimensional matrix;

[0020] Each of the convolution kernels filters out the feature vectors of the spliced feature map in different dimensions according to target values of the feature vectors of the spliced feature map in different dimensions in the target two-dimensional matrix, and performs convolution processing to determine the first target feature map.

[0021] In a possible implementation, the step of inputting the second target feature map into the activation layer of the feature pyramid network to perform activation processing on the second target feature map to determine the first target weight and the second target weight comprises:

[0022] performing normalization processing on the second target feature map by using a sigmoid activation function to determine a weight vector;

[0023] dividing based on dimensions of the weight vector to determine the first target weight and the second target weight.

[0024] In a possible implementation, the step of inputting the first target weight and the second target weight into the fusion layer of the feature pyramid network to perform weighted processing on the shallow feature map by using the first target weight and on the deep feature map by using the second target weight to determine the fusion feature map comprises:

[0025] performing weighted processing on the shallow feature map by using the first target weight to determine a weighted shallow feature map;

[0026] performing weighted processing on the deep feature map by using the second target weight to determine a weighted deep feature map;

[0027] performing feature addition on the weighted shallow feature map and the weighted deep feature map to determine the fusion feature map.

[0028] In a possible implementation, the target detection model is trained by the following steps:

[0029] obtaining a plurality of sample pictures and label information corresponding to each sample picture, and dividing the plurality of sample pictures into a training set and a verification set;

[0030] performing multiple times of training on an initial training model by using the training set and label information corresponding to the sample pictures in the training set to determine the target detection model;

[0031] testing the target detection model by using the verification set to verify a detection accuracy of the target detection model.

[0032] Embodiments of the present application further provide a target image detection device, which comprises:

[0033] an acquisition module configured to acquire a target image to be detected;

[0034] a feature extraction module configured to input the target image to be detected into a backbone network of a pre-trained target detection model, perform feature extraction on the target image to be detected, and determine a shallow feature map and a deep feature map of the target image to be detected;

[0035] a feature fusion module configured to input the shallow feature map and the deep feature map into a feature pyramid network of the target detection model, perform weighted processing on the shallow feature map and the deep feature map respectively with different weights, and determine a fusion feature map;

[0036] a determination module configured to input the fusion feature map into a head network of the target detection model, and determine a target object in the target image to be detected.

[0037] In a possible implementation, when the feature fusion module is configured to input the shallow feature map and the deep feature map into the feature pyramid network of the target detection model, perform weighted processing on the shallow feature map and the deep feature map respectively with different weights, and determine a fusion feature map, the feature fusion module is specifically configured to:

[0038] input the shallow feature map and the deep feature map into the feature pyramid network for feature concatenation, and determine a concatenated feature map;

[0039] input the concatenated feature map into a convolution layer of the feature pyramid network, perform convolution processing on the concatenated feature map, and determine a first target feature map;

[0040] input the first target feature map into a pooling layer of the feature pyramid network, perform global average pooling processing on the first target feature map, and determine a second target feature map;

[0041] input the second target feature map into an activation layer of the feature pyramid network, perform activation processing on the second target feature map, and determine a first target weight and a second target weight;

[0042] input the first target weight and the second target weight into a fusion layer of the feature pyramid network, perform weighted processing on the shallow feature map by using the first target weight and on the deep feature map by using the second target weight, and determine the fusion feature map.

[0043] The embodiment of the present application also provides an electronic device, comprising a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the target image detection method.

[0044] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the steps of the target image detection method.

[0045] The embodiment of the present application provides a target image detection method, device, electronic device and storage medium, the detection method comprises: acquiring a target image to be detected; inputting the target image to be detected into a backbone network of a pre-trained target detection model, performing feature extraction on the target image to be detected, and determining shallow feature maps and deep feature maps of the target image to be detected; inputting the shallow feature maps and the deep feature maps into a feature pyramid network of the target detection model, respectively weighting the shallow feature maps and the deep feature maps with different weights, and determining fusion feature maps; inputting the fusion feature maps into a head network of the target detection model, and determining target objects in the target image to be detected. By inputting the shallow feature maps and the deep feature maps of the target image into the feature pyramid network, the shallow feature maps and the deep feature maps are weighted with different weights in the feature pyramid network, the fusion feature maps are obtained, and the detection efficiency and accuracy of the target image are improved.

[0046] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0048] Figure 1 A flowchart of a target image detection method provided by the embodiment of the present application;

[0049] Figure 2 A schematic diagram of convolution of a convolution kernel in a target image detection method provided by the embodiment of the present application;

[0050] Figure 3 A schematic diagram of a fusion feature map determination process in a target image detection method provided by an embodiment of the present application;

[0051] Figure 4 A structural schematic diagram of a target image detection device provided by an embodiment of the present application;

[0052] Figure 5 A structural schematic diagram of a target image detection device provided by an embodiment of the present application;

[0053] Figure 6 A structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of description and illustration, and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportions. The flowcharts show the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowcharts or removed from the flowcharts by those skilled in the art under the guidance of the content of the present application.

[0055] In addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0056] In order to enable those skilled in the art to use the content of the present application, the following implementation is given in combination with a specific application scenario "detecting an image", and those skilled in the art can apply the general principles defined herein to other embodiments and application scenarios without departing from the spirit and scope of the present application.

[0057] The method, device, electronic equipment or computer readable storage medium provided in the embodiments of the present application can be applied to any scene requiring image detection, and the embodiments of the present application do not limit the specific application scene, and any solution using the target image detection method, device, electronic equipment and storage medium provided in the embodiments of the present application is within the protection scope of the present application.

[0058] Firstly, the application scenarios applicable to the present application are introduced. The present application can be applied to the field of image detection technology.

[0059] It is found through research that at present, in the current image detection method, the neck network generally adopts a feature pyramid (Feature Pyramid Networks, FPN for short), which mainly functions to fuse local features at a shallow layer and semantic features at a deep layer. However, in the process of feature fusion, the importance of different features is often different, which may result in that the fused features are not optimal, thereby affecting the detection result of the head network on the image. Therefore, how to improve the accuracy of image detection has become a technical problem that cannot be ignored.

[0060] Based on this, the embodiments of the present application provide a target image detection method, which inputs the shallow feature map and the deep feature map of the target image into the feature pyramid network, performs weighted processing on the shallow feature map and the deep feature map in the feature pyramid network with different weights, and obtains a fused feature map, thereby improving the detection efficiency and accuracy of the target image.

[0061] Please refer to Figure 1 , Figure 1 The flowchart of the target image detection method provided in the embodiments of the present application. As shown in Figure 1 , the detection method provided in the embodiments of the present application comprises:

[0062] S101: Obtain a target image to be detected.

[0063] In this step, the target image to be detected is obtained. Here, the target image to be detected can be a contraband image to be audited by an auditor, or an image to be audited and detected. Here, the type of the target image is not limited, and any image to be audited and detected can be used.

[0064] S102: Input the target image to be detected into a backbone network of a pre-trained target detection model, extract features of the target image to be detected, and determine a shallow feature map and a deep feature map of the target image to be detected.

[0065] In this step, the target image to be detected is input into the backbone network of the pre-trained target detection model. The backbone network extracts features of the target image to be detected, and determines shallow feature maps and deep feature maps of the target image to be detected.

[0066] Here, the shallow feature map is closer to the input target image to be detected and contains more pixel point information, some fine-grained information, such as color, texture, edge and corner information of the target image to be detected. The shallow feature map has a small receptive field and a small overlapping area between receptive fields, so that the network can capture more details.

[0067] Here, the deep feature map contains some coarse-grained information, which contains more abstract information, i.e. semantic information. The deep feature map has an increased receptive field and an increased overlapping area between receptive fields. When image information is compressed, some information of the whole target image to be detected is obtained.

[0068] In one possible implementation, the target detection model is trained by the following steps:

[0069] A: Obtain a plurality of sample pictures and label information corresponding to each sample picture, and divide the plurality of sample pictures into a training set and a validation set.

[0070] Here, a plurality of sample pictures and label information corresponding to each sample picture are collected in the network, and the plurality of sample pictures are divided into a training set and a validation set in a ratio of 9:1.

[0071] B: Use the training set and the label information corresponding to the sample pictures in the training set to train the initial training model multiple times to determine the target detection model.

[0072] Here, an initial training model is first constructed. The initial training model is a neural network model. The initial training model is trained multiple times using the training set and the label information corresponding to the plurality of sample pictures in the training set to obtain the target detection model.

[0073] C: Test the target detection model using the validation set to verify the detection accuracy of the target detection model.

[0074] Here, after training, the target detection model is tested using the validation set to verify the performance of the target detection model. If the performance of the target detection model is high, it means that the target detection model can detect target images. If the performance of the target detection model is low, the target detection model needs to be further trained.

[0075] In specific embodiments, pictures of a plurality of sample guns are acquired, the pictures of the plurality of sample guns are made into a detection picture data set, and the detection picture data set is divided into a training set and a verification set; an initial training model is constructed, the initial training model is trained multiple times using the training set and label information corresponding to the plurality of sample gun pictures in the training set, a target detection model is obtained, and after training is completed, the target detection model is tested using the verification set to verify the performance of the target detection model, so that the target detection model can detect contraband guns according to a target image.

[0076] S103: input the shallow feature map and the deep feature map into a feature pyramid network of the target detection model, respectively weight process the shallow feature map and the deep feature map with different weights, and determine a fused feature map.

[0077] In this step, the shallow feature map and the deep feature map obtained in the backbone network are input into the feature pyramid network, the shallow feature map and the deep feature map are weighted processed with different weights in the feature pyramid network and feature fusion is performed, and a fused feature map is determined.

[0078] Here, the weights for weighting processing the shallow feature map and the deep feature map are different.

[0079] In a possible implementation, the inputting the shallow feature map and the deep feature map into the feature pyramid network of the target detection model, respectively weighting processing the shallow feature map and the deep feature map with different weights, and determining a fused feature map, comprises:

[0080] a: inputting the shallow feature map and the deep feature map into the feature pyramid network for feature splicing to determine a spliced feature map.

[0081] Here, the shallow feature map and the deep feature map are input into the feature pyramid network, and the shallow feature map and the deep feature map are spliced to obtain a spliced feature map.

[0082] Wherein, the dimensions of the shallow feature map and the deep feature map are consistent, if the dimensions of the shallow feature map and the deep feature map are W*H*C, then the dimension of the spliced feature map is W*H*2C.

[0083] b: inputting the spliced feature map into a convolution layer of the feature pyramid network to perform convolution processing on the spliced feature map to determine a first target feature map.

[0084] Here, the spliced feature map is input into the convolution layer of the feature pyramid network, the spliced feature map is convolution processed to determine the first target feature map.

[0085] The first target feature map is obtained by performing convolution on the spliced feature map.

[0086] In a possible implementation, the convolutional layer of the feature pyramid network is configured to perform convolution on the spliced feature map to determine the first target feature map.

[0087] (1) obtaining an initialized two-dimensional matrix, wherein the dimension of the initialized two-dimensional matrix is determined according to the number of convolution kernels and the feature vectors of the spliced feature map in different dimensions.

[0088] In the formula, if the dimension of the spliced feature map is W*H*2C, and the feature vectors of the spliced feature map in different dimensions are the feature vectors corresponding to the spliced feature map in W*H*1, W*H*2, …, and W*H*2C dimensions.

[0089] In the formula, the number of convolution kernels in the initialized two-dimensional matrix is consistent with the number of channels of the spliced feature map. If the dimension of the spliced feature map is W*H*2C, the dimension of the initialized two-dimensional matrix is 2C*2C.

[0090] In the formula, the columns of the initialized two-dimensional matrix represent 2C convolution kernels, and the rows of the initialized two-dimensional matrix represent the feature vectors of the spliced feature map in 2C different dimensions.

[0091] (2) determining target values of the feature vectors of the spliced feature map in different dimensions based on the feature information corresponding to the feature vectors of the spliced feature map in different dimensions.

[0092] Here, the target values of the feature vectors of the spliced feature map in different dimensions are determined according to the importance of the feature information corresponding to the feature vectors of the spliced feature map in different dimensions.

[0093] Here, in the process of training and learning, the initialized two-dimensional matrix selects the feature vectors of the spliced feature map in the dimensions that need to be convolved according to the importance of the feature information corresponding to the feature vectors of the spliced feature map in different dimensions, and assigns the target values of the feature vectors of the spliced feature map in the dimensions that need to be convolved to be greater than 0, and assigns the target values of the feature vectors of the spliced feature map in the dimensions that do not need to be convolved to be less than 0.

[0094] In the formula, the importance of the feature information corresponding to the feature vectors of the spliced feature map in different dimensions is determined according to expert experience or network learning.

[0095] If the importance of the feature information corresponding to the feature vector of the spliced feature map in different dimensions is high, the target value of the feature vector is assigned to be greater than 0, and if the importance is low, the target value of the feature vector is assigned to be less than 0.

[0096] For example, the spliced feature map is a contraband image of a tool, and the feature images of the contraband image of the tool in different dimensions are different. For example, the feature image of the contraband image of the tool in a certain dimension includes color and texture information, and the importance of the feature information in this dimension is low. For example, the feature image of the contraband image of the tool in a certain dimension includes the contour of the tool, and the importance of the feature information in this dimension is high.

[0097] (3) Adding the target values of the feature vectors of the spliced feature map in different dimensions to the initialized two-dimensional matrix to generate a target two-dimensional matrix.

[0098] Here, the target values of the feature vectors of the spliced feature map in different dimensions are added to the initialized two-dimensional matrix to generate a target two-dimensional matrix.

[0099] Each row in the target two-dimensional matrix is a feature vector of a spliced feature map in 2C different dimensions and a target value corresponding to the feature vector of the spliced feature map in the dimension, and each column in the target two-dimensional matrix is a convolution kernel.

[0100] (4) Each convolution kernel filters out the corresponding feature vector of the spliced feature map in different dimensions according to the target values of the feature vectors of the spliced feature map in different dimensions in the target two-dimensional matrix, and performs convolution processing to determine the first target feature map.

[0101] Here, each convolution kernel filters out the corresponding feature vector of the spliced feature map in different dimensions according to the target values of the feature vectors of the spliced feature map in different dimensions in the target two-dimensional matrix, and performs convolution processing to determine the first target feature map.

[0102] Here, each convolution kernel needs to determine the corresponding feature vector of the spliced feature map in the dimension that needs to be convolved, and does not need to perform convolution on the feature vector of the spliced feature map in each dimension.

[0103] In the target two-dimensional matrix (M), the values are constantly updated during the training process. If M(i,j) >= 0, it means that the i-th convolution kernel needs to perform convolution on the feature vector of the spliced feature map in the j-th dimension, and if M(i,j) < 0, it means that the i-th convolution kernel does not perform convolution on the feature vector of the spliced feature map in the j-th dimension.

[0104] Each convolution kernel is used to convolve the feature vectors of the spliced feature maps in different corresponding dimensions to obtain a plurality of convolution feature maps, and the plurality of convolution feature maps are spliced to obtain a first target feature map.

[0105] Here, refer to Figure 2 , Figure 2 A schematic diagram of convolution of a convolution kernel in a target image detection method provided by an embodiment of the present application is shown in Figure 2 The black dots represent target values of the feature vectors of the spliced feature maps in the dimension greater than 0, and the white dots represent target values of the feature vectors of the spliced feature maps in the dimension less than 0. The first square is the feature vector of the spliced feature map in the different dimension corresponding to the target two-dimensional matrix, and the second square is the convolution kernel corresponding to the target matrix. For example, the first, third, fourth and seventh black dots in the first row of the target two-dimensional matrix represent that the first convolution kernel needs to convolve the feature vectors of the spliced feature maps in the first, third, fourth and seventh dimensions; the first, fourth, fifth and eighth black dots in the second row of the target two-dimensional matrix represent that the second convolution kernel needs to convolve the feature vectors of the spliced feature maps in the first, fourth, fifth and eighth dimensions, and the same applies to the following. In this way, the technical problem that the fused features are not optimal because different feature vectors are fused using the same weight during the feature fusion process of the FPN is avoided. Instead, the feature vectors of the spliced feature maps in the dimensions corresponding to each convolution kernel are selected for convolution, without convolving the feature vectors of the spliced feature maps in all dimensions, so that the feature semantics of the first target feature map is higher.

[0106] c: input the first target feature map into the pooling layer of the feature pyramid network, perform global average pooling processing on the first target feature map, and determine a second target feature map.

[0107] Here, the first target feature map is input into the pooling layer, the first target feature map is subjected to global average pooling processing, and a second target feature map is determined.

[0108] d: input the second target feature map into the activation layer of the feature pyramid network, perform activation processing on the second target feature map, and determine a first target weight and a second target weight.

[0109] Herein, the second target feature map is input to the activation layer of the feature pyramid network, the second target feature map is activated, and the first target weight and the second target weight are determined.

[0110] In a possible implementation, the second target feature map is input to the activation layer of the feature pyramid network, the second target feature map is activated, and the first target weight and the second target weight are determined, including:

[0111] I: The second target feature map is normalized by using a sigmoid activation function, and a weight vector is determined.

[0112] Herein, the second target feature map is normalized by using a sigmoid activation function, and a weight vector is determined.

[0113] II: The first target weight and the second target weight are determined based on the dimension of the weight vector.

[0114] Herein, the first target weight and the second target weight are determined by dividing the dimension of the weight vector. For example, if the dimension of the first weight is 1*1*2C, the weight vector is restored to the first target weight and the second target weight with the size of 1*1*C, and the first target weight and the second target weight have the same dimension but different weight contents.

[0115] e: The first target weight and the second target weight are input to the fusion layer of the feature pyramid network, the shallow feature map is weighted by using the first target weight, and the deep feature map is weighted by using the second target weight, and the fusion feature map is determined.

[0116] Herein, the first target weight and the second target weight are input to the fusion layer of the feature pyramid network, the shallow feature map is weighted, and the deep feature map is weighted, and the fusion feature map is determined.

[0117] In a possible implementation, the first target weight and the second target weight are input to the fusion layer of the feature pyramid network, the shallow feature map is weighted by using the first target weight, and the deep feature map is weighted by using the second target weight, and the fusion feature map is determined, including:

[0118] 1): The shallow feature map is weighted by using the first target weight, and the weighted shallow feature map is determined.

[0119] Herein, the shallow feature map is weighted by using the first target weight, and the weighted shallow feature map is determined.

[0120] 2) using the second target weight to perform weighting processing on the deep feature map, to determine a weighted deep feature map.

[0121] Here, the second target weight is used to perform weighting processing on the deep feature map, to determine a weighted deep feature map.

[0122] 3) performing feature addition on the weighted shallow feature map and the weighted deep feature map, to determine the fusion feature map.

[0123] Here, the weighted shallow feature map and the weighted deep feature map are added to determine the fusion feature map.

[0124] Here, the first target weight can also be used to perform weighting processing on the deep feature map, to determine a weighted deep feature map, and the second target weight is used to perform weighting processing on the shallow feature map, to determine a weighted shallow feature map.

[0125] Further, please refer to Figure 3 , Figure 3 is a schematic diagram of a fusion feature map determination process in a target image detection method provided by an embodiment of the present application. As shown in Figure 3 , the dimension of the shallow feature map is W*H*C, the dimension of the deep feature map is W*H*C, the shallow feature map and the deep feature map are spliced to obtain a spliced feature map, the dimension of the spliced feature map is W*H*2C, each convolution kernel filters out a feature vector of the spliced feature map in different dimensions according to a target two-dimensional matrix, and performs convolution processing to determine a first target feature map, the dimension of the first target feature map is W*H*2C, the first target feature map is subjected to pooling processing to determine a second target feature map, the dimension of the second target feature map is 1*1*2C, the second target feature map is subjected to activation processing to determine a first target weight 1*1*C and a second target weight 1*1*C, the first target weight is used to perform weighting processing on the shallow feature map to determine a weighted shallow feature map, the second target weight is used to perform weighting processing on the deep feature map to determine a weighted deep feature map, and the weighted shallow feature map and the weighted deep feature map are added to determine a fusion feature map. Because the feature semantics of the shallow feature map and the deep feature map are different, if equal weights are used for weighting processing, it will cause the technical problem that the semantic information of the fusion feature map is inaccurate, and the present application uses the determined first target weight to perform weighting processing on the shallow feature map and uses the second target weight to perform weighting processing on the deep feature map, thereby avoiding the technical problem that the fused features are not optimal because different feature vectors are fused using the same weight in the feature fusion process of the FPN, and improving the accuracy of the semantic information of the fusion feature map.

[0126] S104: input the fusion feature map into the head network of the target detection model to determine the target object in the target image to be detected.

[0127] In this step, the fusion feature map is input into the head network of the target detection model to determine the target object in the target image to be detected.

[0128] Here, if the target image is a contraband image that needs to be audited by an auditor, the identified target object can be a knife, a gun or other contraband.

[0129] In specific embodiments, a target image to be detected is obtained, and the target image to be detected is input into the backbone network of a target detection model to output a shallow feature map and a deep feature map; the shallow feature map and the deep feature map are input into a feature pyramid network of the target detection model, and the shallow feature map and the deep feature map are respectively processed with different weights to output a fusion feature map; the feature pyramid network layer is increased with a two-dimensional matrix to enable the fusion of the shallow feature map and the deep feature map in a non-equal weight manner; the fusion feature map is input into the head network of the target detection model to predict the fusion feature map, and the target image to be detected includes contraband.

[0130] The detection method for a target image provided by the embodiments of the present application includes: obtaining a target image to be detected; inputting the target image to be detected into a backbone network of a pre-trained target detection model to extract features of the target image to be detected and determine a shallow feature map and a deep feature map of the target image to be detected; inputting the shallow feature map and the deep feature map into a feature pyramid network of the target detection model to respectively process the shallow feature map and the deep feature map with different weights and determine a fusion feature map; and inputting the fusion feature map into a head network of the target detection model to determine a target object in the target image to be detected. By inputting the shallow feature map and the deep feature map of the target image into the feature pyramid network, the shallow feature map and the deep feature map are processed with different weights in the feature pyramid network to obtain the fusion feature map, thereby improving the detection efficiency and accuracy of the target image.

[0131] Please refer to Figure 4 、 Figure 5 , Figure 4 Figure 1 is a structural schematic diagram of a detection device for a target image provided by the embodiments of the present application, Figure 5 Figure 2 is another structural schematic diagram of a detection device for a target image provided by the embodiments of the present application. As shown in Figure 4 Figure 2, the detection device 400 for a target image includes:

[0132] The acquisition module 410 is configured to acquire a target image to be detected.

[0133] The feature extraction module 420 is configured to input the target image to be detected into a backbone network of a pre-trained target detection model, perform feature extraction on the target image to be detected, and determine a shallow feature map and a deep feature map of the target image to be detected.

[0134] The feature fusion module 430 is configured to input the shallow feature map and the deep feature map into a feature pyramid network of the target detection model, perform weighted processing on the shallow feature map and the deep feature map respectively with different weights, and determine a fusion feature map.

[0135] The determination module 440 is configured to input the fusion feature map into a head network of the target detection model, and determine a target object in the target image to be detected.

[0136] Further, when the feature fusion module 430 is configured to input the shallow feature map and the deep feature map into the feature pyramid network of the target detection model, perform weighted processing on the shallow feature map and the deep feature map respectively with different weights, and determine a fusion feature map, the feature fusion module 430 is specifically configured to:

[0137] input the shallow feature map and the deep feature map into the feature pyramid network for feature splicing, and determine a spliced feature map;

[0138] input the spliced feature map into a convolution layer of the feature pyramid network, perform convolution processing on the spliced feature map, and determine a first target feature map;

[0139] input the first target feature map into a pooling layer of the feature pyramid network, perform global average pooling processing on the first target feature map, and determine a second target feature map;

[0140] input the second target feature map into an activation layer of the feature pyramid network, perform activation processing on the second target feature map, and determine a first target weight and a second target weight;

[0141] input the first target weight and the second target weight into a fusion layer of the feature pyramid network, perform weighted processing on the shallow feature map by using the first target weight and perform weighted processing on the deep feature map by using the second target weight, and determine the fusion feature map.

[0142] Further, the feature fusion module 430 is specifically used for:

[0143] obtaining an initialized two-dimensional matrix, wherein dimensions of the initialized two-dimensional matrix are determined according to a number of convolution kernels and a number of feature vectors of the spliced feature map in different dimensions;

[0144] determining target values of the feature vectors of the spliced feature map in different dimensions based on feature information corresponding to the feature vectors of the spliced feature map in different dimensions;

[0145] adding the target values of the feature vectors of the spliced feature map in different dimensions in the initialized two-dimensional matrix to generate a target two-dimensional matrix;

[0146] each of the convolution kernels filters corresponding feature vectors of the spliced feature map in different dimensions according to the target values of the feature vectors of the spliced feature map in different dimensions in the target two-dimensional matrix, and performs convolution processing to determine the first target feature map.

[0147] Further, the feature fusion module 430 is specifically used for:

[0148] performing normalization processing on the second target feature map by using a sigmoid activation function to determine a weight vector;

[0149] dividing dimensions of the weight vector to determine the first target weight and the second target weight.

[0150] Further, the feature fusion module 430 is specifically used for:

[0151] performing weighting processing on the shallow feature map by using the first target weight to determine a weighted shallow feature map;

[0152] performing weighting processing on the deep feature map by using the second target weight to determine a weighted deep feature map;

[0153] The weighted shallow feature map and the weighted deep feature map are added to determine the fusion feature map.

[0154] Further, as shown in Figure 5 The target image detection device 400 further includes a model training module 450, which trains the target detection model through the following steps:

[0155] A plurality of sample pictures and corresponding label information of each sample picture are obtained, and the plurality of sample pictures are divided into a training set and a validation set;

[0156] The initial training model is trained multiple times using the training set and the label information corresponding to the sample pictures in the training set to determine the target detection model;

[0157] The target detection model is tested using the validation set to verify the detection accuracy of the target detection model.

[0158] The target image detection device provided in the embodiments of the present application includes an acquisition module for acquiring a target image to be detected, a feature extraction module for inputting the target image to be detected into a backbone network of a pre-trained target detection model to extract features of the target image to be detected and determine a shallow feature map and a deep feature map of the target image to be detected, a feature fusion module for inputting the shallow feature map and the deep feature map into a feature pyramid network of the target detection model, weighting the shallow feature map and the deep feature map with different weights respectively, and determining a fusion feature map, and a determination module for inputting the fusion feature map into a head network of the target detection model to determine a target object in the target image to be detected. The shallow feature map and the deep feature map of the target image are input into the feature pyramid network, the shallow feature map and the deep feature map are weighted with different weights in the feature pyramid network, and the fusion feature map is obtained, thereby improving the detection efficiency and accuracy of the target image.

[0159] Please refer to Figure 6 , Figure 6 A structural schematic diagram of an electronic device provided in the embodiments of the present application. As shown in Figure 6 The electronic device 600 includes a processor 610, a memory 620 and a bus 630.

[0160] The memory 620 stores machine readable instructions executable by the processor 610, when the electronic device 600 is running, the processor 610 and the memory 620 communicate through the bus 630, the machine readable instructions are executed by the processor 610, can execute the above Figure 1 The steps of the detection method of the target image in the method embodiment are shown, and the specific implementation can be referred to the method embodiment, which will not be repeated here.

[0161] The embodiment of the application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is run by the processor, and the computer program can execute the above Figure 1 The steps of the detection method of the target image in the method embodiment are shown, and the specific implementation can be referred to the method embodiment, which will not be repeated here.

[0162] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

[0163] In several embodiments provided in the application, it should be understood that the disclosed system, device and method can be implemented by other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interface, device or unit, which can be electrical, mechanical or other forms.

[0164] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0165] In addition, the functional units in each embodiment of the application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0166] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application essentially or the parts of the prior art that make contributions or parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0167] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit them. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present application, or make equivalent replacements to some of the technical features. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of detecting a target image, characterized by, The detection method comprises: obtaining a target image to be detected; inputting the target image to be detected into a backbone network of a pre-trained target detection model, performing feature extraction on the target image to be detected, and determining shallow feature maps and deep feature maps of the target image to be detected; inputting the shallow feature maps and the deep feature maps into a feature pyramid network of the target detection model, respectively weighting the shallow feature maps and the deep feature maps with different weights, and determining fusion feature maps; inputting the fusion feature maps into a head network of the target detection model, and determining target objects in the target image to be detected; the inputting the shallow feature maps and the deep feature maps into the feature pyramid network of the target detection model, respectively weighting the shallow feature maps and the deep feature maps with different weights, and determining the fusion feature maps comprises: inputting the shallow feature maps and the deep feature maps into the feature pyramid network for feature splicing, and determining spliced feature maps; inputting the spliced feature maps into a convolution layer of the feature pyramid network, performing convolution processing on the spliced feature maps, and determining first target feature maps; inputting the first target feature maps into a pooling layer of the feature pyramid network, performing global average pooling processing on the first target feature maps, and determining second target feature maps; inputting the second target feature maps into an activation layer of the feature pyramid network, performing activation processing on the second target feature maps, and determining first target weights and second target weights; inputting the first target weights and the second target weights into a fusion layer of the feature pyramid network, weighting the shallow feature maps with the first target weights and weighting the deep feature maps with the second target weights, and determining the fusion feature maps.

2. The detection method according to claim 1, characterized in that, the inputting the spliced feature maps into the convolution layer of the feature pyramid network, performing convolution processing on the spliced feature maps, and determining the first target feature maps comprises: obtaining an initialized two-dimensional matrix, wherein the dimensions of the initialized two-dimensional matrix are determined according to the number of convolution kernels and the number of feature vectors of the spliced feature maps in different dimensions; determining target values of the feature vectors of the spliced feature maps in different dimensions based on feature information corresponding to the feature vectors of the spliced feature maps in different dimensions; adding the target values of the feature vectors of the spliced feature maps in different dimensions in the initialized two-dimensional matrix to generate a target two-dimensional matrix; each convolution kernel filters corresponding feature vectors of the spliced feature maps in different dimensions according to the target values of the feature vectors of the spliced feature maps in different dimensions in the target two-dimensional matrix, and performs convolution processing to determine the first target feature maps.

3. The method of claim 1, wherein, the inputting the second target feature maps into the activation layer of the feature pyramid network, performing activation processing on the second target feature maps, and determining the first target weights and the second target weights comprises: The second target feature map is normalized by using a sigmoid activation function to determine a weight vector; The weight vector is divided based on its dimension to determine the first target weight and the second target weight.

4. The method of claim 1, wherein The first target weight and the second target weight are input into the fusion layer of the feature pyramid network, the shallow feature map is weighted by using the first target weight, and the deep feature map is weighted by using the second target weight to determine the fusion feature map, including: The shallow feature map is weighted by using the first target weight to determine a weighted shallow feature map; The deep feature map is weighted by using the second target weight to determine a weighted deep feature map; The weighted shallow feature map and the weighted deep feature map are added in feature to determine the fusion feature map.

5. The method of claim 1, wherein The target detection model is trained by the following steps: A plurality of sample pictures and label information corresponding to each sample picture are obtained, and the plurality of sample pictures are divided into a training set and a validation set; The initial training model is trained multiple times by using the training set and the label information corresponding to the sample picture in the training set to determine the target detection model; The target detection model is tested by using the validation set to verify the detection accuracy of the target detection model.

6. An apparatus for detecting a target image, characterized by comprising: The detection device includes: An acquisition module configured to acquire a target image to be detected; A feature extraction module configured to input the target image to be detected into a backbone network of a pre-trained target detection model to extract features of the target image to be detected and determine a shallow feature map and a deep feature map of the target image to be detected; A feature fusion module configured to input the shallow feature map and the deep feature map into a feature pyramid network of the target detection model to weight the shallow feature map and the deep feature map with different weights and determine a fusion feature map; A determination module configured to input the fusion feature map into a head network of the target detection model to determine a target object in the target image to be detected. The feature fusion module is specifically configured to: input the shallow feature map and the deep feature map into the feature pyramid network for feature splicing to determine a spliced feature map; input the spliced feature map into a convolution layer of the feature pyramid network to perform convolution processing on the spliced feature map and determine a first target feature map; input the first target feature map into a pooling layer of the feature pyramid network to perform global average pooling processing on the first target feature map and determine a second target feature map; The second target feature map is input to an activation layer of the feature pyramid network, and the second target feature map is subjected to activation processing to determine a first target weight and a second target weight; The first target weight and the second target weight are input to a fusion layer of the feature pyramid network, and the shallow layer feature map is subjected to weighting processing by using the first target weight and the deep layer feature map is subjected to weighting processing by using the second target weight to determine the fusion feature map.

7. An electronic device, comprising: Comprise: A processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to execute the steps of the target image detection method in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to execute the steps of the target image detection method in any one of claims 1 to 5.

Images