A target detection system and acquisition method

Abstract

The application discloses a target detection system, a feature extraction module distributes the feature of the pixel dimension of the current frame image to the channel dimension, extracts feature maps of at least three scales from the processed image, and correspondingly obtains a first scale feature map to an Nth scale feature map; a feature fusion module fuses the i th scale feature map to the Nth scale feature map of the current frame image to obtain an i th fused feature map, obtains an Nth fused feature map according to the Nth scale feature map of the current frame image, and a detection module detects a target according to the first fused feature map to the Nth fused feature map of the current frame image respectively to obtain a detection result of the current frame image. When the image feature is extracted, the pixel dimension information is focused to the channel dimension, and then the image feature is extracted and the target is detected, so that the original pixel information of a small size target can be reserved to a large extent, and the detection precision and the detection efficiency of small target detection can be improved. The application further discloses a target detection system acquisition method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, in particular to a target detection system. The present application also relates to a target detection system acquisition method. BACKGROUND

[0002] Pedestrian target detection is a key technology in the field of video monitoring, and it is the basis for pedestrian tracking, pedestrian recognition, intelligent decision-making and other technologies in intelligent monitoring systems. When the monitoring camera terminal has a large field of view or is far from the ground, the imaging size of the pedestrian target in the image is small, and the texture features are not clear enough.

[0003] On the one hand, in a multi-scale detection network, the detection of small targets is mostly performed on shallow feature maps. When the absolute size of the target is 8-16 pixels and the relative size is 1%-4%, the color and texture information of the pedestrian target is very scarce, and it cannot be well expressed even on a shallow feature map with high resolution. On the other hand, there is a large amount of complex background information around each pedestrian target, which makes the semantic information of the deep feature map contain more noise and cannot well represent small-size targets. These two reasons greatly affect the detection performance of the neural network on small-size targets. Therefore, how to improve the detection accuracy and efficiency of small targets has become a problem to be solved by those skilled in the art. SUMMARY

[0004] The purpose of the present application is to provide a target detection system that can improve the detection accuracy and efficiency of small targets in images. The present application also provides a target detection system acquisition method.

[0005] To achieve the above purpose, the present application provides the following technical solutions:

[0006] A target detection system includes a feature extraction module, a feature fusion module and a detection module.

[0007] The feature extraction module is used to distribute the feature distribution of the current frame image pixel dimension to the channel dimension, and extract at least three scale feature maps from the processed image, to obtain the first scale feature map to the Nth scale feature map, N is a positive integer greater than 2.

[0008] The feature fusion module is used to fuse the i-th scale feature map to the N-th scale feature map of the current frame image to obtain the i-th fusion feature map, and obtain the N-th fusion feature map according to the N-th scale feature map of the current frame image, i∈[1,N-1];

[0009] The detection module is used to detect targets according to the first fusion feature map to the Nth fusion feature map of the current frame image respectively, to obtain the detection result of the current frame image.

[0010] Preferably, the feature extraction module comprises a preprocessing submodule, which is configured to divide the current frame image into a plurality of cells according to pixels, splice pixels at corresponding positions of the cells into a same image, and connect the obtained images in a channel dimension to distribute features of the current frame image in a pixel dimension to the channel dimension.

[0011] Preferably, the feature extraction module comprises a plurality of feature extraction submodules connected in sequence, and each feature extraction submodule is configured to perform feature extraction processing and down-sampling processing on a feature map output by a previous feature extraction submodule to obtain a feature map of a corresponding scale.

[0012] Preferably, the feature extraction submodule outputting the feature map of the Nth scale is specifically configured to perform feature extraction processing on the feature map output by the previous feature extraction submodule, perform pooling on the obtained feature map by using a plurality of kernels of different sizes, connect the pooled feature maps in the channel dimension, and then perform down-sampling processing on the obtained feature map to output the feature map of the Nth scale.

[0013] Preferably, the feature fusion module comprises a first fusion submodule and a second fusion submodule.

[0014] The first fusion submodule is configured to perform processing on the feature map of the Nth scale to obtain a feature map of the Nth pre-fusion, and connect a feature map obtained by performing up-sampling processing on the feature map of the i+1th pre-fusion and the feature map of the ith scale in the channel dimension to obtain the feature map of the ith pre-fusion.

[0015] The second fusion submodule is configured to perform down-sampling processing on the feature map of the first pre-fusion to obtain a feature map of the first fusion, and perform down-sampling processing on a feature map obtained by connecting the feature map of the ith fusion and the feature map of the i+1th pre-fusion in the channel dimension to obtain the feature map of the i+1th fusion.

[0016] Preferably, the feature fusion module comprises an attention mechanism submodule, which is configured to obtain a feature map of the ith fusion feature map of the current frame image subjected to the attention mechanism in the channel dimension according to hidden features corresponding to the ith fusion feature map of the current frame image and the ith fusion feature map of the previous frame image, obtain a feature map subjected to the attention mechanism in the pixel dimension according to the feature map subjected to the attention mechanism in the channel dimension, and perform effective information processing on the ith fusion feature map of the current frame image according to the feature map subjected to the attention mechanism and the hidden features corresponding to the ith fusion feature map of the previous frame image, and take the obtained feature map as the ith fusion feature map for target detection.

[0017] Preferably, the detection module is specifically configured to detect a target from the fusion feature map to obtain a (t x , ty , t w , t h , t o ), wherein (t x , t y ) represents the offset of the predicted target frame center (x c , y c ) relative to the upper left corner (c x , c y ) of the block where it is located, t w represents the ratio of the width of the predicted target frame to the width of the prior target frame, t h represents the ratio of the height of the predicted target frame to the height of the prior target frame, and t o represents the confidence of the predicted target frame. The confidence of the target frame reflects whether the target frame contains a target and the accuracy of the size and position of the target frame.

[0018] Preferably, the detection module is specifically configured to calculate the width ratio and height ratio of the target frame detected from the jth fusion feature map to the prior target frame for the jth fusion feature map, and if the obtained width ratio and height ratio do not meet the preset requirements, the target frame of the jth fusion feature map is excluded, j∈[1, N].

[0019] Preferably, the detection module is specifically configured to detect a target from the fusion feature map to obtain a block in the image where the center of the target frame is located, find a preset number of adjacent blocks of the block in the image, and use the block where the center of the target frame is located and the found preset number of blocks to predict the target frame.

[0020] A target detection system acquisition method is used to obtain the above-mentioned target detection system, wherein the data set used to train the target detection system includes a reference data set and an auxiliary data set, and the auxiliary data set is expanded according to the reference data set, including:

[0021] Obtain the size of the target in the image of the reference data set, and obtain the probability density distribution of the target size of the reference data set;

[0022] Randomly extract images from the auxiliary data set, and scale the extracted images so that the median of the target size in the image is consistent with the target size randomly sampled from the probability density distribution of the reference data set, to form the auxiliary data set from the adjusted images;

[0023] Randomly extract multiple images from the adjusted auxiliary data set and place them in a blank image to obtain the final auxiliary data set.

[0024] According to the technical scheme, the target detection system provided by the application focuses pixel dimension information to channel dimension when extracting image features, and then extracts image features and detects targets, so that the original pixel information of small-size targets can be retained to a large extent, and the detection precision and detection efficiency of small target detection can be improved.

[0025] The target detection system provided by the application focuses pixel dimension information to channel dimension when extracting image features, and then extracts image features and detects targets, so that the original pixel information of small-size targets can be retained to a large extent, and the detection precision and detection efficiency of small target detection can be improved.

[0026] The target detection system provided by the application focuses pixel dimension information to channel dimension when extracting image features, and then extracts image features and detects targets, so that the original pixel information of small-size targets can be retained to a large extent, and the detection precision and detection efficiency of small target detection can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only constitute some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0028] Figure 1 A schematic diagram of a target detection system provided by an embodiment of the application;

[0029] Figure 2 A schematic diagram of a target detection system provided by an embodiment of the application;

[0030] Figure 3 A schematic diagram of a target detection system provided by an embodiment of the application;

[0031] Figure 4 A schematic diagram of a convolution module used by the target detection system of the embodiment of the application;

[0032] Figure 5 A schematic diagram of a down-sampling module used by the feature extraction submodule of the embodiment of the application;

[0033] Figure 6 A schematic diagram of a pooling module used by the feature extraction submodule of the embodiment of the application for outputting the Nth scale feature map;

[0034] Figure 7 The schematic diagram of the target detection system of the embodiment of the present application obtaining the first fusion feature map to the Nth fusion feature map according to the current frame image through the feature fusion module;

[0035] Figure 8 The schematic diagram of the attention mechanism submodule of the embodiment of the present application;

[0036] Figure 9 The schematic diagram of the convolution attention mechanism submodule of the attention mechanism submodule of the embodiment of the present application;

[0037] Figure 10 The schematic diagram of the detection module of the embodiment of the present application detecting the target frame in the feature map;

[0038] Figure 11 The method flow chart of the embodiment of the present application expanding the auxiliary data set according to the reference data set. DETAILED DESCRIPTION

[0039] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present application.

[0040] Please refer to Figure 1 , Figure 1 The schematic diagram of a target detection system provided by the embodiment of the present application, as shown in the figure, the target detection system comprises a feature extraction module 10, a feature fusion module 11 and a detection module 12.

[0041] The feature extraction module 10 is used for distributing the feature distribution of the pixel dimension of the current frame image to the channel dimension, and extracting at least three scale feature maps from the processed image, and correspondingly obtaining the first scale feature map to the Nth scale feature map, N is a positive integer greater than 2.

[0042] The feature fusion module 11 is used for fusing the ith scale feature map to the Nth scale feature map of the current frame image to obtain the ith fusion feature map, and obtaining the Nth fusion feature map according to the Nth scale feature map of the current frame image, i∈[1,N-1];

[0043] The detection module 12 is used for detecting the target according to the first fusion feature map to the Nth fusion feature map of the current frame image respectively, and obtaining the detection result of the current frame image.

[0044] The feature extraction module 10 converts the feature information of the pixel dimension of the current frame image to the channel dimension of the image. A feature map of a scale refers to a feature map obtained by extracting features from an image at a sampling rate. Fusing feature maps of different scales refers to that the fused feature map contains the features contained in each of the different scale feature maps.

[0045] The target detection system of the embodiment focuses the pixel dimension features of the image to the channel dimension when extracting image features, then extracts multiple different scale features from the image, fuses the feature maps of the different scales, and detects targets, which can greatly retain the original pixel information of small size targets and improve the detection accuracy and detection efficiency of small target detection.

[0046] Optionally, refer to Figure 2 , Figure 2 A structural schematic diagram of the target detection system of the embodiment is shown in the figure. The feature extraction module 10 can include a preprocessing submodule 100, which is used to divide the current frame image into multiple unit cells according to pixels, splice the pixels at the corresponding positions of each unit cell into the same image, and connect the obtained images in the channel dimension to distribute the features of the pixel dimension of the current frame image to the channel dimension.

[0047] The current frame image is divided into multiple unit cells according to pixels, each unit cell includes multiple pixels, and the length and width of the unit cell can be equal, i.e., the length and width of the unit cell include the same number of pixels, or the length and width of the unit cell can be unequal, i.e., the length and width of the unit cell include different numbers of pixels. In actual applications, the number of pixels included in the divided unit cell can be set according to actual application requirements. For example, refer to Figure 3 , Figure 3 A schematic diagram for distributing the features of the pixel dimension of the image to the channel dimension in a specific example is shown in the figure. The original image is divided into unit cells with a length and a width of 2 pixels, the pixels at the corresponding positions of each unit cell are spliced to form a new image, 2x2=4 images of 2 times down-sampling are obtained, then the obtained images are connected in the channel dimension, and finally a convolution operation is performed.

[0048] The target detection system of the embodiment focuses the pixel dimension information of the image to the channel dimension through the above down-sampling processing of the image, which can greatly retain the original pixel information of small size targets, reduce the loss of original image information, and increase the receptive field and reduce the calculation amount.

[0049] Optionally, refer to Figure 2As shown, the feature extraction module 10 can include a plurality of feature extraction sub-modules 101 connected in sequence, which are used to extract and down-sample the feature map output by the previous feature extraction sub-module 101 to obtain a feature map of a corresponding scale. In combination with the above Figure 2 As shown, the image output by the preprocessing sub-module 100 is input to a plurality of feature extraction sub-modules 101 connected in sequence, and the first scale feature map to the Nth scale feature map are output in sequence through the processing of each feature extraction sub-module 101.

[0050] Specifically, the feature extraction sub-module 101 can include a convolution module for extracting features from a feature map and a down-sampling module for down-sampling the feature map output by the convolution module, which are connected in sequence. Please refer to Figure 4 , Figure 4 The schematic diagram of the convolution module used by the target detection system of the present embodiment includes an input layer, a convolution layer, a normalization layer, and an activation layer, where c_in represents the input channel number, c_out represents the output channel number, k represents the convolution kernel size, and s represents the step size.

[0051] Please refer to Figure 5 , Figure 5 The schematic diagram of the down-sampling module used by the feature extraction sub-module of the present embodiment, where the four numbers in the brackets represent (input channel number, output channel number, convolution kernel size, step size), the two numbers represent (input channel number, output channel number), and the single number represents (output channel number). The input data first passes through the convolution module to reduce the channel number from c_in to c_, then passes through N Bottleneck modules, and then passes through a convolution layer to unify the channel number to c_ to obtain a branch; on the other hand, the input data directly passes through another convolution layer to reduce the channel number from c_in to c_ to obtain another branch. The data obtained from the two branches is spliced in the channel dimension, and then passes through a normalization layer and an activation layer, and finally passes through an additional convolution layer to obtain the final output result. The Bottleneck module includes an input layer and a convolution module, and has a shortcut channel. Exemplary Figure 5 The Bottleneck module can include an input layer, a convolution module (c_, c_, 1, 1), and a convolution module (c_, c_, 3, 1) connected in sequence.

[0052] In the target detection system of the present embodiment, the use of the down-sampling module in the feature extraction module can further reduce the computational cost, eliminate redundant gradient information, and the use of the above down-sampling module can enhance the learning ability, so that the accuracy is maintained while being lightweight.

[0053] Preferably, the feature extraction submodule 101 that outputs the Nth scale feature map is specifically used to: extract features from the feature map output by the previous feature extraction submodule 101, pool the obtained feature maps using multiple kernels of different sizes, and concatenate the pooled feature maps along the channel dimension. Then, the obtained feature maps are downsampled to output the Nth scale feature map. Accordingly, the feature extraction submodule 101 that outputs the Nth scale feature map may specifically include a convolution module for extracting features from the feature map, a pooling module, and a downsampling module connected in sequence.

[0054] Please refer to the examples. Figure 6 , Figure 6 This diagram illustrates the pooling module within the feature extraction submodule that outputs the Nth scale feature map in this embodiment. The module includes an input layer, a convolutional module, multiple max-pooling layers, a connection layer, and another convolutional module. Each max-pooling layer uses a kernel of a different size. First, the input data is passed through the convolutional module to reduce the number of channels from c_in to c_in / 2. Then, it is passed through each max-pooling layer with a different kernel size, and zero-padding is applied. The pooled tensors are concatenated along the channel dimension. Finally, a convolutional module compresses the tensor dimension to c_out to obtain the final output result. Figure 6 The diagram includes a 1×1 max-pooling layer (equivalent to preserving the original tensor), and may also include max-pooling layers with kernels of 5×5, 9×9, or 13×13 and a stride of 1. The feature extraction submodule 101 performs the above processing to achieve multi-scale feature fusion, which enriches the semantic expressive power of the deep feature map.

[0055] For example, in a specific instance, the feature extraction module 10 includes a preprocessing submodule 100 and four feature extraction submodules 101 connected in sequence. The second, third and fourth feature extraction submodules 101 output three feature maps of different scales with downsampling rates of 8, 16 and 32, respectively, with sizes of 100×56, 50×28 and 25×14, denoted as route1, route2 and route3.

[0056] Preferred options can be referenced. Figure 2 and Figure 7 , Figure 7 This is a schematic diagram illustrating how the feature fusion module obtains the first to Nth fused feature maps based on the current frame image in this embodiment. The feature fusion module 11 includes a first fusion submodule 110 and a second fusion submodule 111, wherein:

[0057] The first fusion submodule 110 is configured to process the N-th scale feature map to obtain an N-th pre-fusion feature map, and connect a feature map obtained by up-sampling the (i+1)-th pre-fusion feature map with the i-th scale feature map in the channel dimension to obtain an i-th pre-fusion feature map.

[0058] The second fusion submodule 111 is configured to perform down-sampling on the 1-st pre-fusion feature map to obtain a 1-st fusion feature map, and perform down-sampling on a feature map obtained by connecting the i-th fusion feature map with the (i+1)-th pre-fusion feature map in the channel dimension to obtain an (i+1)-th fusion feature map.

[0059] For example, taking the three different scale feature maps with down-sampling rates of 8, 16 and 32 output by the feature extraction module 10 as an example, the first fusion submodule 110 first processes the 32 times down-sampled feature map route3 output by the last feature extraction submodule 101 of the feature extraction module 10 through a convolution module to obtain a corresponding feature map route4; then performs 2 times up-sampling through an up-sampling layer using interpolation, and splices the 16 times down-sampled feature map route2 output by the feature extraction module 10 in the channel dimension, sequentially passes through a down-sampling module and a convolution module to obtain a corresponding feature map route5; then performs up-sampling through an up-sampling layer, and splices the 8 times down-sampled feature map route1 output by the feature extraction module 10 in the channel dimension to obtain a corresponding feature map route6.

[0060] The second fusion submodule 111 processes the route6 through a down-sampling module to obtain route7; splices the route5 output by the first fusion submodule 110 after down-sampling through a convolution module with a step of 2, and processes the route7 through a down-sampling module to obtain route8; then splices the route4 output by the first fusion submodule 110 through a convolution module with a step of 2, and processes the route8 through a down-sampling module to obtain route9. The route7, route8 and route9 output by the second fusion submodule 111 are respectively used as fusion feature maps of different scales for detecting targets. Optionally, the down-sampling module can be implemented by using the down-sampling module shown in the figure, and the shortcut channel can be cancelled. Figure 5

[0061] Since the shallow appearance information and the deep semantic information of the small size target are both insufficient, the combination of the first fusion submodule 110 and the second fusion submodule 111 can guide the shallow feature map with the semantic information of the deep feature map, and the deep feature map can be supplemented with the appearance details of the shallow feature map, so that the detection rate of the small target is greatly improved.

[0062] ​Further preferably, the feature fusion module 11 can further comprise an attention mechanism submodule 112, which is configured to: obtain a feature map of the i-th fusion feature map of the current frame image subjected to the attention mechanism in the channel dimension according to hidden features corresponding to the i-th fusion feature map of the previous frame image and the i-th fusion feature map of the current frame image, obtain a feature map subjected to the attention mechanism in the pixel dimension according to the feature map subjected to the attention mechanism in the channel dimension, and perform effective information extraction processing on the i-th fusion feature map of the current frame image according to the feature map subjected to the attention mechanism and the hidden features corresponding to the i-th fusion feature map of the previous frame image, and take the obtained feature map as the i-th fusion feature map for target detection. The hidden features corresponding to the i-th fusion feature map of a frame image refer to features obtained by performing effective information extraction processing on the i-th fusion feature map of the frame image.

[0063] For example, refer to Figure 8 and Figure 9 , Figure 8 FIG. 4 is a schematic diagram of the attention mechanism submodule of the embodiment, Figure 9 FIG. 5 is a schematic diagram of the convolutional attention mechanism submodule in the attention mechanism submodule of the embodiment. The processing process is as follows: first, the feature map of the current frame image with a size of [h, w, c] and the hidden information of the previous frame image are spliced to generate a tensor with a size of [h, w, 2c], which is input into a convolutional layer, a global average pooling layer, a 1x1 convolutional layer (equivalent to a fully connected layer) to generate a channel attention matrix with a size of [1, 1, 2c], and then multiplied with the feature map of the current frame image in the channel dimension to realize the soft attention mechanism in the channel dimension. Second, the feature map after the channel dimension background semantic information suppression is input into two convolutional layers to generate a spatial attention matrix with a size of [h, w, 1], and then multiplied with the feature map in the pixel dimension to realize the soft attention mechanism in the spatial dimension. The finally output tensor is spliced with the hidden information of the previous frame image again, and then input into the next processing process.

[0064] In addition, the effective information extraction processing on the i-th fusion feature map of the current frame image can be performed according to the feature map of the current frame image subjected to the attention mechanism by a convolutional gated recurrent unit comprising a reset gate and an update gate. For details, refer to Figure 8As shown, the feature map of the current frame image and the hidden feature corresponding to the i-th fusion feature map of the previous frame image are connected and input into reset gate r and update gate z respectively for processing, to obtain the feature map of the i-th fusion feature map of the current frame image for extracting effective information processing, as the i-th fusion feature map for target detection. The reset gate r is used to control the proportion of the feature map passing through the attention mechanism and the hidden feature corresponding to the i-th fusion feature map of the previous frame image, and the update gate z is used to remember effective information and forget invalid information. Optionally, the calculation formula is as follows:

[0065] z = σ(W Z *[h t-1 ,x t ]), r = σ(W r *[h t-1 ,x t ]);

[0066]

[0067] wherein x t represents input data, h t-1 represents memory content, i.e., hidden feature of the previous frame image, z represents update gate, r represents reset gate, h' represents an intermediate variable, which can be understood as preliminary memory information generated by the current frame, W z , W r , and W h represent weights respectively, and h t represents output hidden feature. The output hidden feature h t can be used for subsequent frame image target frame regression and classification tasks.

[0068] The target tracking system of the embodiment fuses the feature map of the previous frame and the feature map of the current frame before sending the feature map into the detection module, combines the spatial and temporal information, and suppresses a large number of background negative samples through the attention mechanism to reduce the false detection rate and improve the detection rate. Under the monitoring video, for the small target size of pedestrians, most of the content in the picture is background information, and since different depth feature layers are responsible for predicting different scale targets, which leads to the fact that the target information contained in the deep feature layer is much less than the background information. However, the system of the embodiment adopts a hybrid attention mechanism of spatial dimension and channel dimension, which can suppress the background feature information in the spatial dimension and strengthen the target semantic information in the channel dimension, so as to reduce false positives and improve the detection rate. And at the same time, the information in the time dimension of the video data is used to improve the accuracy.

[0069] The detection module 12 detects targets according to the first fusion feature map to the Nth fusion feature map of the current frame image respectively to obtain the detection result of the current frame image. Specifically, the detection module 12 is specifically configured to detect targets on the fusion feature map to obtain the target frame (t x , ty , t w , t h , t o ), wherein (t x , t y ) represents the offset of the center (x c , y c ) of the predicted target frame relative to the upper left corner (c x , c y ) of the block in which it is located, t w represents the ratio of the width of the predicted target frame to the width of the prior target frame, t h represents the ratio of the height of the predicted target frame to the height of the prior target frame, and t o represents the confidence of the predicted target frame, which reflects whether the target is contained in the predicted target frame and the accuracy of the size and position of the predicted target frame.

[0070] wherein (x c , y c ) represents the center coordinates of the predicted target frame, w and h represent the width and height of the predicted target frame, respectively, and obj represents the confidence of the predicted target frame. (t x , t y , t w , t h , t o ) and (x c , y c , w, h, obj) have the following corresponding relationship:

[0071] x c = (σ (t x ) * 2 - 0.5 + c x ) * s, y c = (σ (t y ) * 2 - 0.5 + c y ) * s;

[0072] w = (σ (t w ) * τ) 2 * p w , h = (σ (t h ) * τ) 2 * p h ;

[0073] Pr (obj) * loU (pred, gt) = σ (t o );

[0074] wherein s represents the down-sampling rate of the current feature map, and σ () represents the Sigmoid function. It is worth noting that (c x , c y) represents the number of blocks from the center of the target frame to the top left corner of the image, and the normalized (x c ,y c ) is actually the relative value under the feature map, multiplied by the current feature map downsampling rate s, to get the absolute coordinates relative to the original image. pw represents the prior target frame width, and ph represents the prior target frame height. τ represents the ratio coefficient, which limits the width-height ratio to (0~τ 2 ) to prevent the predicted target frame from being too different from the prior target frame. An exemplary τ = 2 can be taken.

[0075] The frame confidence has two meanings, the first is whether the frame contains the target, and the second is whether the frame size and position are accurate. The latter is measured by the complete intersection over union (CIoU) of the two frames, which is expressed as:

[0076]

[0077]

[0078] ρ(*) represents the Euclidean distance between the centers of the two frames, and c represents the diagonal distance of the minimum circumscribed rectangle of the two frames.

[0079] Further preferably, the detection module 12 is specifically configured to: for the jth fused feature map, calculate the width ratio and height ratio of the target frame detected from the jth fused feature map and the prior target frame, and if the obtained width ratio and height ratio do not meet the preset requirement, exclude the target frame of the jth fused feature map, j ∈ [1, N].

[0080] Since the size difference between pedestrian targets is generally small, in order to solve the problem of uneven distribution of predicted target frames on different feature layers, the system directly uses the size matching rule, that is, the width-height ratio of all predicted target frames in the image and the prior target frame of the current layer is calculated. The width ratio of the predicted target frame and the prior target frame of the current layer and the height ratio of the predicted target frame and the prior target frame of the current layer are calculated. If the width ratio is greater than or less than the threshold τ 2 set in the previous section, and the height ratio is greater than or less than the threshold τ 2 set in the previous section, it means that the size difference between the predicted target frame and the prior target frame is too large, and the matching degree is not enough. The predicted target frame is considered as background in the prediction of this layer. In solving the problem of uneven distribution of predicted target frames, the number of positive samples is increased, and the network convergence is accelerated.

[0081] ​More preferably, the detection module 12 is specifically used to detect the target in the fused feature map, obtain the block in the image where the center of the target box is located, find a preset number of adjacent blocks in the image, and use the block where the center of the target box is located and the preset number of adjacent blocks to predict the target box. An example can be found in the following document. Figure 10 , Figure 10 This is a schematic diagram of the detection module detecting target boxes in the feature map in this embodiment. In each layer of the feature map, for all valid target boxes, it calculates which grid they fall into. This can be done using rounding rules to find the nearest preset number of grids, for example... Figure 10 The diagram shows the two nearest grid cells. The black dot represents the target center, and the gray grid cells represent the grid containing the target center and the two nearest grid cells. All three grid cells are considered responsible for predicting the target bounding box. This allows for a further increase in the number of positive samples, especially when small target pixels constitute a small proportion of the overall image, thus accelerating network convergence. Therefore, corresponding to the above-mentioned (x... c ,y c The range of predicted branch values ​​changes from 0 to 1 to -0.5 to 1.5.

[0082] In addition, during the network training phase, in order to balance positive and negative samples and balance sample categories, the loss function for target recognition and classification tasks is as follows:

[0083]

[0084] in, γ and α represent control parameters. For example, γ = 2 and α = 0.25 can be chosen. To enhance the detection capability of small targets, the loss of feature layers at different scales is multiplied by a balancing coefficient, i.e.:

[0085]

[0086] The smaller the value of the subscript i, the larger the size of the prior anchor box corresponding to the feature map. For example, ε1 = 1.0, ε2 = 2.0, and ε3 = 4.0 can be chosen. The target box (xc, yc, w, h) regression task uses the CIoU loss function, expressed as:

[0087]

[0088] The definition of CIoU is the same as described above. Therefore, the total loss is expressed as:

[0089] L = γ box L box +γ obj L obj +γ cls L cls ;

[0090] where γ denotes the loss weight of different tasks. An exemplary acceptable γ1= γ2= γ3= 1.0.

[0091] In addition, for video data, a hard attention mechanism is used to generate original image labels using historical prior information. In the video detection stage, for the case where the monitoring video scene is constant, after running for a period of time, a prior label that is constantly updated is generated for each image according to the detection of targets in historical frames, representing the probability density distribution of the pedestrian target appearing in the picture. At this time, only the high-probability region is enhanced in brightness, and the low-probability region is not suppressed at the pixel level, so the label is added with a bias of 1, and after being superimposed (multiplied by pixels) with the original image, it is sent into the network according to the above method to obtain the final detection result, thereby improving the detection rate.

[0092] Joint detection is combined with the foreground detection algorithm. In the detection stage of each frame of the video image, two pictures with a size of (w, h) are put into a batch. The first picture is obtained by scaling the original picture with a size of 1920x1080, and the second picture is obtained by splicing the extracted foreground after running the foreground detection algorithm (such as VIBE) on the original picture. Therefore, the network processes two pictures at a time, that is, the input dimension is (2, c, h, w), and the output results of the two pictures are combined together to run the non-maximum suppression algorithm (Non-Maximum Suppression) to obtain the joint detection result, thereby further improving the detection rate of small size targets.

[0093] Correspondingly, the embodiment of the present application also provides a target detection system acquisition method for obtaining the above-mentioned target detection system. The data set used for training the target detection system includes a benchmark data set and an auxiliary data set, wherein the auxiliary data set is expanded according to the benchmark data set. Please refer to Figure 11 , Figure 11 The method flow chart for expanding the auxiliary data set according to the benchmark data set in the embodiment is shown in the figure, and the method includes the following steps:

[0094] S20: Obtain the size of the target in the image of the benchmark data set, and obtain the probability density distribution of the target size of the benchmark data set.

[0095] Optionally, the distribution histogram of the target size can be counted according to the training set of the benchmark data set to estimate the probability density distribution P size (s;D train ), D trainThe training set represents the benchmark dataset, and D represents the benchmark dataset. According to the basic assumption of machine learning, the distribution of the randomly sampled training set is basically consistent with the distribution of the original dataset, and the probability density distribution of the training set can be used to estimate the probability density distribution P of the original dataset size (s; D).

[0096] Optionally, the absolute size of the target can be calculated according to the following formula:

[0097] Wherein, w, h respectively represent the width, height of the target, which can be specifically the width, height described in pixels.

[0098] In order to facilitate calculation, it is necessary to discretize the probability density distribution Psize(s; Dtrain) into a histogram (H, R). The number of groups of the histogram is set to K, and R[k]-, R[k]+ respectively represent the lower bound and the upper bound of the kth group. If the total number of targets included in the training set is N, let G ij (D train ) represents the jth target of the ith image in the training set, and the value of the kth group is:

[0099]

[0100] At the same time, in order to alleviate the problem of long-tailed distribution of target size in the dataset, it is necessary to perform truncation processing on the histogram, that is, to discard the groups with tail height less than a certain threshold, and then adjust the value of R[k] and re-count the histogram (H, R) to obtain the final distribution. Exemplarily, K=100 can be taken.

[0101] S21: Randomly sampling images in the auxiliary dataset, and scaling the sampled images so that the median of the target size in the image is consistent with the target size randomly sampled from the probability density distribution of the benchmark dataset, to form the auxiliary dataset by the adjusted images.

[0102] In the auxiliary dataset E, the image is extracted and the size is adjusted so that the absolute size conforms to the distribution Psize(s; D). According to the target size probability density distribution Psize(s; D) of the benchmark dataset D, a size s_dst is randomly sampled. In order to avoid the destruction of the structure of the image caused by directly scaling the target, the randomly sampled image is scaled so that the median s_mid of the target size in the scaled image is consistent with the size s_dst sampled.

[0103] Wherein, randomly sampling the target size from the probability density distribution of the benchmark dataset includes: randomly sampling the index k of the group from the probability density distribution of the benchmark dataset, and then sampling the size s_dst in the interval (R[k - , R[k +R[k] is the lower bound of the kth group, and R[k] - R[k] is the lower bound of the kth group, and R[k] + R[k] is the lower bound of the kth group, and R[k]

[0104] S22: Randomly extract multiple images from the adjusted auxiliary data set and place them into a blank image to obtain a final auxiliary data set.

[0105] In each batch of pre-training, a blank image is initialized, 1 picture is randomly extracted from the adjusted large-scale auxiliary data set E, the picture is scaled, and the scaled picture is randomly placed into the blank image. This is repeated k times to obtain an image containing k scaled backup data set pictures, which is then sent to the constructed neural network for training. Finally, the pre-training model is obtained on the auxiliary data set E, and the final network model is obtained by fine-tuning on the benchmark data set D. Optionally, the scale matching algorithm can be used to scale the picture, and the cutmix algorithm can be used to randomly place the scaled picture into the blank image.

[0106] Optionally, the benchmark data set can use the cleaned VIRAT data set. The portrait subset of the COCO data set can be used as the initial auxiliary data set. In the method of the embodiment, the scale matching algorithm is used to make the pedestrian target size probability density distribution of the large-scale auxiliary data set consistent with the benchmark data set, to expand the data set online and improve the model generalization ability.

[0107] The above describes in detail a target detection system and an acquisition method provided by the present application. The principles and implementation modes of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. It should be noted that for those skilled in the art, without departing from the principles of the present application, the present application can be improved and modified in several ways, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A target detection system, characterized in that, It includes a feature extraction module, a feature fusion module, and a detection module; The feature extraction module is used to distribute the pixel-dimensional features of the current frame image to the channel dimension, and extract feature maps at least three scales from the processed image to obtain feature maps at scales 1 to N, where N is a positive integer greater than 2. The feature fusion module is used to fuse the i-th scale feature map to the N-th scale feature map of the current frame image to obtain the i-th fused feature map, and to obtain the N-th fused feature map based on the N-th scale feature map of the current frame image, i∈[1,N-1]; The feature fusion module includes an attention mechanism submodule, which is used to: obtain a channel-dimensional attention mechanism feature map of the current frame image based on the hidden features corresponding to the i-th fused feature map of the current frame image and the i-th fused feature map of the previous frame image; obtain a pixel-dimensional attention mechanism feature map based on the channel-dimensional attention mechanism feature map; and extract effective information from the i-th fused feature map of the current frame image based on the attention mechanism feature map of the current frame image and the hidden features corresponding to the i-th fused feature map of the previous frame image, and use the obtained feature map as the i-th fused feature map for object detection; the processing includes: firstly, concatenating the feature map of the current frame image of size [h, w, c] with the hidden information of the previous frame image to generate a tensor of size [h, w, 2c], and feeding it into a convolutional layer, a global average pooling layer, and a fully connected layer to generate [1, 1, ... The first step is to multiply the channel attention matrix of size [2c] with the feature map of the current frame image in the channel dimension. The second step is to feed the feature map after the channel dimension background semantic information suppression is completed into two convolutional layers to generate a spatial attention matrix of size [h, w, 1]. Then, multiply it with the feature map in the pixel dimension. The output tensor is then concatenated with the hidden information of the previous frame image and then fed into the next processing step. The detection module is used to detect targets based on the first to Nth fusion feature maps of the current frame image, and obtain the detection result of the current frame image.

2. The target detection system according to claim 1, characterized in that, The feature extraction module includes a preprocessing submodule, which is used to divide the current frame image into multiple cells according to pixels, stitch the corresponding pixels of each cell into the same image, and connect the resulting images in the channel dimension to distribute the features of the current frame image in the pixel dimension to the channel dimension.

3. The target detection system according to claim 1, characterized in that, The feature extraction module includes multiple feature extraction sub-modules connected in sequence. The feature extraction sub-module is used to extract features and downsample the feature map output by the previous feature extraction sub-module to obtain a feature map of the corresponding scale.

4. The target detection system according to claim 3, characterized in that, The feature extraction submodule that outputs the Nth scale feature map is specifically used to: extract features from the feature map output by the previous feature extraction submodule, pool the obtained feature maps with multiple kernels of different sizes, connect the pooled feature maps in the channel dimension, and then downsample the obtained feature maps to output the Nth scale feature map.

5. The target detection system according to claim 1, characterized in that, The feature fusion module includes a first fusion submodule and a second fusion submodule; The first fusion submodule is used to process the Nth scale feature map to obtain the Nth pre-fused feature map, and to connect the feature map obtained by upsampling the (i+1)th pre-fused feature map with the ith scale feature map in the channel dimension to obtain the ith pre-fused feature map. The second fusion submodule is used to downsample the first pre-fused feature map to obtain the first fused feature map, and to downsample the feature map obtained by connecting the i-th fused feature map and the i+1-th pre-fused feature map in the channel dimension to obtain the i+1-th fused feature map.

6. The target detection system according to claim 1, characterized in that, The detection module is specifically used to detect targets from the fused feature map and obtain the target bounding box (t). x , t y , t w , t h , t o ), where (t x , t y ) represents the predicted center of the target box (x c ,y c ) relative to the top left corner of its block (c x , c y The offset of t w t represents the ratio of the predicted bounding box width to the prior bounding box width. h t represents the ratio of the predicted bounding box height to the prior bounding box height. o This represents the confidence level of the predicted target box. The confidence level of the target box reflects whether the target box contains the target and the accuracy of the target box size and position.

7. The target detection system according to claim 1, characterized in that, The detection module is specifically used to calculate the width ratio and height ratio of the target box detected from the j-th fusion feature map to the prior target box for the j-th fusion feature map. If the obtained width ratio and height ratio do not meet the preset requirements, the target box of the j-th fusion feature map is excluded, j∈[1,N].

8. The target detection system according to claim 1, characterized in that, The detection module is specifically used to detect targets in the fused feature map, obtain the block in the image where the center of the target box is located, find a preset number of adjacent blocks in the image, and the block where the center of the target box is located and the preset number of adjacent blocks are responsible for predicting the target box.

9. A method for acquiring a target detection system, characterized in that, For obtaining the object detection system according to any one of claims 1-8, wherein the dataset used to train the object detection system includes a benchmark dataset and an auxiliary dataset, and the auxiliary dataset is expanded according to the benchmark dataset including: Obtain the size of the target in the image of the benchmark dataset, and obtain the probability density distribution of the target size in the benchmark dataset; Images are randomly selected from the auxiliary dataset and scaled so that the median target size in the images is consistent with the target size randomly sampled from the probability density distribution of the baseline dataset, so that the adjusted images constitute the auxiliary dataset. Multiple images are randomly selected from the adjusted auxiliary dataset and placed into a blank image to obtain the final auxiliary dataset.

Images