Target detection method, system and storage medium

Abstract

The application discloses a target detection method, a system and a storage medium, and relates to the field of vehicle detection.The method comprises the following steps: acquiring original image data of a current frame of a target to be processed; inputting the original image data into a feature extraction network structure, and performing a down-sampling operation on the original image data by using the feature extraction network structure; performing an up-sampling operation on the down-sampled image data by using the feature extraction network structure; performing a feature extraction operation on the second up-sampled image data by using the feature extraction network structure; inputting the second up-sampled image data into a mask branch structure; performing a frame selection operation on the target to be processed according to feature classification information and second feature region information, obtaining a second mask region, and acquiring a first mask region corresponding to a previous frame; and performing a matching operation on the first mask region and the second mask region, and obtaining target detection data.The target detection method can improve the success matching rate of the detected object and improve the accuracy of tracking.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicle detection, in particular to a target detection method and system and a storage medium. BACKGROUND

[0002] In an automatic driving system, vehicle detection and tracking are the most basic and core technologies, and many assisted driving functions are realized on the basis of identifying effective vehicle targets. With the development of computer vision recognition theory, vehicle recognition and tracking technology has extended many improved methods from the earliest simple image processing to basic pattern recognition, and from traditional machine learning to computer deep learning.

[0003] For the task of road vehicle detection and tracking, most existing methods directly use the feature map and regression information output by the feature extraction network as the input for tracking. Since the regression box of the output target may contain some background information or other detected objects, these information may interfere with the extracted features, affect feature matching, and further affect tracking effect. SUMMARY

[0004] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application provides a target detection method, system and storage medium, which can improve the success matching rate of the detected object and improve the accuracy of tracking.

[0005] To solve the above technical problems, the present application proposes the following technical solutions:

[0006] The first aspect of the present application provides a target detection method, comprising:

[0007] obtaining original image data of a current frame of a target to be processed;

[0008] inputting the original image data into a feature extraction network structure, and performing a down-sampling operation on the original image data by the feature extraction network structure to obtain down-sampled image data;

[0009] performing an up-sampling operation on the down-sampled image data by the feature extraction network structure to obtain up-sampled image data, wherein the up-sampled image data includes first up-sampled image data and second up-sampled image data, and the second up-sampled image data is obtained from the first up-sampled image data by the up-sampling operation;

[0010] performing a feature extraction operation on the second up-sampled image data by the feature extraction network structure to obtain feature classification information corresponding to the target to be processed;

[0011] input the second up-sampling image data into a mask branch structure to obtain second feature region information corresponding to the target to be processed;

[0012] perform a frame selection operation on the target to be processed according to the feature classification information and the second feature region information to obtain a second mask region, and obtain a first mask region corresponding to a previous frame;

[0013] perform a matching operation on the first mask region and the second mask region to obtain target detection data corresponding to the target to be processed.

[0014] The target detection method according to the first aspect of the present application has at least the following beneficial effects: the target detection method of the present application inputs the original image data into the feature extraction network structure, performs a down-sampling operation on the original image data through the feature extraction network structure, obtains down-sampling image data with more rich semantic information, so that the extracted features have higher discrimination. After obtaining the down-sampling image data, an up-sampling operation is performed thereon, and further, a plurality of feature information in the original image data is classified through the feature extraction network structure to obtain feature classification information. After performing the second up-sampling operation to obtain the second up-sampling image data, the second up-sampling image data is input into the mask branch structure to obtain the feature region information corresponding to the target to be processed, which plays a role in region division for the occlusion problem of multi-target detection. Moreover, the target detection method of the present application combines the feature classification information and the feature region information to calculate the second mask region of the current frame, matches the calculated second mask region with the first mask region corresponding to the previous frame to obtain the target detection data, increases the successful matching rate of the target, and further improves the accuracy of tracking.

[0015] According to some embodiments of the first aspect of the present application, the down-sampling image data includes first down-sampling image data, second down-sampling image data, third down-sampling image data and fourth down-sampling image data, and the down-sampling operation performed on the original image data by the feature extraction network structure to obtain down-sampling image data includes:

[0016] perform a down-sampling operation on the original image data through 64 convolution kernels in the feature extraction network structure to obtain first down-sampling image data;

[0017] perform a down-sampling operation on the first down-sampling image data through 128 convolution kernels in the feature extraction network structure to obtain second down-sampling image data;

[0018] perform a down-sampling operation on the second down-sampling image data through 256 convolution kernels in the feature extraction network structure to obtain third down-sampling image data;

[0019] The 512 convolution kernels in the feature extraction network structure perform a downsampling operation on the third down-sampling image data to obtain fourth down-sampling image data.

[0020] According to some embodiments of the first aspect of the present application, the performing, by the feature extraction network structure, the up-sampling operation on the down-sampling image data to obtain up-sampling image data comprises:

[0021] The deformable convolution layer and the 256 convolution kernels in the feature extraction network structure perform an up-sampling operation on the fourth down-sampling image data, and fuse the third down-sampling image data to obtain first up-sampling image data.

[0022] The deformable convolution layer and the 128 convolution kernels in the feature extraction network structure perform an up-sampling operation on the first up-sampling image data, and fuse the second down-sampling image data to obtain second up-sampling image data.

[0023] According to some embodiments of the first aspect of the present application, the performing, by the feature extraction network structure, the feature extraction operation on the second up-sampling image data to obtain the feature classification information corresponding to the target to be processed comprises:

[0024] The feature extraction network structure performs the feature extraction operation on the second up-sampling image data according to one-hot coding to obtain the feature classification information corresponding to the target to be processed.

[0025] According to some embodiments of the first aspect of the present application, the inputting the second up-sampling image data into the mask branch structure to obtain the second feature region information corresponding to the target to be processed comprises:

[0026] The mask branch structure performs five convolution operations on the second up-sampling image data, and performs a channel reduction operation in the fifth convolution operation to obtain third up-sampling image data.

[0027] The mask branch structure performs mask processing on the third up-sampling image data to obtain the second feature region information corresponding to the target to be processed.

[0028] According to some embodiments of the first aspect of the present application, the performing, by the mask branch structure, the mask processing on the third up-sampling image data to obtain the second feature region information corresponding to the target to be processed comprises:

[0029] The mask branch structure performs a shielding operation on background information in the third up-sampling image data to obtain the second feature region information, wherein the second feature region information comprises predicted offset information and frame size information.

[0030] According to some embodiments of the first aspect of the present application, the performing a shielding operation on the background information in the third up-sampling image data through the mask branch structure to obtain the second feature region information comprises one of the following:

[0031] extracting the first feature region information of the first mask region and the second feature region information in the second mask region, performing a similarity calculation operation on the first feature region information and the second feature region information to obtain similarity data, and obtaining target detection data corresponding to the to-be-processed target according to the similarity data; or,

[0032] calculating first mask area information of the first mask region and second mask area information of the second mask region, performing an overlap calculation operation on the first mask area information and the second mask area information to obtain overlap data, and obtaining target detection data corresponding to the to-be-processed target according to the overlap data.

[0033] According to some embodiments of the first aspect of the present application, the calculation formula of the overlap data is as follows:

[0034]

[0035] wherein, the IoU(t, t+1) represents the overlap data, the DetMask(t) represents the first mask area information, and the DetMask(t+1) represents the second mask area information.

[0036] The second aspect of the present application provides a target detection system, comprising:

[0037] at least one memory;

[0038] at least one processor;

[0039] at least one program;

[0040] The program is stored in the memory, and the processor executes at least one of the programs to implement:

[0041] The target detection method according to any one of the first aspect of the present application.

[0042] The third aspect of the present application provides a computer readable storage medium, which stores computer executable signals, and the computer executable signals are used to execute:

[0043] The target detection method according to any one of the first aspect of the present application.

[0044] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the attendant drawings or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0045] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the attendant drawings or can be learned by practice of the application.

[0046] Figure 1 Flow chart of a target detection method provided by an embodiment of the application;

[0047] Figure 2 Flow chart of a down-sampling operation on original image data in a target detection method provided by an embodiment of the application;

[0048] Figure 3 Flow chart of an up-sampling operation on down-sampled image data in a target detection method provided by an embodiment of the application;

[0049] Figure 4 Flow chart of a feature extraction operation on second up-sampled image data in a target detection method provided by an embodiment of the application;

[0050] Figure 5 Flow chart of inputting second up-sampled image data into a mask branch structure in a target detection method provided by an embodiment of the application;

[0051] Figure 6 Flow chart of mask processing on third up-sampled image data in a target detection method provided by an embodiment of the application;

[0052] Figure 7 Module block diagram of a target detection system provided by some embodiments of the application. DETAILED DESCRIPTION

[0053] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0054] It should be noted that although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that in the flow chart. The terms in the specification and claims and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0055] In the description of the present application, if the first, second, only for the purpose of distinguishing technical features are described, and can not be understood as indicating or implying the relative importance or implicit indication of the number of indicated technical features or implicit indication of the order of the indicated technical features.

[0056] In the description of the present application, unless otherwise expressly limited, the words such as setting, installation, connection and the like should be broadly understood, and the person skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical scheme.

[0057] Reference Figure 1 In the first aspect, the embodiments of the present application provide a target detection method, including but not limited to steps S110, S120, S130, S140, S150, S160, S170.

[0058] Step S110, obtaining the original image data of the current frame of the target to be processed;

[0059] Step S120, inputting the original image data into the feature extraction network structure, and performing down-sampling operation on the original image data through the feature extraction network structure to obtain down-sampled image data;

[0060] Step S130, performing up-sampling operation on the down-sampled image data through the feature extraction network structure to obtain up-sampled image data, wherein the up-sampled image data includes first up-sampled image data and second up-sampled image data, and the second up-sampled image data is obtained by the first up-sampled image data through up-sampling operation;

[0061] Step S140, performing feature extraction operation on the second up-sampled image data through the feature extraction network structure to obtain the feature classification information corresponding to the target to be processed;

[0062] Step S150, inputting the second up-sampled image data into the mask branch structure to obtain the second feature region information corresponding to the target to be processed;

[0063] Step S160, performing frame selection operation on the target to be processed according to the feature classification information and the second feature region information to obtain the second mask region, and obtaining the first mask region corresponding to the previous frame;

[0064] Step S170, performing matching operation on the first mask region and the second mask region to obtain the target detection data corresponding to the target to be processed.

[0065] It should be noted that in the detection and tracking process of the vehicle, the target in the fixed view angle can move or deform, and since different positions in the same network layer can correspond to objects of different scales or different deformations, the traditional convolution cannot well adapt to the scale or shape of the target, which can reduce the efficiency of target tracking. The target detection method of the present application is based on the identification and matching of the vehicle in the continuous frame, adopts multiple downsampling operations and upsampling operations, and fuses deformable convolution in the process of performing upsampling on the original image data, more accurately adapts to the scale or shape of the target, and tracks the moving vehicle. In actual application, the target detection method of the present application can be applied to autonomous vehicles and real-time vehicle flow counters. In other embodiments, the target detection method of the present application can also be applied to the tracking and detection of other moving targets, which is not limited by the present application.

[0066] According to one embodiment of the present application, the target detection method of the present application inputs the original image data into the feature extraction network structure, performs downsampling operation on the original image data through the feature extraction network structure, obtains downsampling image data with more rich semantic information, so that the extracted features have higher discrimination. And after obtaining the downsampling image data, it performs upsampling operation, it can be understood that deformable convolution is used in the process of performing upsampling operation, so that the convolution kernel better self-adapts the receptive field, can flexibly adapt to the scale or shape of the target, and further classifies the numerous feature information in the original image data through the feature extraction network structure, and obtains feature classification information. After performing the second upsampling operation to obtain the second upsampling image data, the second upsampling image data is input into the mask branch structure to obtain the second feature region information corresponding to the target to be processed, and the second feature region information plays a role in region division for the occlusion problem of multi-target detection. And the target detection method of the present application combines the feature classification information and the second feature region information to calculate the second mask region of the current frame, matches the calculated second mask region and the first mask region corresponding to the previous frame to obtain target detection data, increases the success matching rate of the target, and further improves the accuracy of tracking.

[0067] More specifically, the feature extraction network structure of the present application is designed based on a feature pyramid structure, which includes 6 blocks, each block includes multiple convolution layers, activation function layers and BN layers, and the network further adds a deformable convolution layer to adapt to the irregular changes of the target. When the target deforms in the video frame due to time lapse, the deformable convolution layer can automatically adjust to the actual situation of the target appearing in the video frame, which is convenient for better extracting the features of the original image data of the current frame.

[0068] Reference Figure 2In a first aspect, another embodiment of the present application provides an operation method for performing downsampling on original image data. The operation method comprises the following steps of: performing downsampling on the original image data by using a feature extraction network structure to obtain down-sampled image data.

[0069] In step S210, downsampling is performed on the original image data by using 64 convolution kernels in the feature extraction network structure to obtain first down-sampled image data.

[0070] In step S220, downsampling is performed on the first down-sampled image data by using 128 convolution kernels in the feature extraction network structure to obtain second down-sampled image data.

[0071] In step S230, downsampling is performed on the second down-sampled image data by using 256 convolution kernels in the feature extraction network structure to obtain third down-sampled image data.

[0072] In step S240, downsampling is performed on the third down-sampled image data by using 512 convolution kernels in the feature extraction network structure to obtain fourth down-sampled image data.

[0073] According to an embodiment of the present application, the down-sampled image data comprises the first down-sampled image data, the second down-sampled image data, the third down-sampled image data, and the fourth down-sampled image data. The feature extraction network structure performs downsampling on the original image data by using 64 convolution kernels to obtain first down-sampled image data which is 1 / 2 times the original image data, then continues to perform downsampling on the first down-sampled image data by using 128 convolution kernels to obtain second down-sampled image data which is 1 / 4 times the original image data, then continues to perform downsampling on the second down-sampled image data by using 256 convolution kernels to obtain third down-sampled image data which is 1 / 8 times the original image data, and then continues to perform downsampling on the third down-sampled image data by using 512 convolution kernels to obtain fourth down-sampled image data which is 1 / 16 times the original image data. The target detection method of the present application performs four times of downsampling on the original image data, so that the deep layer has more semantic information, the extracted features have higher discrimination, and different to-be-processed targets such as vehicles can be better distinguished in the subsequent upsampling process and feature extraction process, thereby improving the detection accuracy of the target detection method of the present application.

[0074] With reference to Figure 3 In a first aspect, another embodiment of the present application provides an operation method for performing downsampling on original image data. The operation method comprises the following steps of: performing downsampling on the original image data by using a feature extraction network structure to obtain down-sampled image data.

[0075] Step S310, performing an up-sampling operation on the fourth down-sampling image data through a deformable convolution layer and 256 convolution kernels in the feature extraction network structure, and fusing the third down-sampling image data to obtain first up-sampling image data;

[0076] Step S320, performing an up-sampling operation on the first up-sampling image data through a deformable convolution layer and 128 convolution kernels in the feature extraction network structure, and fusing the second down-sampling image data to obtain second up-sampling image data.

[0077] According to an embodiment of the present application, the feature extraction network structure of the present application uses a deformable convolution layer, performs an up-sampling operation on the fourth down-sampling image data through 256 convolution kernels to obtain first up-sampling image data which is 1 / 8 times of the original image data, and then performs an up-sampling operation on the first up-sampling image data through 128 convolution kernels to obtain second up-sampling image data which is 1 / 4 times of the original image data. The target detection method of the present application performs up-sampling on the fourth down-sampling image data and the first up-sampling image data through the deformable convolution layer, so that the convolution kernel can better adaptively adjust the receptive field, and the positioning of the features is more accurate. Moreover, the first up-sampling image data and the second up-sampling image data obtained by up-sampling are fused with the information of the same resolution layer before down-sampling, which enhances the feature information in the fused feature map. The features in the first up-sampling image data and the second up-sampling image data are enhanced from the aspects of space and channel, which improves the quality of the extracted features and makes the target detection method of the present application more accurate.

[0078] Specifically, the deformable convolution layer in the feature extraction network structure of the present application can use irregular convolution to sample the first up-sampling image data and the second up-sampling image data, and then perform weighted summation on the sampling values. The output of the standard convolution is The deformable convolution adds a 2D offset {Δp n |n=1,...,N} to each grid sampling point on the basis of the standard convolution. For each position p0 in the output feature map f, it is determined by formula (1):

[0079]

[0080] Where p0 is the center node of the grid, and p n is the nth point of the grid. Through deformable convolution, the target detection method of the present application can be performed at irregular and offset positions p n +Δp n Since the offset amount Δp n is usually a fraction, the feature value of formula (1) needs to be realized through bilinear interpolation. The offset amount Δp nThe calculation is determined by equation (2), which is:

[0081] x(p) = ∑ q G(q, p) x(q),

[0082] where p represents any position (p = p0+ p n + Δp n ), q enumerates all spatial positions in the feature map x, and G(·, ·) is a bilinear interpolation kernel.

[0083] It can be understood that the feature extraction network structure of the present application is based on a feature pyramid structure, which can extract corresponding target features in different resolutions, and fuse the feature maps of layers with the same resolution in the downsampling and upsampling processes, so that the feature map output by the feature extraction network structure, i.e. the second upsampling image data, has good features and can provide information-rich feature maps for subsequent classification and regression tasks.

[0084] Referring to Figure 4 , in a first aspect, another embodiment of the present application provides an operation method for performing feature extraction on second upsampling image data, performing feature extraction operation on the second upsampling image data by the feature extraction network structure to obtain feature classification information corresponding to the target to be processed, including but not limited to step S410.

[0085] Step S410, performing feature extraction operation on the second upsampling image data according to one-hot encoding by the feature extraction network structure to obtain feature classification information corresponding to the target to be processed.

[0086] It can be understood that the feature extraction network structure of the present application outputs the second upsampling image data to locate the target to be processed, for example, a vehicle, in order to judge the category of the target to be processed, for example, the category of different vehicles, so as to more accurately distinguish the target to be processed. The present application also needs to perform feature extraction operation on the second upsampling image data in the feature extraction network structure according to one-hot encoding to obtain feature classification information, which facilitates subsequent matching operation. Specifically, one-hot encoding is a representation of a categorical variable as a binary vector. This first requires mapping the classification value to an integer value. Then, each integer value is represented as a binary vector, except for the index of the integer, which is zero, and it is marked as 1.

[0087] It can be understood that the feature extraction network structure outputs the classification branch and the second up-sampling image data, and the second up-sampling image data output by the feature extraction network structure is input into the mask branch structure, and the regression branch is output, wherein the classification branch is to perform feature extraction operation on the second up-sampling image data according to one-hot encoding to obtain feature classification information, and the regression branch is to perform 4 convolution operations with 128 channels on the second up-sampling image data, and when performing the 5th convolution operation, a channel reduction operation is combined to obtain third up-sampling image data with 80 channels, and the third up-sampling image data is subjected to mask processing to obtain the second feature region information corresponding to the target to be processed. In the two branches of classification and regression, the classification loss L cls is a log loss, which is determined by formula (3), and formula (3) is:

[0088]

[0089] wherein m represents the number of rows of the target in the second up-sampling image data, n represents the number of columns of the target in the second up-sampling image data, L cls represents the classification loss, x ij represents the target in the i-th row and the j-th column of the second up-sampling image data, p(x ij ) is the true value of the target, and q(x ij ) is the output target prediction value.

[0090] Referring to Figure 5 , in a first aspect, another embodiment of the present application provides a method for inputting the second up-sampling image data into the mask branch structure, inputting the second up-sampling image data into the mask branch structure to obtain the second feature region information corresponding to the target to be processed, including but not limited to steps S510 and S520.

[0091] Step S510, performing 5 convolution operations on the second up-sampling image data by the mask branch structure, and performing a channel reduction operation in the 5th convolution operation to obtain third up-sampling image data;

[0092] Step S520, performing mask processing on the third up-sampling image data by the mask branch structure to obtain the second feature region information corresponding to the target to be processed.

[0093] It can be understood that the feature extraction network structure of the present application outputs the second up-sampling image data to locate the target to be processed, such as a vehicle. In order to further accurately process each target to be processed in the second up-sampling image data, the second up-sampling image data is input into the mask branch structure. The mask branch structure first performs 4 convolution operations with 128 channels on the second up-sampling image data, and performs a channel reduction operation in the 5th convolution operation to compress the channels, and output the third up-sampling image data with 80 channels. The target detection method of the present application obtains the second feature region information corresponding to the target to be processed in the third up-sampling image data by performing mask processing on the third up-sampling image data.

[0094] It should be noted that the "outputting the third up-sampling image data with 80 channels" described in the present application does not constitute a limitation on the present application. It can be understood that the number of channels of the third up-sampling image data of the present application can be 80, 70, 60, etc. The number of channels is less than 128.

[0095] It can be understood that the regression loss of the present application includes mask loss L mask , box offset and box size loss L box , the mask loss L mask is the average value of binary cross entropy, which is determined by formula (4), and formula (4) is:

[0096]

[0097] Where p n is the true value of the target, q n is the output prediction value. N is the number of all pixels in the mask region.

[0098] Referring to Figure 6 , in a first aspect, another embodiment of the present application provides a method for performing mask processing on the third up-sampling image data, including but not limited to step S610.

[0099] Step S610, performing a shielding operation on the background information in the third up-sampling image data by the mask branch structure to obtain the second feature region information, wherein the second feature region information includes prediction offset information and box size information.

[0100] It can be understood that, in order to further accurately obtain the features of the to-be-processed target such as a vehicle in the third up-sampling image data, it is necessary to reduce the background noise with interference, and therefore the target detection method of the present application also needs to perform a shielding operation on the background information in the third up-sampling image data to obtain second feature region information. The prediction offset information and the box size information are accurately calculated, and the size and position of the bounding box of the to-be-processed target such as a vehicle in the third up-sampling image data are obtained. Among them, the prediction offset and the box size loss L box determined by formula (5), formula (5) is:

[0101]

[0102] where C represents the number of categories, p i is the true value, q i is the predicted value.

[0103] It can be understood that, by performing the matching operation on the first mask region and the second mask region, the target detection data corresponding to the to-be-processed target is obtained, including one of the following:

[0104] The first feature region information of the first mask region and the second feature region information in the second mask region are extracted, the similarity calculation operation is performed on the first feature region information and the second feature region information, the similarity data is obtained, and the target detection data corresponding to the to-be-processed target is obtained according to the similarity data; or,

[0105] The first mask area information of the first mask region and the second mask area information of the second mask region are calculated, the overlap calculation operation is performed on the first mask area information and the second mask area information, the overlap data is obtained, and the target detection data corresponding to the to-be-processed target is obtained according to the overlap data.

[0106] It should be noted that the matching operation is to match the to-be-processed target such as a vehicle in the continuous frame data, and therefore it can also be understood that the matching operation of the present application is based on at least two times of target detection operation, i.e., comparing the second feature region information in the second mask region of the current frame with the first feature region information in the first mask region of the last frame. The present application matches the second feature region information and the first feature region information of the to-be-processed target such as a vehicle extracted by the mask branch structure, calculates the similarity data of the first feature region information and the second feature region information, and obtains the target detection data corresponding to the to-be-processed target by further comparing the similarity data.

[0107] Specifically, the target detection method of the present application compares the similarity data of the first feature region information and the second feature region information with the pre-set similarity data threshold, when the similarity data is greater than the similarity data threshold, the target detection data is matched successfully, that is, the target vehicle is tracked, when the similarity data is less than the similarity data threshold, the target detection data is not matched successfully, the first mask region and the second mask region are subjected to secondary matching, and the process of secondary matching is:

[0108] The second mask area information of the second mask region of the current frame is calculated, and the first mask area information of the first mask region of the last frame is subjected to IoU matching, that is, secondary matching, and the process of secondary matching is to calculate the Euclidean distance between the same target to be processed, such as a vehicle, in two frames combined with the first mask area information and the second mask area information, and then obtain the overlap data of the first mask region and the second mask region to measure the similarity of the first mask region and the second mask region.

[0109] Specifically, the target detection method of the present application compares the overlap data of the first mask region and the second mask region with the pre-set overlap data threshold, when the overlap data is greater than the overlap data threshold, the target detection data is matched successfully, that is, the target vehicle is tracked, when the overlap data is less than the overlap data threshold, the target detection data is not matched, that is, there is no vehicle in the last frame matched with the target vehicle in the current frame. The target detection method of the present application obtains the target vehicle matched between adjacent frames after combination, better solves the problem of vehicle identity error matching between adjacent frames in a crowded scene, increases the matching success rate, and further improves the tracking accuracy based on the target detection method.

[0110] It should be noted that the calculation formula of the overlap data is determined by formula (6), and formula (6) is:

[0111]

[0112] Wherein, IoU(t,t+1) represents the overlap data, DetMask(t) represents the first mask area information, and DetMask(t+1) represents the second mask area information.

[0113] It should be noted that if the feature extraction network structure of the present application does not obtain any feature value of the target vehicle in the original image data after multiple downsampling and upsampling, the matching operation is not performed, and the original image data of the next frame is directly subjected to feature extraction.

[0114] In a second aspect, referring to Figure 7 The embodiment of the present application provides a target detection system, comprising:

[0115] at least one memory 200;

[0116] at least one processor 100;

[0117] at least one program;

[0118] The program is stored in the memory 200, and the processor 100 executes the at least one program to implement:

[0119] The target detection method as any one of the embodiments of the first aspect of the application.

[0120] The processor 100 and the memory 200 can be connected through a bus or other means.

[0121] The memory 200 as a kind of non-transient readable storage medium, it can be used to store non-transient software instructions and non-transient instructions can be instructed.In addition, the memory 200 can include high-speed random access memory 200, it can also include non-transient memory 200, for example, at least one disk storage device 200, flash memory device or other non-transient solid-state storage device 200.It can be understood that the memory 200 can optionally include the memory 200 arranged remotely relative to the processor 100, and these remote memories 200 can be connected to the processor 100 through network.The above-mentioned network includes but is not limited to Internet, intranet, local area network, mobile communication network and combination thereof.

[0122] The processor 100 runs the non-transient software instructions, instructions and signals stored in the memory 200, so as to realize various functional applications and data processing, that is, to realize the target detection method of the above-mentioned first aspect embodiment.

[0123] The non-transient software instructions and instructions required to realize the target detection system of the above-mentioned embodiment are stored in the memory 200, and when executed by the processor 100, the target detection method of the first aspect of the application is executed, for example, the method steps S110 to S170 in the above-mentioned Figure 1 The method steps S210 to S240 in the above-mentioned Figure 2 The method steps S310 to S320 in the above-mentioned Figure 3 The method step S410 in the above-mentioned Figure 4 The method steps S510 to S520 in the above-mentioned Figure 5 The method step S610 in the above-mentioned Figure 6

[0124] In the third aspect, the embodiments of the application provide a computer readable storage medium, and the computer readable storage medium stores computer executable signals, and the computer executable signals are used to execute:

[0125] The target detection method as any one of the embodiments of the first aspect of the application. ​

[0126] For example, execute the above description. Figure 1 Method steps S110 to S170 in the text Figure 2 Method steps S210 to S240 in the text Figure 3 Method steps S310 to S320 in the text Figure 4 Method steps S410, Figure 5 Method steps S510 to S520 and Figure 6 Method step S610.

[0127] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] Based on the above description of the embodiments, those skilled in the art will understand that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable signals, data structures, instruction modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable signals, data structures, instruction modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0129] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A target detection method characterized by, The method comprises: obtaining original image data of a current frame of a target to be processed; inputting the original image data into a feature extraction network structure, performing a downsampling operation on the original image data through the feature extraction network structure to obtain downsampling image data; performing an upsampling operation on the downsampling image data through the feature extraction network structure to obtain upsampling image data, wherein the upsampling image data comprises first upsampling image data and second upsampling image data, and the second upsampling image data is obtained from the first upsampling image data through the upsampling operation; performing a feature extraction operation on the second upsampling image data through the feature extraction network structure to obtain feature classification information corresponding to the target to be processed; inputting the second upsampling image data into a mask branch structure to obtain second feature region information corresponding to the target to be processed; performing a frame selection operation on the target to be processed according to the feature classification information and the second feature region information to obtain a second mask region, and obtaining a first mask region corresponding to a previous frame; performing a matching operation on the first mask region and the second mask region to obtain target detection data corresponding to the target to be processed; the inputting of the second upsampling image data into the mask branch structure to obtain the second feature region information corresponding to the target to be processed comprises: performing five convolution operations on the second upsampling image data through the mask branch structure, and performing a channel reduction operation in the fifth convolution operation to obtain third upsampling image data; performing mask processing on the third upsampling image data through the mask branch structure to obtain second feature region information corresponding to the target to be processed, the second feature region information representing the size and position of the bounding box of the target to be processed in the third upsampling image data; the performing of the matching operation on the first mask region and the second mask region to obtain the target detection data corresponding to the target to be processed comprises: extracting first feature region information of the first mask region and second feature region information in the second mask region, performing a similarity calculation operation on the first feature region information and the second feature region information to obtain similarity data, and obtaining the target detection data corresponding to the target to be processed according to the similarity data; when the similarity data is less than a similarity data threshold, calculating first mask area information of the first mask region and second mask area information of the second mask region, performing an overlap calculation operation on the first mask area information and the second mask area information to obtain overlap data, and obtaining the target detection data corresponding to the target to be processed according to the overlap data.

2. The object detection method of claim 1, wherein, the downsampling image data comprises first downsampling image data, second downsampling image data, third downsampling image data and fourth downsampling image data, and the performing of the downsampling operation on the original image data through the feature extraction network structure to obtain the downsampling image data comprises: The original image data is subjected to a downsampling operation by 64 convolution kernels in the feature extraction network structure to obtain first downsampled image data; The first downsampled image data is subjected to a downsampling operation by 128 convolution kernels in the feature extraction network structure to obtain second downsampled image data; The second downsampled image data is subjected to a downsampling operation by 256 convolution kernels in the feature extraction network structure to obtain third downsampled image data; The third downsampled image data is subjected to a downsampling operation by 512 convolution kernels in the feature extraction network structure to obtain fourth downsampled image data.

3. The object detection method of claim 2, wherein, The upsampling operation on the downsampled image data by the feature extraction network structure comprises: The fourth downsampled image data is subjected to an upsampling operation by a deformable convolution layer and 256 convolution kernels in the feature extraction network structure, and the third downsampled image data is fused to obtain first upsampling image data; The first upsampling image data is subjected to an upsampling operation by a deformable convolution layer and 128 convolution kernels in the feature extraction network structure, and the second downsampled image data is fused to obtain second upsampling image data.

4. The object detection method according to any one of claims 1 to 3, characterized in that, The feature extraction operation on the second upsampling image data by the feature extraction network structure comprises: The feature extraction operation on the second upsampling image data by the feature extraction network structure according to one-hot coding comprises:

5. The object detection method of claim 1, wherein, The mask processing on the third upsampling image data by the mask branch structure comprises: The mask processing on the third upsampling image data by the mask branch structure comprises:

6. The object detection method of claim 1, wherein, The formula for calculating the overlap degree data is as follows: ， Among them, the Indicates the overlap data, and the Indicates the first mask area information, and the Indicates the second mask area information.

7. A target detection system characterized by, The computer readable storage medium stores computer executable signals, and the computer executable signals are used to execute: The computer readable storage medium stores computer executable signals, and the computer executable signals are used to execute: The computer readable storage medium stores computer executable signals, and the computer executable signals are used to execute: ​ ​ ​ 8. A computer-readable storage medium, characterized in that, ​ ​

Images