A target detection method, apparatus, equipment and medium

Abstract

Some embodiments of this application provide a target detection method, apparatus, device, and medium. In this method, before inputting each cropped image into a pre-trained first feature extraction network model for feature extraction, the image to be detected is divided into a preset number of cropped images according to a preset shape. This effectively avoids the problem of the feature extraction layer and spatial pyramid pooling layer losing image detail information during scaling in the target detection process. In the subsequent target detection process, the fully connected layer determines the weight matrix of the image to be detected based on the attention mechanism, and assigns more weights to the region containing the target according to the weight matrix and the first image feature vector, thereby improving the accuracy and recall rate for small target detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a target detection method, apparatus, device, and medium. Background Technology

[0002] Small object detection in panoramic images is a key and challenging area of ​​image processing technology. Current methods aim to accurately detect objects with extremely small visual features within an image; these are typically smaller than 32x32 pixels. Furthermore, if the panoramic image has a large depth of field, the object occupies an even smaller proportion of the frame. Small objects lack sufficient appearance information, making them difficult to distinguish from the background or similar objects. In addition, real-world scenes are complex, often involving drastic changes in lighting, object occlusion, densely packed objects, and variations in object scale. These factors have a more pronounced impact on the features of small objects, further increasing the difficulty of small object detection.

[0003] Therefore, how to accurately detect small targets has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides a target detection method, apparatus, device, and medium to solve the problem of inaccurate detection of small targets.

[0005] Firstly, this application provides a target detection method, the method comprising:

[0006] The image to be detected is divided into a preset number of cropped images according to a preset shape;

[0007] Each cropped image is input into a pre-trained first feature extraction network model. The feature extraction layer of the first feature extraction network model extracts a feature image of each cropped image and inputs the feature image into a spatial pyramid pooling layer. The spatial pyramid pooling layer obtains a pooled first image feature vector and inputs the first image feature vector into a fully connected layer. The fully connected layer determines the weight matrix of the image to be detected based on an attention mechanism, and determines and outputs the detected first target based on the weight matrix and the first image feature vector.

[0008] Secondly, this application also provides a target detection device, the device comprising:

[0009] The cropping module is used to cut the image to be detected into a preset number of cropped images according to a preset shape;

[0010] The detection module is used to input each of the cropped images into a pre-trained first feature extraction network model. The feature extraction layer of the first feature extraction network model extracts feature images of each cropped image and inputs the feature images into a spatial pyramid pooling layer. The spatial pyramid pooling layer obtains a pooled first image feature vector and inputs the first image feature vector into a fully connected layer. The fully connected layer determines the weight matrix of the image to be detected based on an attention mechanism, and determines and outputs the detected first target based on the weight matrix and the first image feature vector.

[0011] Thirdly, this application also provides an electronic device, which includes at least a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of any of the target detection methods described above.

[0012] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the target detection methods described above.

[0013] This application provides a target detection method, apparatus, device, and medium. In this method, the image to be detected is divided into a preset number of cropped images according to a preset shape. Each cropped image is input into a pre-trained first feature extraction network model. The feature extraction layer of the first feature extraction network model extracts the feature image of each cropped image and inputs the feature image into a spatial pyramid pooling layer. The spatial pyramid pooling layer obtains the pooled first image feature vector and inputs the first image feature vector into a fully connected layer. The fully connected layer determines the weight matrix of the image to be detected based on an attention mechanism, and determines and outputs the detected first target based on the weight matrix and the first image feature vector. In this embodiment, before each cropped image is input into the pre-trained first feature extraction network model for feature extraction, the image to be detected is divided into a preset number of cropped images according to a preset shape. This effectively avoids the problem of the feature extraction layer and spatial pyramid pooling layer losing image detail information during scaling in the target detection process. In the subsequent target detection process, the fully connected layer determines the weight matrix of the image to be detected based on the attention mechanism, and assigns more weights to the region containing the target according to the weight matrix and the first image feature vector, thereby improving the accuracy and recall rate for small target detection. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the target detection process provided in an embodiment of this application;

[0016] Figure 2 This is a schematic diagram of target detection provided for an embodiment of this application;

[0017] Figure 3 This is a schematic diagram of the fourth target detection process provided in an embodiment of this application;

[0018] Figure 4 This is a schematic diagram of the fourth target detected in an embodiment of this application;

[0019] Figure 5 A schematic diagram of the target detection process provided for the implementation of this application;

[0020] Figure 6 This is a schematic diagram of the target detection device provided in the embodiments of this application;

[0021] Figure 7 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art are within the scope of protection of this application.

[0023] This application provides a target detection method, apparatus, device, and medium. In this method, the image to be detected is divided into a preset number of cropped images according to a preset shape. Each cropped image is input into a pre-trained first feature extraction network model. The feature extraction layer of the first feature extraction network model extracts the feature image of each cropped image and inputs the feature image into a spatial pyramid pooling layer. The spatial pyramid pooling layer obtains the pooled first image feature vector and inputs the first image feature vector into a fully connected layer. The fully connected layer determines the weight matrix of the image to be detected based on an attention mechanism, and determines and outputs the detected first target based on the weight matrix and the first image feature vector. In this embodiment, before each cropped image is input into the pre-trained first feature extraction network model for feature extraction, the image to be detected is divided into a preset number of cropped images according to a preset shape. This effectively avoids the problem of the feature extraction layer and spatial pyramid pooling layer losing image detail information during scaling in the target detection process. In the subsequent target detection process, the fully connected layer determines the weight matrix of the image to be detected based on the attention mechanism, and assigns more weights to the region containing the target according to the weight matrix and the first image feature vector, thereby improving the accuracy and recall rate for small target detection.

[0024] Example 1:

[0025] Figure 1 This is a schematic diagram of the target detection process provided in an embodiment of this application. The process specifically includes the following steps:

[0026] S101: The image to be detected is divided into a preset number of cropped images according to a preset shape.

[0027] The target detection process provided in this application embodiment is applicable to electronic devices, such as servers, PCs, image acquisition devices, etc.

[0028] To avoid losing detailed image information during target detection, this embodiment preprocesses the image to be detected by dividing it into a preset number of cropped images according to a preset shape. The subsequent target detection process then performs detection on each cropped image. The image to be detected can be an image input by the user of the electronic device, an image captured by a smart camera, or an image from a frame in a continuous panoramic video captured by a panoramic camera mounted at a high point. The preset shape and preset number can be pre-configured by the user of the electronic device. The preset shape can be a rectangle, and the preset number can be any square of an integer such as 4, 9, or 16. Specifically, the image to be detected can be cropped into 9 rectangular cropped images.

[0029] S102: Each cropped image is input into a pre-trained first feature extraction network model. The feature extraction layer of the first feature extraction network model extracts the feature image of each cropped image and inputs the feature image into a spatial pyramid pooling layer. The spatial pyramid pooling layer obtains the pooled first image feature vector and inputs the first image feature vector into a fully connected layer. The fully connected layer determines the weight matrix of the image to be detected based on an attention mechanism, and determines and outputs the detected first target based on the weight matrix and the first image feature vector.

[0030] To accurately identify small targets in panoramic images with large depth of field and small target proportions, while suppressing false alarms, this embodiment of the application inputs each cropped image into a pre-trained first feature extraction network model. The feature extraction layer of this first feature extraction network model extracts the feature image of each cropped image. During feature image extraction, the feature extraction layer of the first feature extraction network model can be a ResNet-50. Using ResNet-50 to extract the feature image of each input cropped image effectively avoids the gradient vanishing problem during downsampling.

[0031] In this embodiment, the feature extraction layer of the first feature extraction network model can input the feature image of each cropped image into the Spatial Pyramid Pooling (SPP) layer to obtain the local and global text information of each cropped image. The Spatial Pyramid Pooling layer fuses the feature image and text information of each cropped image to obtain the pooled first image feature vector.

[0032] In this embodiment, the spatial pyramid pooling layer can input the first image feature vector into the fully connected layer. The fully connected layer determines the weight matrix of the image to be detected based on an attention mechanism. The attention mechanism can assign more weights to regions containing the target and fewer weights to regions not containing the target. In other words, in this embodiment, an attention mechanism is introduced to obtain the weight matrix by using a fully connected network and an activation function on the pooled first image feature vector. The attention mechanism can include channel attention or pixel attention.

[0033] After determining the weight matrix, it can be multiplied by the corresponding first image feature vector to obtain the enhanced image feature vector. This enhanced image feature vector is then displayed as an image, resulting in the enhanced feature image. Since the attention mechanism assigns more weight to regions containing the target and fewer weight to regions not containing the target, the highlighted portion in the enhanced feature image represents the detected first target.

[0034] In this embodiment, before each cropped image is input into the pre-trained first feature extraction network model for feature extraction, the image to be detected is divided into a preset number of cropped images according to a preset shape. This effectively avoids the problem of the feature extraction layer and spatial pyramid pooling layer losing image detail information during scaling in the target detection process. In the subsequent target detection process, the fully connected layer determines the weight matrix of the image to be detected based on the attention mechanism, and assigns more weights to the region containing the target according to the weight matrix and the first image feature vector, thereby improving the accuracy and recall rate for small target detection.

[0035] Example 2:

[0036] To further improve the accuracy and recall of target detection, based on the above embodiments, in this embodiment, determining the detected first target according to the weight matrix and the first image feature vector includes:

[0037] The first weight matrix of the image to be detected, determined based on the channel attention mechanism, is multiplied by the first image feature vector to obtain the second image feature vector; the second weight matrix of the image to be detected, determined based on the pixel attention mechanism, is multiplied by the second image feature vector to obtain the detected first target and output it.

[0038] In this embodiment, the fully connected layer can obtain a first weight matrix of the image to be detected based on a channel attention mechanism. This first weight matrix assigns more weights to the channels containing the target. Multiplying the first weight matrix by the first image feature vector yields a weighted enhanced image feature vector, which is the second image feature vector. In this embodiment, a cropped image can be considered as one channel, so n cropped images constitute n channels. The channel attention mechanism can assign more weights to cropped images containing the target.

[0039] To further improve the accuracy and recall of target detection, in this embodiment, the fully connected layer can also obtain a second weight matrix of the image to be detected based on a pixel attention mechanism. This second weight matrix assigns more weight to the region containing the target in the first image feature vector. Multiplying the second weight matrix by the second image feature vector obtained based on the channel attention mechanism yields a further enhanced feature vector. Displaying this enhanced feature vector using a visualization tool reveals the detected first target. In this embodiment, the first target can be labeled in the image to be detected for easy viewing by users of electronic devices. The pixel attention mechanism assigns more weight to pixels on the target, thus assigning more weight to the region containing the target. A region can be understood as an area composed of multiple pixels.

[0040] Specifically, Figure 2 This is a schematic diagram of target detection provided for an embodiment of this application, such as... Figure 2 As shown, the image to be detected is cropped into nine rectangular cropped images. Each cropped image is input into a ResNet-50 processor, which extracts feature images from each cropped image. These feature images are then input into the SPP layer to obtain local and global text information for each cropped image. The feature images and text information of each cropped image are then fused to obtain a pooled first image feature vector. This first image feature vector is input into a fully connected layer. The fully connected layer determines a first weight matrix for the image to be detected based on a channel attention mechanism and a second weight matrix based on a pixel attention mechanism. Many methods for determining weight matrices based on channel or pixel attention mechanisms exist in related technologies and will not be elaborated upon in this embodiment. After determining the first and second weight matrices, the fully connected layer multiplies the first weight matrix by the first image feature vector to obtain the second image feature vector. This second image feature vector is then multiplied by the second weight matrix to obtain the detected first target, which is then output.

[0041] In this embodiment, an attention mechanism is introduced, which assigns more weights to channels and regions containing targets through two dimensions: channel attention mechanism and pixel attention mechanism. On the one hand, this increases the recall rate of small targets in the panoramic image, and on the other hand, it enables model pruning of channels with low relevance to target detection results, thereby improving the frame rate and meeting the real-time requirements of the algorithm.

[0042] Example 3:

[0043] To improve the fault tolerance and accuracy of target detection, based on the above embodiments, in this embodiment, if the image to be detected is any video frame image in a video, the method further includes:

[0044] For each pixel in the image to be detected, the difference between the pixel value of the pixel and the pixel values ​​of each of the second preset number of pixels adjacent to the pixel stored in the historical pixel sample set is calculated; a first number of the difference values ​​are greater than or equal to a first preset value and less than or equal to a second preset value are counted; if the first number is greater than a first preset number threshold, the pixel is determined to be a background pixel; a second number of the difference values ​​are less than the first preset value or greater than the second preset value are counted; if the second number is greater than a second preset number threshold, the pixel is determined to be a foreground pixel, and the foreground pixel is determined as a pixel on the detected second target;

[0045] The first target and the second target in the image to be detected are labeled and output.

[0046] Introducing an attention mechanism during object detection can better ensure the recall rate for small object detection. However, there will still be cases of missed detection. Therefore, in order to improve the fault tolerance rate and enhance the detection accuracy of object detection, in this embodiment, when the image to be detected is any video frame image in a video, the background of the image to be detected can be extracted. Specifically, the video frame position corresponding to the image to be detected in the video can be obtained. By comparing the pixel value of a certain pixel at the current video frame position with the historical pixel value of that pixel, it can be determined whether the pixel belongs to the background, i.e., the background pixel.

[0047] In this embodiment, for each pixel in the image to be detected, the difference between the pixel value of that pixel and the pixel values ​​of each of the second preset number of pixels adjacent to that pixel stored in the historical pixel sample set can be calculated. The historical pixel sample set stores the pixel value of each pixel and the pixel values ​​of the second preset number of pixels adjacent to that pixel, where the second preset number can be any number such as 20, 30, or 50.

[0048] Specifically, V(x, y) represents the pixel value of the current pixel 1, and M(x, y) = {V1(x, y), ..., Vn(x, y)} represents the set of historical pixel samples containing the pixel values ​​of n pixels adjacent to pixel 1, where n can be 50, thus increasing adaptability to complex scenes while ensuring memory efficiency. In this embodiment, the differences between V(x, y) and V1, V2, ..., Vn can be calculated separately.

[0049] In this embodiment, after calculating the difference between the pixel value of the current pixel and the pixel values ​​of a second preset number of pixels, a first number of pixels with a difference greater than a first preset value and less than a second preset value can be counted. If the first number is greater than a first preset number threshold, the pixel is determined to be a background pixel. The first and second preset values ​​can be the allowable range of fluctuation in the pixel value difference. In this embodiment, the first and second preset values ​​can be opposites of each other; that is, if the first preset value is R, then the second preset value is -R. The first preset number threshold can be a value less than the second preset number. For example, if the second preset number is 50, then the first preset number threshold can be 43.

[0050] Specifically, we can count the number N of differences N between V(x, y) and V1, V2, ..., Vn, within the allowable fluctuation range ±R, where R can be 20. Assuming the first preset threshold th is 43, if N > th, it means that the pixel is similar to a historical pixel and can be identified as a background pixel.

[0051] In this embodiment of the application, a second number of pixels with a difference less than a first preset value or greater than a second preset value can be counted. If the second number is greater than a second preset number threshold, the pixel is determined to be a foreground pixel. In this embodiment of the application, the foreground pixel can be determined as a pixel on the detected second target. The second preset number threshold can be the difference between the second preset number and the first preset number threshold.

[0052] Specifically, assuming the second preset quantity n is 50 and the first preset quantity threshold th is 43, then the second preset quantity threshold T is n-th, i.e., T is 7. If the number N1 of the differences between V(x, y) and V1, V2...Vn respectively are greater than R or less than -R is greater than T, then the pixel is determined as a foreground pixel.

[0053] After determining whether each pixel is a foreground pixel, i.e., after identifying the second target, the first and second targets in the image to be detected can be labeled and output. In other words, the first and second targets can be displayed on the same image, specifically highlighted using bounding boxes. Alternatively, after identifying the first and second targets, the pixel position of each detected first and second target can be obtained and saved for further processing by the electronic device based on these saved pixel positions.

[0054] In this embodiment, moving target detection is added based on the target detection in the above embodiments. The background modeling method effectively extracts the second target in the moving foreground, avoiding the problem that relying solely on deep learning for target detection leads to low model detection rate and false positives / false negatives when the panoramic depth of field is large, the target is small, or the image is noisy. Furthermore, by fusing the first and second targets, this embodiment reduces the false positive and false negative rates in complex scenes, and can more effectively handle panoramic small target monitoring in various conditions such as day and night, rain, snow, and fog.

[0055] To further improve the accuracy of target detection, based on the above embodiments, in this embodiment of the application, before annotating and outputting the first target and the second target in the image to be detected, the method further includes:

[0056] For each determined foreground pixel, it is determined whether the foreground pixel is determined as a foreground pixel in a consecutive second preset number of video frame images. If so, the foreground pixel is set as a background pixel.

[0057] After determining whether each pixel in the image to be detected is a background pixel or a foreground pixel, in order to further improve the accuracy of target detection, for each determined foreground pixel, it can be determined whether the foreground pixel has been identified as a foreground pixel in a continuous second preset number of video frames. If so, it means that the pixel has not moved for a long time, and the foreground pixel can be set as a background pixel. The second preset number can be set by those skilled in the art as needed.

[0058] Specifically, in this embodiment, the second preset number can be 20. After determining whether each pixel in the image to be detected is a background pixel or a foreground pixel, for each foreground pixel, the images of the second preset number of video frames adjacent to the image to be detected can be obtained, and it can be determined whether the pixel corresponding to the foreground pixel is a foreground pixel in the adjacent images. If the pixel corresponding to the foreground pixel is determined to be a foreground pixel in 20 consecutive video frames adjacent to the image to be detected, then it is set as a background pixel.

[0059] To further improve the fault tolerance and enhance the detection accuracy of target detection, based on the above embodiments, in this embodiment, the process of determining the historical pixel sample set includes:

[0060] Obtain images of preset video frames from the video, and for each pixel in the image, save the pixel value of that pixel and the second preset number of pixels adjacent to that pixel to the historical pixel sample set.

[0061] In this embodiment, the historical pixel sample set can be determined by selecting any frame from the video. Specifically, a preset video frame can be obtained, where the preset video frame may be the first frame of the video. For each pixel in the image, the pixel value of the pixel and a second preset number of adjacent pixels are saved into the historical pixel sample set, thereby determining the historical pixel sample set.

[0062] Specifically, assuming the second preset quantity is 50, when determining the historical pixel sample set, 50 pixels around each pixel can be selected, and the pixel value of each pixel can be recorded respectively. This reduces the amount of computation while storing the spatiotemporal information of the video.

[0063] To further improve the accuracy of target detection, based on the above embodiments, the method in this application embodiment further includes:

[0064] For each determined background pixel, the pixel value of that background pixel is used to update the pixel value of the corresponding pixel stored in the historical pixel sample set according to a preset update probability.

[0065] Since the historical pixel sample set is determined based on the historical video frames, other objects or weather may change over time. Therefore, in order to further improve the accuracy of target detection, in this embodiment, the historical pixel sample set can be updated at a preset time interval or at a preset update probability.

[0066] In this embodiment of the application, for each determined background pixel, the pixel value of the background pixel can be used to update the pixel value of the corresponding pixel stored in the historical pixel sample set according to a preset update probability.

[0067] Specifically, assuming a preset update probability Where 'l' represents the time sampling factor, and in this embodiment, its value can be set to 16. Therefore, in this embodiment, each defined background pixel has... The probability is used to update the pixel value of the pixel stored in the historical pixel sample set, and the same applies to the pixels adjacent to the pixel. The probability is used to update the pixel value of the corresponding pixel in the historical pixel sample set.

[0068] In this embodiment, the update strategy for the historical pixel sample set utilizes the spatial propagation characteristics of pixel values, which can effectively avoid the generation of Ghost regions. Furthermore, since the pixel value of one pixel is updated randomly, the update of the historical pixel sample set during the target detection process is independent of time, which is quite reasonable.

[0069] Example 4:

[0070] To further improve the accuracy of target detection, based on the above embodiments, the method in this application embodiment further includes:

[0071] For each video frame in the video, obtain each third target detected in the images of a third preset number of video frames adjacent to that video frame, wherein the third target includes each of the first target and the second target detected;

[0072] For each video frame image of the acquired third preset number, the maximum motion gradient of the third target is determined based on the third target detected in the video frame image and the vector velocity of the center pixel of the third target detected in the video frame image adjacent to the video frame in a preset direction.

[0073] The pixel points of potential targets are determined based on the maximum motion gradient of each third target and the center pixel point of each third target.

[0074] Considering the correlation between video frames, in order to make reasonable use of the temporal information of the video and further improve the accuracy of target detection, in this embodiment of the application, for each video frame in the video, the forward and reverse optical flow of each third target can be determined by using the third targets detected in other video frames adjacent to that video frame.

[0075] In this embodiment, for each video frame in the video, each detected third target can be acquired from the images of a third preset number of video frames adjacent to that video frame. The third preset number can be any number, and the third target includes each detected first target and second target. Assuming the third preset number is 2, for each video frame in the video, the images of the preceding and following video frames can be acquired. For the images corresponding to these three consecutive video frames, the forward and reverse optical flow is calculated using the center pixel information of the detected third target in each video frame.

[0076] Specifically, in this embodiment, for each of the acquired third preset number of video frames, the maximum motion gradient of the third target can be determined based on the detected third target in the video frame and the vector velocity of the center pixel of the third target detected in the adjacent video frames in a preset direction. In this embodiment, a coordinate system can be created with the center pixel of the third target as the origin, and the preset direction is the direction of the x-axis and the direction of the y-axis.

[0077] Specifically, in this embodiment, when calculating the forward and backward optical flow of video frame 1, the identifiers of the third preset number of video frames are 0, 1, and 2, respectively. Since the video is captured by an image acquisition device fixed at a certain position, the acquisition range of each frame in the video is generally consistent. In this embodiment, the maximum motion gradient G of each third target object between video frame 0 and video frame 1 can be calculated respectively. 0～1 The maximum motion gradient G of each third target object between video frame 1 and video frame 2. 1～2 The maximum motion gradient G of each third target object between video frame 1 and video frame 0. 1～0 The maximum motion gradient G of each third target object between video frame 2 and video frame 1. 2～1 The method for calculating the maximum motion gradient is as follows: Where u x and v x These represent the vector velocities of the center pixel of the third target object along the x-axis and y-axis, respectively.

[0078] Where u x and v x The target is determined based on the motion displacement of the same third target in two adjacent video frames. In this embodiment, for each third target in the previous video frame, the distance from which the third target moves to the position of each third target in the next video frame can be calculated. If the distance is less than a preset distance threshold, the third targets in the two video frames corresponding to that distance can be identified as the same target. With the starting and ending positions of the third target known, the displacement of the third target from the previous video frame to the next video frame can be determined, as well as its vector velocity along the x and y axes. In this embodiment, after identifying which two third targets in two video frames are the same target, to further improve the accuracy of target detection, the image features of the two third targets that are the same target can be acquired and compared. If the image features of the two third targets are similar, then the two third targets can be identified as the same target.

[0079] After determining the maximum motion gradient of the third target, in this embodiment of the application, the maximum motion gradient of the optical flow of the third target between a third preset number of video frames can be determined based on each maximum motion gradient determined for the same third target. That is, the largest maximum motion gradient is selected from each determined maximum motion gradient as the target maximum motion gradient. Then, based on the target maximum motion gradient and the position of the center pixel of the third target in the image of the video frame, the pixel of the potential target is determined. In other words, based on the pixel position of the third target actually acquired in the video, the maximum maximum motion gradient of the third target between video frames is determined. Then, based on the maximum maximum motion gradient, the position where the third target may appear in the next video frame is predicted. The predicted pixel position is the pixel of the potential target.

[0080] Specifically, after determining the maximum motion gradient G between three video frames... 0～1 G 1～2 G 1～0 G 2～1 Then, the largest maximum motion gradient can be selected as the target maximum motion gradient M, where M can be expressed as M = max(G 0～1 G 1～2 G 2～1 G 1～0 The maximum motion gradient M represents the maximum motion gradient of the optical flow across the three video frames. Once the maximum motion gradient is determined, the pixel location of the third target in video frame 1 is also known, thus allowing the determination of the pixel location of potential targets.

[0081] The method described above, which uses information from the center pixels of the third target detected in a third preset number of video frames to calculate the forward and backward optical flow, can be called the optical flow method. Optionally, using the optical flow method to determine the pixels of potential targets may still miss some pixels of potential targets, thus affecting the accuracy of target detection. Therefore, in this embodiment, after determining the pixels of potential targets, a conditional random field can be used to enhance the correlation between adjacent pixels and expand the area where potential targets are determined. How to enhance the correlation between adjacent pixels based on the conditional random field has been described in detail in related technologies and will not be repeated in this embodiment.

[0082] To further improve the accuracy of target detection, based on the above embodiments, the method in this application embodiment further includes:

[0083] For each pixel of the potential target, a small image is obtained by cropping according to a first preset pixel size standard, with the pixel of the potential target as the center; the small image corresponding to the same third target in a fourth preset number of video frames is selected to obtain a first small image set; and a second small image set with a second preset pixel size is obtained based on bilinear interpolation and the first small image set.

[0084] The second set of small images and the maximum motion gradient of the third target are input into a pre-trained second feature extraction network model to obtain the feature image extracted by the feature extraction layer of the second feature extraction network model; the feature image is input into a spatial pyramid pooling layer, which obtains the pooled third image feature vector; the third image feature vector is input into a fully connected layer, which determines the weight matrix of the feature image based on an attention mechanism, and determines and outputs the detected fourth target based on the weight matrix and the third image feature vector.

[0085] After identifying the pixels of potential targets for the third target in each video frame, a small image can be cropped from each potential target pixel according to a first preset pixel size standard. This first preset pixel size standard can be any preset pixel size, but preferably 100*100 pixels. In other words, a small image is obtained by cropping from each potential target pixel using a 100*100 pixel standard.

[0086] After obtaining the pixel-level matting of each potential target, the first set of small images can be obtained by selecting small images corresponding to the same third target from a fourth preset number of video frames. The fourth preset number can be any number such as 4, 8, 10, or 15; preferably, it can be 8. Specifically, in this embodiment, small images corresponding to the same third target from eight consecutive video frames can be selected to form the first set of small images, which can be identified as 100*100*8.

[0087] After obtaining the first set of small images, bilinear interpolation can be used to convert the small images in the first set with a first preset pixel size standard into small images with a second preset pixel size, thus obtaining the second set of small images. Specifically, in this embodiment, bilinear interpolation can be used to convert the 100*100 pixel small images in the first set into 224*224 small images, thereby obtaining the second set of small images with a size of 224*224*8.

[0088] In this embodiment, the second set of small images and the maximum motion gradient of the corresponding third target in the second set of small images can be input into a pre-trained second feature extraction network model to obtain the feature image extracted by the feature extraction layer of the second feature extraction network model. The second feature extraction network model can use the I3D (Inflated 3D ConvNet) model. The advantage of choosing the I3D network model is mainly reflected in its backbone, which uses a pre-trained GoogleNet as a two-stream network. By performing three-dimensional convolution on RGB two-dimensional image information and optical flow respectively and then fusing them, it has a fast running speed, low memory overhead, and achieves state-of-the-art performance.

[0089] To further improve the accuracy of object detection, the I3D network model can be improved in this embodiment, and the improved I3D network model can be used for feature extraction. GoogleNet in the I3D network model can be replaced with Inception-v4. The improved I3D network model can decompose 5*5 and n*n convolutions to reduce the number of parameters, and introduce residual connections to avoid gradient vanishing and reduce model complexity.

[0090] After extracting the feature image using the improved I3D network model, the feature image can be input again into the spatial pyramid pooling layer. The spatial pyramid pooling layer obtains the pooled third image feature vector, and then inputs the third image feature vector into the fully connected layer. The fully connected layer determines the weight matrix of the feature image based on the attention mechanism, and determines and outputs the detected fourth target based on the weight matrix and the third image feature vector. The process of inputting the feature image into the spatial pyramid pooling layer and determining and outputting the detected fourth target is similar to the process of determining and outputting the detected first target mentioned in the above embodiments, and will not be repeated in this embodiment.

[0091] In this embodiment, the target detection results of multiple consecutive frames are fused and correlated with the motion detection results. After combining optical flow information with the two-dimensional image, the three-dimensional convolution can further expand the candidate detection area of ​​small targets in the panoramic image, effectively filter out false alarm targets in the preliminary detection results and detect undetected targets, thereby improving the robustness of the model.

[0092] The target detection process provided in this application will be described below with reference to a specific embodiment. Figure 3 This is a schematic diagram of the fourth target detection process provided in the embodiments of this application, such as... Figure 3As shown, a first set of small images is generated by selecting eight consecutive frames. The pixels of the potential target are determined based on optical flow. A second set of small images of 224*224*8 is obtained based on bilinear interpolation. The second set of small images is then input into the improved I3D network model to obtain each feature image extracted by the feature extraction layer of the I3D network model. The obtained feature images are then input into SPP to obtain the pooled third image feature vector. The third image feature vector is then input into the fully connected layer. The fully connected layer multiplies the third image feature vector with the first weight matrix determined based on the channel attention mechanism to obtain the third image feature vector. Finally, the third image feature vector is multiplied with the second weight matrix determined based on the pixel attention mechanism to obtain the detected fourth target and output it.

[0093] Figure 4 This is a schematic diagram of the fourth target detected in an embodiment of this application, as shown below. Figure 4 The numbers 1 and 2 marked in the figure represent the fourth detected target.

[0094] Example 5:

[0095] The target detection process is described below with reference to a specific embodiment. Figure 5 A schematic diagram of the target detection process provided for the implementation of this application is provided. The process includes the following steps:

[0096] S501: Acquire panoramic video of the required control scene from a panoramic camera mounted at a high point.

[0097] S502: Each video frame in the panoramic video is taken as the image to be detected. Based on the first feature extraction network model, the spatial pyramid pooling layer and the attention mechanism, the first target is determined to be detected.

[0098] In this embodiment, the object detection module is responsible for performing the operation of determining the detection of a first target based on a first feature extraction network model, a spatial pyramid pooling layer, and an attention mechanism. That is, after acquiring the panoramic video, the object detection module determines a preliminary detection result, which is the detected first target.

[0099] S503: For each pixel in the image to be detected, determine the detected second target based on the pixel value of that pixel and the historical pixel sample set.

[0100] In this embodiment, the motion detection module is responsible for determining the detected second target based on the pixel values ​​of the pixels and a historical pixel sample set. In other words, after acquiring the panoramic video, the motion detection module detects the second target in motion.

[0101] S504: Based on the third target detected in each image to be detected, determine the pixels of the potential target, obtain the second set of small images based on the pixels of the potential target, and determine the detected fourth target based on the second feature extraction network model, the spatial pyramid pooling layer and the attention mechanism.

[0102] In this embodiment, the trajectory tracking module is responsible for performing the following operations: determining the pixels of potential targets based on the third target detected in each video frame; obtaining a second set of small images based on the pixels of potential targets; and determining the detected fourth target based on a second feature extraction network model, a spatial pyramid pooling layer, and an attention mechanism. In other words, the trajectory tracking module can determine potential targets and perform secondary target detection to obtain the final panoramic target detection result, where the final panoramic target detection result is the detected fourth target.

[0103] Example 6:

[0104] Figure 6 This is a schematic diagram of the target detection device provided in the embodiments of this application, such as... Figure 6 As shown, the device includes:

[0105] The cropping module 601 is used to cut the image to be detected into a preset number of cropped images according to a preset shape;

[0106] The detection module 602 is used to input each of the cropped images into a pre-trained first feature extraction network model. The feature extraction layer of the first feature extraction network model extracts the feature image of each cropped image and inputs the feature image into a spatial pyramid pooling layer. The spatial pyramid pooling layer obtains the pooled first image feature vector and inputs the first image feature vector into a fully connected layer. The fully connected layer determines the weight matrix of the image to be detected based on an attention mechanism, and determines and outputs the detected first target based on the weight matrix and the first image feature vector.

[0107] In one possible implementation, the detection module 602 is specifically used to multiply the first weight matrix of the image to be detected, determined based on the channel attention mechanism, with the first image feature vector to obtain a second image feature vector; and to multiply the second weight matrix of the image to be detected, determined based on the pixel attention mechanism, with the second image feature vector to obtain the detected first target and output it.

[0108] In one possible implementation, the detection module 602 is further configured to, if the image to be detected is any video frame image in a video, calculate, for each pixel in the image to be detected, the difference between the pixel value of the pixel and the pixel values ​​of each of a second preset number of pixels adjacent to the pixel stored in the historical pixel sample set; count a first number of differences greater than or equal to a first preset value and less than or equal to a second preset value; if the first number is greater than a first preset number threshold, then determine the pixel as a background pixel; count a second number of differences less than the first preset value or greater than the second preset value; if the second number is greater than a second preset number threshold, then determine the pixel as a foreground pixel, and identify the foreground pixel as a pixel on the detected second target;

[0109] The first target and the second target in the image to be detected are labeled and output.

[0110] In one possible implementation, the detection module 602 is further configured to determine, for each determined foreground pixel, whether the foreground pixel is determined as a foreground pixel in a consecutive second preset number of video frame images; if so, then set the foreground pixel as a background pixel.

[0111] In one possible implementation, the detection module 602 is specifically used to acquire an image at a preset video frame position in the video, and for each pixel in the image, save the pixel value of the pixel and the second preset number of pixels adjacent to the pixel in the image to the historical pixel sample set.

[0112] In one possible implementation, the detection module 602 is specifically used to update the pixel value of the corresponding pixel stored in the historical pixel sample set for each determined background pixel using the pixel value of the background pixel according to a preset update probability.

[0113] In one possible implementation, the detection module 602 is specifically configured to, for each video frame in the video, acquire each detected third target in the images of a third preset number of video frames adjacent to that video frame, wherein the third target includes each detected first target and second target; for each acquired third preset number of video frames, determine the maximum motion gradient of the third target based on the vector velocity of the center pixel of the third target detected in the image of that video frame and the image of the third target detected in the images of the adjacent video frames in a preset direction; and determine the pixel of a potential target based on the maximum motion gradient of each third target and the center pixel of each third target.

[0114] In one possible implementation, the detection module 602 is further configured to: for each pixel of the potential target, crop a small image based on the pixel of the potential target according to a first preset pixel size standard; select the small image corresponding to the same third target in a fourth preset number of video frames to obtain a first small image set; and obtain a second small image set of a second preset pixel size based on bilinear interpolation and the first small image set; input the second small image set and the maximum motion gradient of the third target into a pre-trained second feature extraction network model to obtain a feature image extracted by the feature extraction layer of the second feature extraction network model; input the feature image into a spatial pyramid pooling layer, which obtains a pooled third image feature vector; input the third image feature vector into a fully connected layer, which determines the weight matrix of the feature image based on an attention mechanism, and determines and outputs the detected fourth target based on the weight matrix and the third image feature vector.

[0115] Example 7:

[0116] Figure 7 This application provides a schematic diagram of an electronic device structure as an embodiment of the present application. Based on the above embodiments, the present application also provides an electronic device, such as... Figure 7 As shown, it includes: processor 701, communication interface 702, memory 703 and communication bus 704, wherein processor 701, communication interface 702 and memory 703 communicate with each other through communication bus 704.

[0117] The memory 703 stores a computer program, which, when executed by the processor 701, causes the processor 701 to perform the following steps:

[0118] The image to be detected is divided into a preset number of cropped images according to a preset shape;

[0119] Each cropped image is input into a pre-trained first feature extraction network model. The feature extraction layer of the first feature extraction network model extracts a feature image of each cropped image and inputs the feature image into a spatial pyramid pooling layer. The spatial pyramid pooling layer obtains a pooled first image feature vector and inputs the first image feature vector into a fully connected layer. The fully connected layer determines the weight matrix of the image to be detected based on an attention mechanism, and determines and outputs the detected first target based on the weight matrix and the first image feature vector.

[0120] In one possible implementation, determining the detected first target based on the weight matrix and the first image feature vector includes:

[0121] The first weight matrix of the image to be detected, determined based on the channel attention mechanism, is multiplied by the first image feature vector to obtain the second image feature vector; the second weight matrix of the image to be detected, determined based on the pixel attention mechanism, is multiplied by the second image feature vector to obtain the detected first target and output it.

[0122] In one possible implementation, if the image to be detected is any video frame image in a video, the method further includes:

[0123] For each pixel in the image to be detected, the difference between the pixel value of the pixel and the pixel values ​​of each of the second preset number of pixels adjacent to the pixel stored in the historical pixel sample set is calculated; a first number of the difference values ​​are greater than or equal to a first preset value and less than or equal to a second preset value are counted; if the first number is greater than a first preset number threshold, the pixel is determined to be a background pixel; a second number of the difference values ​​are less than the first preset value or greater than the second preset value are counted; if the second number is greater than a second preset number threshold, the pixel is determined to be a foreground pixel, and the foreground pixel is determined as a pixel on the detected second target;

[0124] The first target and the second target in the image to be detected are labeled and output.

[0125] In one possible implementation, before labeling and outputting the first target and the second target in the image to be detected, the method further includes:

[0126] For each determined foreground pixel, it is determined whether the foreground pixel is determined as a foreground pixel in a consecutive second preset number of video frame images. If so, the foreground pixel is set as a background pixel.

[0127] In one possible implementation, the process of determining the historical pixel sample set includes:

[0128] Obtain an image at a preset video frame position in the video. For each pixel in the image, save the pixel value of that pixel and the second preset number of pixels adjacent to that pixel in the image to the historical pixel sample set.

[0129] In one possible implementation, the method further includes:

[0130] For each determined background pixel, the pixel value of that background pixel is used to update the pixel value of the corresponding pixel stored in the historical pixel sample set according to a preset update probability.

[0131] In one possible implementation, the method further includes:

[0132] For each video frame in the video, obtain each third target detected in the images of a third preset number of video frames adjacent to that video frame, wherein the third target includes each of the first target and the second target detected;

[0133] For each video frame image of the acquired third preset number, the maximum motion gradient of the third target is determined based on the third target detected in the video frame image and the vector velocity of the center pixel of the third target detected in the video frame image adjacent to the video frame in a preset direction.

[0134] The pixel points of potential targets are determined based on the maximum motion gradient of each third target and the center pixel point of each third target.

[0135] In one possible implementation, the method further includes:

[0136] For each pixel of the potential target, a small image is obtained by cropping according to a first preset pixel size standard, with the pixel of the potential target as the center; the small image corresponding to the same third target in a fourth preset number of video frames is selected to obtain a first small image set; and a second small image set with a second preset pixel size is obtained based on bilinear interpolation and the first small image set.

[0137] The second set of small images and the maximum motion gradient of the third target are input into a pre-trained second feature extraction network model to obtain the feature image extracted by the feature extraction layer of the second feature extraction network model; the feature image is input into a spatial pyramid pooling layer, which obtains the pooled third image feature vector; the third image feature vector is input into a fully connected layer, which determines the weight matrix of the feature image based on an attention mechanism, and determines and outputs the detected fourth target based on the weight matrix and the third image feature vector.

[0138] Since the principle of the above-mentioned electronic device in solving the problem is similar to that of the target detection method, the implementation of the above-mentioned electronic device can refer to the above embodiments, and the repeated parts will not be described again.

[0139] The communication bus mentioned in the above-mentioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 702 is used for communication between the above-mentioned electronic device and other devices. The memory can include random access memory (RAM), or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor. The aforementioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processing unit (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0140] Example 8:

[0141] Based on the above embodiments, this application also provides a computer-readable storage medium storing a computer program executable by a processor. When the program is run on the processor, the processor executes the following steps:

[0142] The image to be detected is divided into a preset number of cropped images according to a preset shape;

[0143] Each cropped image is input into a pre-trained first feature extraction network model. The feature extraction layer of the first feature extraction network model extracts a feature image of each cropped image and inputs the feature image into a spatial pyramid pooling layer. The spatial pyramid pooling layer obtains a pooled first image feature vector and inputs the first image feature vector into a fully connected layer. The fully connected layer determines the weight matrix of the image to be detected based on an attention mechanism, and determines and outputs the detected first target based on the weight matrix and the first image feature vector.

[0144] In one possible implementation, determining the detected first target based on the weight matrix and the first image feature vector includes:

[0145] The first weight matrix of the image to be detected, determined based on the channel attention mechanism, is multiplied by the first image feature vector to obtain the second image feature vector; the second weight matrix of the image to be detected, determined based on the pixel attention mechanism, is multiplied by the second image feature vector to obtain the detected first target and output it.

[0146] In one possible implementation, if the image to be detected is any video frame image in a video, the method further includes:

[0147] For each pixel in the image to be detected, the difference between the pixel value of the pixel and the pixel values ​​of each of the second preset number of pixels adjacent to the pixel stored in the historical pixel sample set is calculated; a first number of the difference values ​​are greater than or equal to a first preset value and less than or equal to a second preset value are counted; if the first number is greater than a first preset number threshold, the pixel is determined to be a background pixel; a second number of the difference values ​​are less than the first preset value or greater than the second preset value are counted; if the second number is greater than a second preset number threshold, the pixel is determined to be a foreground pixel, and the foreground pixel is determined as a pixel on the detected second target;

[0148] The first target and the second target in the image to be detected are labeled and output.

[0149] In one possible implementation, before labeling and outputting the first target and the second target in the image to be detected, the method further includes:

[0150] For each determined foreground pixel, it is determined whether the foreground pixel is determined as a foreground pixel in a consecutive second preset number of video frame images. If so, the foreground pixel is set as a background pixel.

[0151] In one possible implementation, the process of determining the historical pixel sample set includes:

[0152] Obtain an image at a preset video frame position in the video. For each pixel in the image, save the pixel value of that pixel and the second preset number of pixels adjacent to that pixel in the image to the historical pixel sample set.

[0153] In one possible implementation, the method further includes:

[0154] For each determined background pixel, the pixel value of that background pixel is used to update the pixel value of the corresponding pixel stored in the historical pixel sample set according to a preset update probability.

[0155] In one possible implementation, the method further includes:

[0156] For each video frame in the video, obtain each third target detected in the images of a third preset number of video frames adjacent to that video frame, wherein the third target includes each of the first target and the second target detected;

[0157] For each video frame image of the acquired third preset number, the maximum motion gradient of the third target is determined based on the third target detected in the video frame image and the vector velocity of the center pixel of the third target detected in the video frame image adjacent to the video frame in a preset direction.

[0158] The pixel points of potential targets are determined based on the maximum motion gradient of each third target and the center pixel point of each third target.

[0159] In one possible implementation, the method further includes:

[0160] For each pixel of the potential target, a small image is obtained by cropping according to a first preset pixel size standard, with the pixel of the potential target as the center; the small image corresponding to the same third target in a fourth preset number of video frames is selected to obtain a first small image set; and a second small image set with a second preset pixel size is obtained based on bilinear interpolation and the first small image set.

[0161] The second set of small images and the maximum motion gradient of the third target are input into a pre-trained second feature extraction network model to obtain the feature image extracted by the feature extraction layer of the second feature extraction network model; the feature image is input into a spatial pyramid pooling layer, which obtains the pooled third image feature vector; the third image feature vector is input into a fully connected layer, which determines the weight matrix of the feature image based on an attention mechanism, and determines and outputs the detected fourth target based on the weight matrix and the third image feature vector.

[0162] Since the principle of solving the problem using the computer-readable medium provided above is similar to that of the target detection method, the steps implemented after the processor executes the computer program in the computer-readable medium can be referred to the above embodiments, and repeated parts will not be described again.

[0163] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0164] For system / device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to in the description of the method embodiments.

[0165] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0166] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0167] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0168] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0169] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A target detection method characterized by, The method comprises: cutting the image to be detected into a preset number of cropped images according to a preset shape; inputting each of the cropped images into a first feature extraction network model which is pre-trained, a feature extraction layer of the first feature extraction network model extracting a feature image of each of the cropped images, and inputting the feature image into a spatial pyramid pooling layer, the spatial pyramid pooling layer obtaining a first image feature vector after pooling, and inputting the first image feature vector into a full connection layer, the full connection layer determining a weight matrix of the image to be detected based on an attention mechanism, and determining a detected first target and outputting according to the weight matrix and the first image feature vector; if the image to be detected is any video frame image in a video, the method further comprises: for each video frame in the video, obtaining each third target detected in images of a third preset number of video frames adjacent to the video frame, the third target comprising each of the first target and a second target detected in the images, wherein the second target is composed of foreground pixel points in the image to be detected, the foreground pixel points being determined based on a difference between a pixel value of a pixel point in the image to be detected and a pixel value of each of a second preset number of pixel points adjacent to the corresponding pixel point in a historical pixel point sample set; for each of the images of the third preset number of video frames obtained, determining a maximum motion gradient of the third target according to a center pixel point of the third target detected in the image of the video frame and a vector velocity of the center pixel point in a preset direction of the third target detected in the images of the video frames adjacent to the video frame; determining a pixel point of a potential target according to the maximum motion gradient of each third target and the center pixel point of each third target.

2. The method of claim 1, wherein, The determination of the detected first target according to the weight matrix and the first image feature vector comprises: multiplying a first weight matrix of the image to be detected determined based on a channel attention mechanism and the first image feature vector to obtain a second image feature vector; multiplying a second weight matrix of the image to be detected determined based on a pixel attention mechanism and the second image feature vector to obtain the detected first target and output.

3. The method of claim 1, wherein, if the image to be detected is any video frame image in a video, the method further comprises: for each pixel point in the image to be detected, calculating a difference between a pixel value of the pixel point and a pixel value of each of a second preset number of pixel points adjacent to the pixel point in a historical pixel point sample set; counting a first number of the difference values that are greater than or equal to a first preset value and less than or equal to a second preset value, and if the first number is greater than a first preset number threshold, determining the pixel point as a background pixel point; counting a second number of the difference values that are less than the first preset value or greater than the second preset value, and if the second number is greater than a second preset number threshold, determining the pixel point as a foreground pixel point, and determining the foreground pixel point as a pixel point on the detected second target; Annotate and output the first target and the second target in the image to be detected.

4. The method of claim 3, wherein, Before the annotating and outputting the first target and the second target in the image to be detected, the method further comprises: For each determined foreground pixel point, it is judged whether the foreground pixel point is determined as a foreground pixel point in continuous second preset number of video frame images, if yes, the foreground pixel point is set as a background pixel point.

5. The method of claim 3, wherein, The determination process of the historical pixel point sample set comprises: An image at a preset video frame position in the video is acquired, and for each pixel point in the image, the pixel values of the pixel point and the second preset number of pixel points adjacent to the pixel point in the image are saved to the historical pixel point sample set.

6. The method of any one of claims 3-5, wherein, The method further comprises: For each determined background pixel point, the pixel value of the background pixel point is used to update the pixel value of the pixel point saved in the historical pixel point sample set corresponding to the background pixel point according to a preset update probability.

7. The method of claim 1, wherein, The method further comprises: For each pixel point of the potential target, a small picture cutout is obtained by cropping according to a first preset pixel size standard with the pixel point of the potential target as the center; a first small picture set is obtained by selecting the small picture cutouts corresponding to the same third target in the fourth preset number of video frames; and a second small picture set of a second preset pixel size is obtained based on bilinear interpolation and the first small picture set; The second small picture set and the target maximum motion gradient of the third target are input into a second feature extraction network model pre-trained, a feature image extracted by a feature extraction layer of the second feature extraction network model is acquired, the feature image is input into a spatial pyramid pooling layer, the spatial pyramid pooling layer acquires a third image feature vector after pooling, and the third image feature vector is input into a full connection layer, the full connection layer determines a weight matrix of the feature image based on an attention mechanism, and determines a detected fourth target and outputs according to the weight matrix and the third image feature vector.

8. A target detection apparatus characterized by comprising: The device comprises: A cropping module configured to cut a to-be-detected image into a preset number of cropped images according to a preset shape; A detection module configured to input each of the cropped images into a first feature extraction network model pre-trained, a feature extraction layer of the first feature extraction network model extracts a feature image of each of the cropped images, and the feature image is input into a spatial pyramid pooling layer; the spatial pyramid pooling layer acquires a first image feature vector after pooling, and the first image feature vector is input into a full connection layer; the full connection layer determines a weight matrix of the to-be-detected image based on an attention mechanism, and determines a detected first target and outputs according to the weight matrix and the first image feature vector. If the image to be detected is any video frame image in a video, the detection module is further configured to: for each video frame in the video, acquire each third target detected in images of a third preset number of video frames adjacent to the video frame, wherein the third target includes each of the first target and the second target detected, the second target is composed of foreground pixel points in the image to be detected, the foreground pixel points are determined based on a difference between a pixel value of a pixel point in the image to be detected and a pixel value of each of a second preset number of pixel points adjacent to the corresponding pixel point in a historical pixel point sample set; for each image of the third preset number of video frames acquired, determine a maximum motion gradient of the third target according to a center pixel point of the third target detected in the image of the video frame and a vector velocity of the center pixel point of the third target detected in the images of the video frames adjacent to the video frame in a preset direction; and determine a pixel point of a potential target according to the maximum motion gradient of each third target and the center pixel point of each third target.

9. An electronic device, comprising: The electronic device comprises at least a processor and a memory, and the processor is configured to execute a computer program stored in the memory to implement the steps of the target detection method in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer program is stored in the memory and is executed by the processor to implement the steps of the target detection method in any one of claims 1-7.

Images