A method, device and application for moving target detection

Abstract

This application proposes a moving target detection method, device, and application. The method utilizes an improved lightweight target detection model to detect suspicious targets, an improved Gaussian mixture model to obtain foreground targets, and a set of foreground targets and suspicious targets to ultimately obtain the moving target. In implementation, the monitoring video is input into the improved lightweight target detection model to output a set of suspicious targets, and the monitoring video is input into the Gaussian mixture model to obtain a set of foreground targets. The overlap rate between foreground targets in the foreground target set and suspicious targets in the suspicious target set is iterated, and suspicious targets with high overlap rates are selected as moving targets. Further processing can then be performed on the moving targets.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target detection, and particularly relates to a moving target detection method and device and application. BACKGROUND

[0002] Moving target detection refers to extracting a moving target from a video image and obtaining feature information of the moving target by using a target detection technology. Since the moving target can only reflect the motion characteristics in a continuous image sequence, the difference features of the moving target and the background target need to be found in the continuous image sequence during detection, so as to accurately obtain the moving target.

[0003] The moving target detection technology has a very wide application in actual life. In particular, in the field of urban management, the moving target detection can replace manual work to complete the detection of many rule violation events. For example, the moving target detection technology can be applied to detect non-motor vehicles driving on the road. At present, common methods for moving target detection include: a continuous frame difference method, which uses the gray level difference of pixel points corresponding to different frames to judge the moving target. However, this scheme is difficult to detect the subtle changes of the moving target, resulting in distortion of the moving target detection result. A background difference method, which compares an input image with a background image to segment out the moving target. However, this scheme has a high requirement for the background image detection, and needs to exclude the influence of objective factors such as light, floating clouds and the like. In addition, since the moving target detection method needs to process and operate on multiple continuous images, the detection and training pressure of the model is very large, and there is a problem of low detection efficiency. SUMMARY

[0004] The embodiments of the present application provide a moving target detection method, device and application, which use an improved lightweight target detection model combined with an improved Gaussian mixture model to accurately and quickly identify the moving target, thereby improving the moving target detection speed and the identification accuracy of the moving target.

[0005] In a first aspect, embodiments of the present application provide a moving target detection method, the method comprising: acquiring a plurality of continuous video images and a background mean value, wherein the background mean value is a background pixel mean value corresponding to a scene where the video images are located; inputting each of the video images into a feature extraction layer of a first target detection model to extract at least two first feature maps with different down-sampling multiples, inputting the first feature maps into a feature fusion layer to fuse the first feature maps with each other to obtain at least two second feature maps; inputting the second feature maps into a detection head to perform prediction using a same group of convolutions with shared weights to obtain a suspicious target set composed of at least one suspicious target; comparing the plurality of continuous video images and the background mean value to obtain a foreground target set composed of at least one foreground target, wherein the foreground target is a target in a moving state relative to a background in the plurality of continuous video images; and obtaining an intersection of the suspicious target set and the foreground target set to obtain a moving target set composed of at least one moving target.

[0006] In a second aspect, embodiments of the present application provide a moving target detection device, comprising: a data acquisition unit configured to acquire a plurality of continuous video images and a background mean value, wherein the background mean value is a background pixel mean value corresponding to a scene where the video images are located; a suspicious target acquisition unit configured to input each of the video images into a feature extraction layer of a first target detection model to extract at least two first feature maps with different down-sampling multiples, input the first feature maps into a feature fusion layer to fuse the first feature maps with each other to obtain at least two second feature maps; input the second feature maps into a detection head to perform prediction using a same group of convolutions with shared weights to obtain a suspicious target set composed of at least one suspicious target; a foreground target acquisition unit configured to compare the plurality of continuous video images and the background mean value to obtain a foreground target set composed of at least one foreground target, wherein the foreground target is an object in a moving state relative to a background in the plurality of continuous video images; and a moving target acquisition unit configured to obtain an intersection of the suspicious target set and the foreground target set to obtain a moving target set composed of at least one moving target.

[0007] In a third aspect, embodiments of the present application provide an electronic device, comprising a memory and a processor, characterized in that the memory stores a computer program, and the processor is configured to run the computer program to execute any of the moving target detection methods described above.

[0008] In a fourth aspect, embodiments of the present application provide a computer program product comprising software code portions, when the computer program product is run on a computer, the software code portions are configured to execute any of the moving target detection methods described above.

[0009] Fifthly, embodiments of this application provide a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the moving target detection method described above.

[0010] The main contributions and innovations of this invention are as follows: This scheme utilizes an improved lightweight object detection model to acquire all suspicious targets in video images, uses an improved Gaussian mixture model for background modeling to obtain the background mean in the video images, acquires foreground targets in the video images based on the background mean, and obtains the overlapping set of suspicious targets and foreground targets to obtain moving targets. This scheme trains the lightweight object detection model using images of a set specification and optimizes and improves the feature fusion layer and convolution settings of the lightweight object detection model, greatly reducing the computational load of the model to improve the computing speed, and can achieve high-speed detection in video exceeding real-time speed (FPS>30); furthermore, this scheme uses the method of stitching small target images to train the lightweight object detection model, thereby improving the accuracy of object detection, so that the lightweight object detection model can balance detection efficiency and quality.

[0011] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0013] Figure 1 This is a flowchart of a moving target detection method according to the first embodiment of this application;

[0014] Figure 2 This is a logic diagram of the moving target detection method according to the second embodiment of this application applied to non-motorized vehicle helmet detection;

[0015] Figure 3 This is a structural block diagram of a moving target detection model according to an embodiment of this application;

[0016] Figure 4 This is a structural block diagram of a moving target detection device according to an embodiment of this application;

[0017] Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0018] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0019] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0020] Example 1

[0021] This application aims to propose a technical solution that utilizes an improved lightweight target detection model to detect suspicious targets, an improved Gaussian mixture model to detect foreground targets, and a set of foreground targets and suspicious targets to finally obtain moving targets. In implementation, a set of suspicious targets is output by inputting a surveillance video image into the improved lightweight target detection model. The background mean obtained by the surveillance video image and the Gaussian mixture model is compared to obtain a set of foreground targets. The overlap rate between the foreground targets in the foreground target set and the suspicious targets in the suspicious target set is traversed, and suspicious targets with high overlap rates are selected as moving targets. Further processing can be performed on the moving targets.

[0022] This application provides a moving target detection method that can quickly and accurately acquire moving targets in video images. Specifically, refer to... Figure 1 The method includes:

[0023] Acquire multiple consecutive frames of video images and the average background value, wherein the average background value is the average background pixel value of the scene in which the video image is located;

[0024] Each frame of the video image is input into the feature extraction layer of the first target detection model to extract at least two first feature maps with different downsampling factors. The first feature maps are input into the feature fusion layer and fused together to obtain at least two second feature maps. The second feature maps are input into the detection head and predicted using the same set of convolutions with shared weights to obtain a set of suspicious targets consisting of at least one of the suspicious targets.

[0025] The mean value of the background is compared and processed to obtain a set of foreground targets consisting of at least one foreground target, wherein the foreground target is a target that is in motion relative to the background in the continuous video images.

[0026] Obtain the intersection of the suspected target set and the foreground target set to obtain a moving target set consisting of at least one moving target.

[0027] Unlike approaches that directly utilize a moving target detection model for one-time moving target detection, this approach determines the target type by performing independent suspicious target detection, and determines the motion attributes by performing independent foreground target detection. Finally, it combines these two key features—motion attributes and target type—to identify the moving target. This significantly reduces the training load and computational complexity of the moving target detection model, while improving its recognition accuracy. In other words, since moving targets need to be detected in continuous video images, using a unified moving target detection model would require training the model on a massive video stream, resulting in a very high computational cost and difficulty in guaranteeing accurate recognition results.

[0028] However, this solution, based on two key features of the moving target to be detected—target type and motion attributes—utilizes an optimized lightweight neural network model to detect moving targets, improving both detection accuracy and speed. It's worth noting that since target detection needs to be performed individually in each frame of the video image, the target detection rate must be significantly higher than the video's frame rate to ensure no target is missed in any frame. Therefore, the lightweight neural network model proposed in this solution is of great significance. This lightweight neural network model can achieve high-speed detection in video, exceeding real-time speed (FPS>30). The first target detection model in this solution is trained based on this lightweight neural network model.

[0029] The lightweight nature of the first object detection model in this scheme is reflected in many aspects. Firstly, the first object detection model in this scheme detects small-sized video images, which greatly reduces the amount of computation. Secondly, the network framework of the first object detection model itself has been optimized, which improves the computation speed without sacrificing the recognition accuracy.

[0030] Correspondingly, before “inputting each frame of the video image into the feature extraction layer of the first target detection model to extract at least two first feature maps with different downsampling factors”, the steps include: preprocessing each frame of the video image to obtain a normalized video image, inputting the normalized video image into the first target detection model to output a predicted target, and processing the position coordinates of the predicted target to obtain the suspicious target.

[0031] In this scheme, in order to optimize the calculation speed of the first target detection model, the video image is first normalized in terms of size and pixels. The normalized video image, which has a smaller memory footprint than the original video image, is then input into the first target detection model for calculation to obtain the position coordinates of the predicted target. The position coordinates of the predicted target are then magnified to obtain the suspicious target in the original video image. This greatly saves the computational resources of the first target detection model.

[0032] Correspondingly, the step of "preprocessing each frame of the video image to obtain a normalized video image" includes the steps of scaling each frame of the video image to its original size and performing image pixel normalization processing to obtain the normalized video image. At this time, the step of "obtaining the suspected target from the processed predicted target's position coordinates" includes the step of reverse-scaling the obtained predicted target's position coordinates. That is, the first target detection model detects the small-sized normalized video image, and the detected result is then enlarged back into the original video image to obtain the suspected target. For example, this solution can scale each frame of the video image to a size of 416*416.

[0033] Furthermore, this solution also optimizes the structure of the first target detection model, referencing... Figure 3 The network structure of the first object detection model is as follows:

[0034] The first object detection model uses ShuffleNetV2 as its backbone network for feature extraction, and extracts the last convolutional layer of ShuffleNetV2. It then extracts features downsampled by 8, 16, and 32 times to input feature fusion layers for multi-scale feature fusion. The feature fusion layer uses PAN (Path Aggregation Network) to improve the accuracy of small object detection. For the sake of lightweight design, all convolutional layers of PAN are completely removed, and only 1x1 convolutions extracted from the backbone network are retained for feature channel dimension alignment. Both upsampling and downsampling are performed using interpolation. The detection head uses a shared-weight detection head, which uses the same set of convolutions to predict detection boxes on the multi-scale feature maps from the feature fusion layer. Then, each layer uses a learnable scale value as a coefficient to scale the predicted boxes. The advantage of this approach is that it can reduce the number of parameters of the detection head to 1 / 3 of that in the non-shared-weight state.

[0035] In other words, the first object detection model in this scheme has undergone many optimizations for lightweight design: First, redundant convolutional layers are removed from the feature extraction layer, retaining only features downsampled by 8, 16, and 32 times; second, all convolutional layers are also removed from the feature fusion layer, and interpolation is used to fuse upsampling and downsampling; finally, a detection head with shared weights is used to predict multiple multi-scale detection maps using the same set of convolutions, and the predicted bounding boxes are scaled. This first object detection model can greatly reduce the amount of computation and improve the processing speed, enabling high-speed detection in video exceeding real-time speed (FPS>30). It is worth noting that the "same set" mentioned in the paper refers to three horizontal convolutional functions that share parameters, and this stacking consists of two layers.

[0036] Of course, other backbone networks can be used for the feature extraction layer. The reason why ShuffleNetV2 is preferred as the backbone network in this scheme is that ShuffleNetV2 has a smaller number of parameters while maintaining the same accuracy, and it is also more CPU-friendly for inference. The PAN used in the feature fusion layer can be simply understood as FPN with an additional bottom-up pathaugmentation. FPN is top-down, while PAN includes both top-down and bottom-up paths.

[0037] In this scheme, the feature extraction layer selects ShuffleNetV2 as the backbone network, and removes the last convolutional layer of ShuffleNetV2, retaining only the 8x, 16x, and 32x downsampled convolutional layers of ShuffleNetV2. The feature fusion layer uses PAN, and removes all convolutional layers of PAN, retaining only the 1x1 convolutions extracted from the backbone network features for feature channel dimension alignment. Both upsampling and downsampling in the feature fusion layer are performed using interpolation. The detection head mimics the FCOS series, using a shared weight detection head. It uses the same set of convolutions on the second feature map to obtain the predicted bounding box, and each layer uses a learnable scale value as a coefficient to scale the predicted bounding box.

[0038] The training method for the first object detection model is as follows: Obtain training samples: Build the network framework of the first object detection model as described above, and train it using training samples.

[0039] In this embodiment of the solution, in order to optimize the effect of small target recognition, multiple samples are stitched together into a large image as training samples to train the network framework of the first target detection model. This training method is equivalent to shrinking large and medium targets into small targets for recognition, increasing the proportion of small target samples in the overall training samples, thereby improving the recognition rate of small targets.

[0040] It is worth noting that different training samples can be used to train the first object detection model to obtain first object detection models for different object types. In this scheme, if the first object detection model is suitable for detecting non-motorized vehicles, a batch of electric vehicle sample data can be collected, and the first object detection model can be used to detect this batch of data. The detection results are divided into two categories: framed images and frameless images. Framed images are further divided into true target images and false alarm target images. Frameless images are further divided into images without detected targets and images that do not contain targets. False alarm target images are used as negative samples, and images containing electric bicycle targets but not detected are used as training positive samples. Data annotation and data augmentation are performed on those images that did not detect electric bicycles. Then, a new model is trained based on the original first object detection model. The effect of the first object detection model is tested to check whether the accuracy meets the standard. If the new first object detection model does not meet the standard, new data is added and the network parameters are adjusted for training. If the accuracy of the first object detection model has reached the requirement and is optimal under the current training data, training is stopped. This process is repeated to achieve a model suitable for the complexity of samples in the actual environment. In this way, a first object detection model suitable for detecting electric vehicles is obtained. Similarly, if the first target detection model is suitable for detecting helmets, a batch of helmet sample data can be collected and trained according to the above method.

[0041] Correspondingly, before "inputting each frame of the video image into the feature extraction layer of the first target detection model to extract at least two first feature maps with different downsampling factors", the steps include: preprocessing each frame of the video image to obtain a normalized video image; inputting the normalized video image into the feature extraction layer of the first target detection model to extract at least two first feature maps with different downsampling factors; inputting the first feature maps into the feature fusion layer and fusing them to obtain at least two second feature maps; inputting the second feature maps into the detection head and using the same set of convolutions with shared weights to predict and obtain a prediction box, the prediction box being the predicted target; and processing the position coordinates of the predicted target to obtain the suspected target.

[0042] Furthermore, this solution improves and optimizes the structure of the Gaussian mixture model. The Gaussian mixture model in this solution is used to distinguish between background and foreground objects in the video image.

[0043] The background mean obtained by comparing multiple consecutive frames of video images with the background mean obtained by the Gaussian mixture model is used to obtain a set of foreground objects consisting of at least one foreground object. A foreground object is any meaningful moving object under the assumption that the background is stationary.

[0044] The following is a brief introduction to the basic concept of Gaussian mixture model. Gaussian mixture model is based on the principle that the pixel values ​​of multiple consecutive frames of video images follow a Gaussian distribution on the time axis. It uses multiple Gaussian distributions to model the background for each pixel value, and adapts to the changes in the background by updating the parameters of the Gaussian mixture model, thereby obtaining a clear and accurate background pixel.

[0045] The basic process of the Gaussian mixture model is as follows:

[0046] Assume that at time t, the pixel value j takes the value X. j The probability is:

[0047]

[0048] Where K is the number of Gaussian mixture models, its size can be determined based on the computer's processing power, and is generally taken as 3-5. This represents the weight of the i-th Gaussian component in the Gaussian mixture model of pixel j at time t. and Let represent the mean and covariance of the i-th Gaussian component, respectively, represent the standard deviation, I represent the identity matrix, and η represent the Gaussian probability density function, where ...

[0049]

[0050] Where d is x i dimensionality;

[0051] During background model initialization, the pixel values ​​of the first frame image are used as the mean, a large variance is chosen, and a small weight is selected. As the scene changes, the Gaussian mixture model for each pixel needs to be continuously learned and updated, using the current pixel value x. i Comparing each of the K Gaussian components in its mixture model, if x i The mean of the i-th Gaussian component The difference between them is less than δ times the standard deviation of the Gaussian component. Then the Gaussian component is x i Updated, the remaining Gaussian components remain unchanged, i.e.:

[0052] (δ is usually set to 2.5-3.5)

[0053] If the conditions are met, update the Gaussian components. as well as Parameters (using a certain learning rate).

[0054] If no model matches, the Gaussian component with the smallest weight is replaced. This model has the mean of the current pixel value, a large variance, and a small weight.

[0055] The K Gaussian components in the mixture model are arranged according to... Sort from largest to smallest, then

[0056]

[0057] T is a threshold representing the proportion of background. During foreground detection, if the detected pixel X matches any one of the B Gaussian mixture models describing the background, then the pixel belongs to the background; otherwise, it belongs to the foreground. Using the aforementioned Gaussian mixture model as the background model, the video image input to the Gaussian mixture model distinguishes between foreground and background targets.

[0058] However, traditional Gaussian mixture models (GMMs) describe each pixel using multiple Gaussian models. This leads to excessively frequent pixel reading and matching updates, causing the background model to fail to adapt promptly to sudden scene changes. Considering that the background of consecutive video frames is relatively stable, typically exhibiting only minor local changes while most other areas remain unchanged, and many adjacent pixels in the background may have the same pixel value, the model parameters for these points should be identical or similar when modeling them using Gaussian models. Therefore, this solution optimizes and improves the Gaussian mixture model: It divides the pixels of each video frame into multiple pixel blocks, each containing multiple pixels. The average pixel value of each block replaces all pixel values ​​within that block. Correspondingly, the Gaussian mixture model is calculated on a block-by-block basis during model building and matching. This approach significantly saves computation time and system storage space.

[0059] Correspondingly, the process of “comparing and processing multiple consecutive video images with the mean value of the background to obtain a set of foreground targets consisting of at least one foreground target” includes: dividing the pixels of each frame of the video image into at least one pixel block consisting of multiple pixels, wherein the pixel block includes at least two pixels, and comparing all the pixel blocks of the same video image with the mean value of the background to obtain a set of foreground targets consisting of at least one foreground target.

[0060] It is worth noting that the background mean is obtained by inputting images of the same scene as the video image into the Gaussian mixture model. The background mean is the average background pixel value of the scene where the video image is located. In other words, the background mean is obtained by inputting video images of the same scene into the Gaussian mixture model. The calculation and establishment logic of the Gaussian mixture model are as described above. The difference is that this solution also uses pixel blocks of video images as the unit when establishing the Gaussian mixture model.

[0061] In this scheme, the Gaussian mixture model is established and matched on a pixel block basis. Each frame of the video image is processed to obtain multiple uniform pixel blocks, meaning that the number of pixels within each pixel block is uniform. In an embodiment of this scheme, the pixel block includes 10*10 pixels.

[0062] Specifically, "comparing all the pixel blocks of the same video image with the mean value of the background to obtain a foreground target set consisting of at least one foreground target" includes the steps of: obtaining a comparison value between each pixel block of the same video image and the mean value of the background, selecting pixel blocks whose comparison value is greater than a set value to form a pixel block set, and connecting the pixel blocks in the pixel block set to obtain the foreground target.

[0063] In this scheme, the video image can be used as the first frame. The first frame is input into the Gaussian mixture model. Each pixel block in the first frame is compared with the background mean of the Gaussian mixture model to obtain a comparison value. If the comparison value is greater than a certain threshold, it is considered as foreground; otherwise, it is considered as background. Pixel blocks with comparison values ​​greater than the set value are assigned a value of 1, and the remaining pixel blocks are assigned a value of 0 to obtain a binarized second frame. In the second frame, the circumscribed contour judgment method is used to obtain each connected object. For connected objects exceeding a certain size, a bounding rectangle is used to calculate the circumscribed rectangle of the contours of these connected objects (obtaining the maximum and minimum values ​​of the x and y coordinates; the circumscribed rectangle is the rectangle enclosed by the two diagonal coordinate vertices (xmin, ymin) and (xmax, ymax). All rectangular objects are used as foreground targets.

[0064] Correspondingly, in the step of “selecting the pixel blocks whose comparison value is greater than the set value to form a pixel block set, and connecting the pixel blocks in the pixel block set to obtain the foreground target”, the pixel blocks whose comparison value is greater than the set value are assigned a value of 1, and the remaining pixel blocks are assigned a value of 0 to obtain a binary image, and the connected objects in the binary image are connected to obtain the foreground target.

[0065] Furthermore, in some embodiments, a rectangular frame is used to process the connected objects to obtain the foreground target. Specifically, for connected objects exceeding a certain size, a rectangular frame is used to calculate the circumscribed rectangle of the outlines of these connected objects (obtaining the maximum and minimum values ​​of the x and y coordinates; the circumscribed rectangle is the rectangle enclosed by the two diagonal coordinate vertices (xmin, ymin) and (xmax, ymax), thus obtaining a rectangular frame, which serves as the foreground target.

[0066] "Obtaining the intersection of the suspicious target set and the foreground target set to obtain a moving target set consisting of at least one moving target" includes the following steps: for each suspicious target in the suspicious target set, calculating the overlap rate between the suspicious target and all the foreground targets in the foreground target set; if the overlap rate is greater than a set threshold, then determining the suspicious target as the moving target.

[0067] In some embodiments, the foreground targets that overlap with the suspected target in the foreground target set can be obtained based on the position coordinates of the rectangle. In this case, only the overlap rate between the suspected target and the foreground target that overlaps with the suspected target needs to be compared.

[0068] In this scheme, the overlap rate is obtained through IOU calculation, and the specific calculation method is as follows: Where F is the IOU calculation function, Ri is each foreground target in the foreground target set, and O is a suspicious target. In some embodiments, the threshold can be set to 0.5.

[0069] It is worth mentioning that the moving target detection method provided in this solution can be used to detect non-motorized vehicles on the road. In this case, the first target detection model in the moving target detection method of this application embodiment is used to detect non-motorized vehicles. The moving target is a moving non-motorized vehicle, and the continuous multi-frame video images can be selected from video data of surveillance cameras accessed from the transportation department, public security department, and urban management department.

[0070] In addition, the moving target detection method provided in this solution can also be used to detect non-motorized vehicle drivers not wearing helmets. In this case, refer to Figure 2 The moving target detection method includes the following steps: The method includes:

[0071] Acquire multiple consecutive frames of video images and the average background value, wherein the average background value is the average background pixel value of the scene in which the video image is located;

[0072] Each frame of the video image is input into the feature extraction layer of the first target detection model to extract at least two first feature maps with different downsampling factors. The first feature maps are input into the feature fusion layer and fused together to obtain at least two second feature maps. The second feature maps are input into the detection head and predicted using the same set of convolutions with shared weights to obtain a set of suspicious targets consisting of at least one of the suspicious targets.

[0073] The mean value of the background is compared and processed to obtain a set of foreground targets consisting of at least one foreground target, wherein the foreground target is a target that is in motion relative to the background in the continuous video images.

[0074] Obtain the intersection of the suspected target set and the foreground target set to obtain a moving target set consisting of at least one moving target;

[0075] The set of moving targets is input into the second target detection model to output the target objects.

[0076] In this embodiment, the first target detection model and the second target detection model have the same model architecture, the difference being that the first target detection model is used to detect non-motorized vehicles, and the second target detection model is used to detect helmets. Correspondingly, the suspicious target is a non-motorized vehicle, and the target object is a helmet. Since the model architecture of the second target detection model is completely the same as that of the first target detection model, the only difference being the target type of the training samples, the second target detection model will not be described again. In this embodiment, if the set of moving targets is input into the second target detection model and the output is "helmet," it indicates that the moving non-motorized vehicle is wearing a helmet. During management, it is often necessary to obtain information on non-motorized vehicles not wearing helmets; therefore, the method further includes:

[0077] Moving targets that were not detected are identified as violating targets. Here, a violating target refers to a person riding a non-motorized vehicle without a helmet. After obtaining the violating targets, facial recognition technology can be used for subsequent operations such as facial identification of the violators.

[0078] Example 2

[0079] Based on the same concept, referencing Figure 4 This application also proposes a moving target detection device, comprising:

[0080] The data acquisition unit 301 is used to acquire multiple consecutive frames of video images and the average background value, wherein the average background value is the average background pixel value of the scene in which the video image is located.

[0081] Suspicious target acquisition unit 302 is used to input the video image of each frame into the feature extraction layer of the first target detection model to extract at least two first feature maps with different downsampling factors, and input the first feature maps into the feature fusion layer to fuse them to obtain at least two second feature maps; the second feature maps are input into the detection head to use the same set of convolutions with shared weights for prediction to obtain a suspicious target set composed of at least one of the suspicious targets;

[0082] The foreground target acquisition unit 303 is used to compare and process the mean value of the background with the continuous multi-frame video images to obtain a foreground target set consisting of at least one foreground target, wherein the foreground target is an object in motion relative to the background in the continuous multi-frame video images;

[0083] The moving target acquisition unit 304 is used to acquire the intersection of the suspected target set and the foreground target set to obtain a moving target set consisting of at least one moving target.

[0084] The operating logic and method of this moving target detection device are the same as those of the moving target detection method described in Embodiment 1, so they will not be repeated here.

[0085] Example 3

[0086] This embodiment also provides an electronic device, see reference. Figure 5 It includes a memory 404 and a processor 402, the memory 404 storing a computer program and the processor 402 being configured to run the computer program to perform the steps in any of the moving target detection method embodiments described above.

[0087] Specifically, the processor 402 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0088] The memory 404 may include a mass storage device for data or instructions. For example, and not limitingly, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 404 may include removable or non-removable (or fixed) media. Where appropriate, the memory 404 may be internal or external to a data processing device. In a particular embodiment, the memory 404 is non-volatile memory. In a particular embodiment, the memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0089] The memory 404 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 402.

[0090] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any of the moving target detection methods in the above embodiments.

[0091] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408, wherein the transmission device 406 is connected to the processor 402, and the input / output device 408 is connected to the processor 402.

[0092] The transmission device 406 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 406 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0093] Input / output device 408 is used to input or output information. In this embodiment, the input information may be video streams monitoring various scenes, and the output information may be moving targets in the video stream, specific objects on the moving targets, etc.

[0094] Optionally, in this embodiment, the processor 402 can be configured to perform the following steps via a computer program:

[0095] S101. Obtain multiple consecutive frames of video images and the average background value, wherein the average background value is the average background pixel value of the scene where the video image is located.

[0096] S201. Input each frame of the video image into the feature extraction layer of the first target detection model to extract at least two first feature maps with different downsampling factors. Input the first feature maps into the feature fusion layer and fuse them to obtain at least two second feature maps. Input the second feature maps into the detection head and use the same set of convolutions with shared weights for prediction to obtain a set of suspicious targets consisting of at least one of the suspicious targets.

[0097] S203. The continuous multi-frame video images and the background mean are compared and processed to obtain a foreground target set consisting of at least one foreground target, wherein the foreground target is a target that is in motion relative to the background in the continuous multi-frame video images.

[0098] S204. Obtain the intersection of the suspicious target set and the foreground target set to obtain a moving target set consisting of at least one moving target.

[0099] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0100] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0101] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer-executable components configured to perform embodiments when the program is run. One or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted that any block in the logical flow of the figures may represent a program step, or interconnected logical circuitry, blocks and functions, or a combination of program steps and logical circuitry, blocks and functions. The software may be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0102] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0103] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A moving object detection method characterized by, The method comprises the steps of: obtaining a plurality of continuous video images and a background mean value, wherein the background mean value is a background pixel mean value corresponding to a scene where the video images are located, and the background mean value is obtained by inputting images of the same scene where the video images are located into a Gaussian mixture model; the Gaussian mixture model uses a plurality of Gaussian distributions to model each pixel value according to the principle that the pixel values of the plurality of continuous video images obey Gaussian distribution on a time axis, and adapts to changes in the background by updating parameters of the Gaussian mixture model, thereby obtaining background pixels; scaling the size of each frame of the video image by a scaling ratio and performing image pixel normalization to obtain a normalized video image, wherein the normalized video image is input into a first target detection model; inputting each frame of the video image into a feature extraction layer of the first target detection model to extract at least two first feature maps of different down-sampling multiples, inputting the first feature maps into a feature fusion layer to fuse the first feature maps with each other to obtain at least two second feature maps; inputting the second feature maps into a detection head to perform prediction using the same group of convolutions with shared weights, inversely enlarging the prediction by the scaling ratio to obtain the position coordinates of a predicted target to obtain a suspicious target, obtaining a suspicious target set composed of at least one suspicious target, and determining a target type through suspicious target detection; comparing the plurality of continuous video images and the background mean value to obtain a foreground target set composed of at least one foreground target, wherein the foreground target is a target in a moving state relative to the background in the plurality of continuous video images, and a motion attribute is determined through foreground target detection; obtaining the intersection of the suspicious target set and the foreground target set to obtain a moving target set composed of at least one moving target.

2. The moving object detection method according to claim 1, characterized in that, The feature extraction layer selects ShuffleNetV2 as a backbone network, and removes the last convolution layer of ShuffleNetV2; the feature fusion layer adopts PAN, and removes all convolution layers of PAN, and only retains a 1x1 convolution after feature extraction of the backbone network to align the feature channel dimension, and the upsampling and downsampling of the feature fusion layer are completed by interpolation.

3. The moving object detection method of claim 1, wherein, The comparison of the plurality of continuous video images and the background mean value to obtain a foreground target set composed of at least one foreground target comprises: dividing the pixels of each frame of the video image to obtain at least one pixel block composed of a plurality of pixels, comparing all pixel blocks of the same video image with the background mean value to obtain a foreground target set composed of at least one foreground target.

4. The moving object detection method of claim 1, wherein, The comparison of the plurality of continuous video images and the background mean value to obtain a foreground target set composed of at least one foreground target comprises: dividing the pixels of each frame of the video image to obtain at least one pixel block composed of a plurality of pixels, obtaining a comparison value of each pixel block of the same video image and the background mean value, selecting the pixel blocks with comparison values greater than a set value to form a pixel block set, and connecting the pixel blocks in the pixel block set to obtain the foreground target.

5. The moving object detection method according to claim 4, characterized in that, The "obtaining a set of foreground objects by connecting the pixel blocks in the pixel block set" includes: assigning 1 to the pixel blocks with the comparison value greater than the set value, and assigning 0 to the rest of the pixel blocks to obtain a binary image, and connecting the connected objects in the binary image to obtain the foreground objects.

6. The moving object detection method of claim 1, wherein, The "obtaining a set of moving objects by obtaining the intersection of the set of suspicious objects and the set of foreground objects" includes the steps of: for each suspicious object in the set of suspicious objects, calculating the overlap rate of the suspicious object and all the foreground objects in the set of foreground objects, and determining the suspicious object as the moving object if the overlap rate is greater than a set threshold.

7. The moving object detection method according to claim 1, characterized by, The method comprises: inputting the set of moving objects into a second target detection model to output a target object.

8. The moving object detection method according to claim 7, characterized in that, The suspicious object is a non-motor vehicle, and the target object is a helmet.

9. A moving object detection apparatus characterized by comprising: The method comprises: a data acquisition unit configured to acquire a plurality of continuous video images and a background mean value, wherein the background mean value is a background pixel mean value corresponding to a scene in which the video images are located, and the background mean value is obtained by inputting images of the same scene in which the video images are located into a mixture Gaussian model; a suspicious object acquisition unit configured to scale the size of each frame of the video images by a scaling ratio and perform image pixel normalization processing to obtain normalized video images, input the normalized video images into a first target detection model, extract at least two first feature maps with different down-sampling multiples by inputting each frame of the video images into a feature extraction layer of the first target detection model, fuse the first feature maps with each other in a feature fusion layer to obtain at least two second feature maps, and use the same group of convolutions with shared weights in a detection head to predict the second feature maps, inversely enlarge the prediction by the scaling ratio, and obtain the position coordinates of a predicted target to obtain a set of suspicious objects composed of at least one suspicious object, and determine the type of the target by suspicious object detection; a foreground object acquisition unit configured to compare the plurality of continuous video images with the background mean value to obtain a set of foreground objects composed of at least one foreground object, wherein the foreground object is an object in a moving state relative to the background in the plurality of continuous video images, and determine the motion attribute by foreground object detection; a moving object acquisition unit configured to obtain the intersection of the set of suspicious objects and the set of foreground objects to obtain a set of moving objects composed of at least one moving object. 10.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to run the computer program to execute the moving object detection method according to any one of claims 1 to 8. The memory stores a computer program, and the processor is configured to run the computer program to execute the moving object detection method according to any one of claims 1 to 8.

11. A computer program product, characterised in that, comprising software code portions for performing the motion target detection method according to any one of claims 1-8 when the computer program product is run on a computer.

12. A readable storage medium, characterized by, The computer program is stored in the readable storage medium, and the computer program comprises program code for controlling a process to execute the process, and the process comprises the motion target detection method according to any one of claims 1-8.

Images