Moving target monitoring system based on image recognition

Abstract

The invention provides a moving target monitoring system based on image recognition. A moving target monitoring system based on image recognition comprises a monitoring video acquisition module, a picture set generation and extraction module, a to-be-detected picture preprocessing module, an image recognition model management module, a moving target monitoring model management module, a moving area prior determination module and a moving target marking module. A moving target area generation module; and a moving target identification module. According to the method, the motion area where the motion target possibly exists is extracted for motion target monitoring, invalid calculation on the area where the motion target does not appear all the time during follow-up motion target monitoring is avoided, the calculation power is saved, and the monitoring efficiency of the motion target is improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and more specifically, to a moving target monitoring system based on image recognition. Background Art

[0002] With the continuous development of computer vision, moving target recognition has also been widely used in fields such as video surveillance. At present, the monitoring of moving targets is generally achieved by using Gaussian mixture models. Since the distribution of each pixel in the time series is Gaussian, and the pixel values ​​that exist for a long time are background points, and the pixel values ​​that exist for a short time are foreground points, the Gaussian mixture model is used to monitor moving targets with good results. However, when using the Gaussian mixture model, Gaussian distribution matching needs to be performed for each pixel in the image from the video frame. However, in fact, there is a part of the image from the video frame where no moving targets appear. The operation for this is invalid and will waste computing power. Summary of the invention

[0003] The present invention provides a moving target monitoring system based on image recognition, which performs moving target monitoring by extracting moving areas where moving targets may exist, thereby avoiding invalid calculations for areas where no moving targets have appeared during subsequent moving target monitoring, saving computing power, and improving the monitoring efficiency of moving targets.

[0004] A moving target monitoring system based on image recognition, comprising:

[0005] A surveillance video acquisition module is used to acquire surveillance videos of a specified area;

[0006] The picture set generation and extraction module is used to divide the surveillance video into frames, and form a picture set from all frames. The pictures in the picture set are numbered in chronological order and recorded as the test picture F. n , where n = 1, 2, 3...N, N is the total number of frames in the picture set, and the picture to be tested F n is the video frame corresponding to the nth moment;

[0007] The test picture preprocessing module is used to convert the test picture F n Perform pre-processing;

[0008] An image recognition model management module is used to establish and store image recognition models, and the image recognition models are used to identify moving targets;

[0009] A moving target monitoring model management module is used to update and store a moving target monitoring model, where the moving target monitoring model is established based on a Gaussian mixture model;

[0010] The motion region prior determination module is used to determine the image F to be tested based on the image recognition model and training set. n The movement area in

[0011] The moving target marking module is used to detect the image F according to the moving target monitoring model. n The moving area in the image is used to monitor the moving target and mark the positioning rectangular frame where the moving target is located;

[0012] The moving target area generation module is used to generate a moving target area according to the positioning rectangle frame in the corresponding image to be tested F n Segment the moving target area;

[0013] The moving target recognition module is used to perform image recognition on the moving target area according to the moving target area and the image recognition model, and determine whether the moving target area contains a preset moving target.

[0014] As a preferred embodiment of the present invention, the following contents are performed by determining the motion area through the motion area prior determination module: obtaining a training set, which is a set of pictures formed by framing all the surveillance videos acquired within a monitoring cycle; performing picture recognition on the pictures in the training set through an image recognition model, and outputting a moving target rectangular frame; adding all coordinates in all moving target rectangular frames to the moving target coordinate set, removing duplicate coordinates in the moving target coordinate set, and then selecting the maximum value of the ordinate in the moving target coordinate set, recorded as y p-max , select the minimum value of the horizontal coordinate from the moving target coordinate set, recorded as x p-min , select the maximum value of the ordinate from the moving target coordinate set, recorded as y p-min , with the upper left corner coordinate (x p-min ,y p-min ) and the lower right corner coordinate (x p-max ,y p-max )Establish exercise areas.

[0015] As a preferred embodiment of the present invention, the target monitoring model management module has a built-in moving target monitoring model, and the establishment of the moving target monitoring model includes the following contents: for the image F to be tested n For each pixel point M in , K Gaussian distribution layers are set, and the Gaussian distribution layer has a built-in Gaussian distribution function.

[0016] As a preferred embodiment of the present invention, according to the moving target monitoring model, the image F to be tested is n The moving target monitoring in the moving area includes the following steps: n The motion area in the moving target detection model is sent to the moving target detection model, and the moving target detection model is used for the image to be tested. nEach pixel point M in the motion area in the image is matched with its corresponding K Gaussian distribution layers. If a match is successful, the mean, standard deviation and weight of the successfully matched Gaussian distribution layer are updated, and the weight of the unmatched Gaussian distribution layer is updated. If a match is not successful, a new Gaussian distribution layer is established based on the pixel point M to replace the Gaussian distribution layer with the smallest weight among the original K Gaussian distribution layers; then all Gaussian distribution layers are sorted, the first B Gaussian distribution layers are selected as the background layer, and then the image to be tested F is sequentially placed on the image. n Each pixel point M in the motion area in the image is matched with the background layer. If the match is successful, it means that the pixel point is a background point. If the match fails, it means that the pixel point is a foreground point. The coordinates corresponding to the foreground point are calculated through the connected domain, and the coordinates corresponding to each generated connected domain are added to the foreground point coordinate set. For each foreground point coordinate set, the following content is performed: Select the maximum horizontal coordinate from the foreground point coordinate set, recorded as x q-max , select the maximum ordinate from the foreground point coordinate set, recorded as y q-max , select the minimum value of the horizontal coordinate from the previous scenic spot coordinate set, denoted as x q-min , select the maximum ordinate from the foreground point coordinate set, recorded as y q-min , with the upper left corner coordinate (x q-min ,y q-min ) and the lower right corner coordinate (x q-max ,y q-max )Create a positioning rectangle.

[0017] As a preferred embodiment of the present invention, the moving target area is image recognized by the moving target recognition module, including the following contents: the moving target area is sent to the image recognition model, and it is determined whether the moving target area is a preset moving target. If there is a moving target in the moving target area, a marked rectangular frame corresponding to the moving target is output, and the marked rectangular frame is mapped to the corresponding image to be tested F n .

[0018] As a preferred embodiment of the present invention, the present invention further includes a moving target segmented video generation module for generating a moving target segmented video in the image set. n Convert to moving target segmented video and output it.

[0019] As a preferred embodiment of the present invention, the image recognition model adopts the YOLO-V3 algorithm.

[0020] The present invention has the following advantages:

[0021] 1. The present invention monitors moving targets by extracting moving areas where moving targets may exist, thereby avoiding invalid calculations for areas where no moving targets have appeared during subsequent moving target monitoring, saving computing power, and improving the monitoring efficiency of moving targets.

[0022] 2. The present invention establishes a positioning rectangular frame through a Gaussian mixture model to pre-position the moving target, which facilitates the subsequent accurate monitoring of the moving target through image recognition. Moreover, since the positioning rectangular frame has fewer pixels than the entire image to be tested, the efficiency of image recognition is also higher.

[0023] 3. The present invention realizes monitoring of moving targets by performing image recognition on moving targets within a positioning rectangular frame; and the moving targets can be monitored more intuitively by marking a rectangular frame on the image to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the structure of a moving target monitoring system based on image recognition in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable persons skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0026] Example 1

[0027] See also Figure 1 , a moving target monitoring system based on image recognition, comprising:

[0028] A surveillance video acquisition module is used to acquire surveillance videos of a specified area;

[0029] The picture set generation and extraction module is used to divide the surveillance video into frames, and form a picture set from all frames. The pictures in the picture set are numbered in chronological order and recorded as the test picture F. n , where n = 1, 2, 3...N, N is the total number of frames in the picture set, and the picture to be tested F n is the video frame corresponding to the nth moment;

[0030] The test picture preprocessing module is used to convert the test picture F n Perform preprocessing, which includes filtering and grayscale processing;

[0031] Image recognition model management module, used to establish and store image recognition models, which are used to identify moving targets and use the YOLO-V3 algorithm;

[0032] A moving target monitoring model management module is used to update and store a moving target monitoring model, where the moving target monitoring model is established based on a Gaussian mixture model;

[0033] The motion region prior determination module is used to determine the image F to be tested based on the image recognition model and training set. n The movement area in

[0034] The moving target marking module is used to detect the image F according to the moving target monitoring model. n The moving area in the image is used to monitor the moving target and mark the positioning rectangular frame where the moving target is located;

[0035] The moving target area generation module is used to generate a moving target area according to the positioning rectangle frame in the corresponding image to be tested F n Segment the moving target area;

[0036] The moving target recognition module is used to perform image recognition on the moving target area according to the moving target area and the image recognition model, and determine whether the moving target area contains a preset moving target.

[0037] For the motion region prior determination module, since the Gaussian mixture model is used to determine the image F n When matching, each pixel point needs to be matched, but in fact the image F n Not all places will have moving targets. Some areas are always background. All matching operations for these areas are invalid, which is a waste of computing power. Therefore, we can now use the image F to n The moving area where the moving target may exist is extracted, and the moving area is determined by the moving area prior determination module to perform the following: obtain a training set, which is a set of pictures formed by framing all the surveillance videos acquired in a monitoring cycle; use the YOLO-V3 algorithm to perform picture recognition on the pictures in the training set, and output the moving target rectangular frame; add all the coordinates in the moving target rectangular frame to the moving target coordinate set, remove the duplicate coordinates in the moving target coordinate set, and then select the maximum value of the ordinate in the moving target coordinate set, recorded as y p-max , select the minimum value of the horizontal coordinate from the moving target coordinate set, denoted as x p-min , select the maximum value of the ordinate from the moving target coordinate set, recorded as y p-min , with the upper left corner coordinate (x p-min ,y p-min ) and the lower right corner coordinate (x p-max ,y p-max )Establish exercise areas.

[0038] The present invention monitors moving targets by extracting moving areas where moving targets may exist, thereby avoiding invalid calculations for areas where no moving targets have appeared during subsequent moving target monitoring, saving computing power, and improving the monitoring efficiency of moving targets.

[0039] For the motion target monitoring model management module, the establishment of the motion target monitoring model includes the following contents: for the image to be tested F n For each pixel point M in the K Gaussian distribution layer, the Gaussian distribution layer has a built-in Gaussian distribution function η(M n ,μ i,n ,∑ i,n )=1 / ((2π) (i / 2) ·|∑ i,n | 0.5 )·exp((1 / 2∑ i,n )(M n -μ i,n ) T (M n -μ i,n )), assign weight W to the i-th Gaussian distribution layer at time n i,n , where M n is the probability density of pixel M at time n, μ i,n is the mean value corresponding to the i-th Gaussian distribution layer at time n, ∑ i,n is the covariance corresponding to the i-th Gaussian distribution layer at time n, and ∑ i,n =σ i,n I,σ i,n is the standard deviation corresponding to the i-th Gaussian distribution layer at time n, and I is the unit matrix;

[0040] For the moving target marking module, according to the moving target monitoring model, the image F to be tested is n The moving target monitoring in the moving area includes the following steps: n The motion area in the moving target detection model is sent to the moving target detection model, and the moving target detection model is used for the image to be tested. n Each pixel point M in the motion area is matched with its corresponding K Gaussian distribution layers, and the matching condition is “|D n (x,y)-μ i,n |<2.5×σ i,n-1 "Is it true, where D n (x, y) is the pixel value corresponding to the pixel point M at the nth moment, σ i,n-1For this Gaussian distribution layer, the standard deviation corresponding to the previous moment is used. If a match is successful, the mean, standard deviation and weight of the successfully matched Gaussian distribution layer are updated, and the weight of the unmatched Gaussian distribution layer is updated. If a match is not successful, a new Gaussian distribution layer is established based on this pixel point M to replace the Gaussian distribution layer with the smallest weight in the original K Gaussian distribution layers. The specific setting of the learning rate and the update rate and the update of the Gaussian distribution layer are mature existing technologies and will not be described in detail here. Then, according to the priority φ i,n =W i,n / |∑ i,n | Sort all Gaussian distribution layers, select the first B Gaussian distribution layers as the background layer, and then sequentially put the image F n Each pixel point M in the motion area in the image is matched with the background layer. If the match is successful, it means that the pixel point is a background point. If the match fails, it means that the pixel point is a foreground point. The coordinates corresponding to the foreground point are calculated through the connected domain, and the coordinates corresponding to each generated connected domain are added to the foreground point coordinate set. For each foreground point coordinate set, the following content is performed: Select the maximum horizontal coordinate from the foreground point coordinate set, recorded as x q-max , select the maximum ordinate from the foreground point coordinate set, recorded as y q-max , select the minimum value of the horizontal coordinate from the previous scenic spot coordinate set, denoted as x q-min , select the maximum ordinate from the foreground point coordinate set, recorded as y q-min , with the upper left corner coordinate (x q-min ,y q-min ) and the lower right corner coordinate (x q-max ,y q-max )Create a positioning rectangle.

[0041] The present invention establishes a positioning rectangular frame through a Gaussian mixture model to pre-position the moving target, which is convenient for subsequent accurate monitoring of the moving target through image recognition. Moreover, since the positioning rectangular frame has fewer pixels than the entire image to be tested, the efficiency of image recognition is also higher.

[0042] For the moving target recognition module, considering that when monitoring moving targets, the moving targets may be affected by factors such as light, wind, and movement of irrelevant objects, such as the movement of plastic bags on the ground with the wind, the moving target recognition module performs image recognition on the moving target area, including the following contents: sending the moving target area to the image recognition model, judging whether the moving target area is a preset moving target, if there is a moving target in the moving target area, outputting the marked rectangular frame corresponding to the moving target, and mapping the marked rectangular frame to the corresponding image to be tested F n , to realize the monitoring of moving targets.

[0043] It should be noted that the image recognition model uses the YOLO-V3 algorithm and is trained using a dataset with marked moving target types, such as cars and pedestrians. The dataset is randomly selected from different frames in the surveillance video.

[0044] The present invention realizes monitoring of a moving target by performing image recognition on the moving target within a positioning rectangular frame; and the moving target can be monitored more intuitively by marking a rectangular frame on a picture to be tested.

[0045] Example 2

[0046] like Figure 1 As shown, the system also includes a moving target segmented video generation module, which is used for the image F to be tested in which there are continuous moving targets in the image set. n Convert it into moving target segmented video and output it. In special cases, such as tracking down people, users need to view the surveillance video by themselves. However, in the entire surveillance video, only the information corresponding to the moving target is valid, and the video corresponding to the moving target may be only a few in the entire surveillance video. If you watch the entire surveillance video directly, it will waste a lot of time. By viewing the output moving target segmented video, you can get effective information more quickly.

[0047] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A moving target monitoring system based on image recognition, characterized in that: include: A surveillance video acquisition module is used to acquire surveillance videos of a specified area; The picture set generation and extraction module is used to divide the surveillance video into frames, and form a picture set from all frames. The pictures in the picture set are numbered in chronological order and recorded as the test picture F. n , where n = 1, 2, 3...N, N is the total number of frames in the picture set, and the picture to be tested F n is the video frame corresponding to the nth moment; The test picture preprocessing module is used to convert the test picture F n Perform pre-processing; An image recognition model management module is used to establish and store image recognition models, and the image recognition models are used to identify moving targets; A moving target monitoring model management module is used to update and store a moving target monitoring model, where the moving target monitoring model is established based on a Gaussian mixture model; The motion region prior determination module is used to determine the image F to be tested based on the image recognition model and training set. n The moving area in the moving target marking module is used to detect the moving target according to the moving target monitoring model. n The moving area in the image is used to monitor the moving target and mark the positioning rectangular frame where the moving target is located; The moving target area generation module is used to generate a moving target area according to the positioning rectangle frame in the corresponding image to be tested F n Segment the moving target area; The moving target recognition module is used to perform image recognition on the moving target area according to the moving target area and the image recognition model, and determine whether the moving target area contains a preset moving target.

2. The moving target monitoring system based on image recognition according to claim 1, characterized in that: The motion region is determined by the motion region prior determination module to perform the following: obtain a training set, which is a set of pictures formed by dividing all surveillance videos obtained in a monitoring cycle into frames; perform image recognition on the pictures in the training set through an image recognition model, and output a moving target rectangular frame; Add all coordinates in the moving target rectangular box to the moving target coordinate set, remove the duplicate coordinates in the moving target coordinate set, and then select the maximum value of the ordinate in the moving target coordinate set, recorded as y p-max , select the minimum value of the horizontal coordinate from the moving target coordinate set, recorded as x p-min , select the maximum value of the ordinate from the moving target coordinate set, recorded as y p-min , with the upper left corner coordinate (x p-min ,y p-min ) and the lower right corner coordinate (x p-max , y p-max )Establish exercise areas.

3. The moving target monitoring system based on image recognition according to claim 2 is characterized in that: The target monitoring model management module has a built-in moving target monitoring model. The establishment of the moving target monitoring model includes the following contents: n For each pixel point M in , K Gaussian distribution layers are set, and the Gaussian distribution layer has a built-in Gaussian distribution function.

4. The moving target monitoring system based on image recognition according to claim 3 is characterized in that: According to the motion target monitoring model, the image F to be tested n The moving target monitoring in the moving area includes the following steps: n The motion area in the moving target detection model is sent to the moving target detection model, and the moving target detection model is used for the image to be tested. n Each pixel point M in the motion area in the image is matched with its corresponding K Gaussian distribution layers. If a match is successful, the mean, standard deviation and weight of the successfully matched Gaussian distribution layer are updated, and the weight of the unmatched Gaussian distribution layer is updated. If a match is not successful, a new Gaussian distribution layer is established based on the pixel point M to replace the Gaussian distribution layer with the smallest weight among the original K Gaussian distribution layers; then all Gaussian distribution layers are sorted, the first B Gaussian distribution layers are selected as the background layer, and then the image to be tested F is sequentially placed on the image. n Each pixel point M in the motion area in the image is matched with the background layer. If the match is successful, it means that the pixel point is a background point. If the match fails, it means that the pixel point is a foreground point. The coordinates corresponding to the foreground point are calculated through the connected domain, and the coordinates corresponding to each generated connected domain are added to the foreground point coordinate set. For each foreground point coordinate set, the following content is performed: Select the maximum horizontal coordinate from the foreground point coordinate set, recorded as x q-max , select the maximum ordinate from the foreground point coordinate set, recorded as y q-max , select the minimum value of the horizontal coordinate from the previous scenic spot coordinate set, denoted as x q-min , select the maximum ordinate from the foreground point coordinate set, recorded as y q-min , with the upper left corner coordinate (x q-min ,y q-min ) and the lower right corner coordinate (x q-max ,y q-max )Create a positioning rectangle.

5. The moving target monitoring system based on image recognition according to claim 4 is characterized in that: The moving target area is image recognized by the moving target recognition module, including the following contents: the moving target area is sent to the image recognition model, and it is determined whether the moving target area is a preset moving target. If there is a moving target in the moving target area, a marked rectangular frame corresponding to the moving target is output, and the marked rectangular frame is mapped to the corresponding image to be tested F n .

6. The moving target monitoring system based on image recognition according to claim 5, characterized in that: It also includes a moving target segmented video generation module for the image F to be tested in which there are continuous moving targets in the image set. n Convert to moving target segmented video and output it.

7. The moving target monitoring system based on image recognition according to claim 6, characterized in that: The image recognition model uses the YOLO-V3 algorithm.

Images