Multidirectional identification video monitoring system

Abstract

The invention relates to the technical field of video monitoring, in particular to a multi-azimuth identification video monitoring system, which comprises a data acquisition module, a data analysis module and a result analysis module, and is characterized in that a picture of a monitoring area is acquired through the data acquisition module, then a target detection model in the data analysis module is used for detecting the picture, and a result is obtained through the result analysis module; determining a detection frame of a target position and a detection result thereof, wherein the detection result comprises a position, a size, a category and confidence; the result analysis module ranks all the detection frames from high to low according to confidence coefficients, selects the detection frame with the highest confidence coefficient as the current optimal detection frame, and calculates a dynamic overlap suppression threshold value according to the position, size and category of the current optimal detection frame; and then, calculating the intersection-to-union ratio of the other detection frames and the current optimal detection frame, gradually adjusting the confidence coefficients of the other detection frames according to the overlap suppression threshold value and the intersection-to-union ratio, and repeating the steps after reordering until all the detection frames are processed.

Description

Technical Field

[0001] The present invention relates to the technical field of video monitoring, in particular to a multi-directional recognition video monitoring system. Background Art

[0002] In traditional multi-directional recognition video surveillance systems, a fixed overlap suppression threshold is usually used in the target detection process. This method has obvious technical problems when dealing with targets of different sizes and categories.

[0003] First, a fixed overlap suppression threshold cannot flexibly cope with the characteristics of different targets. For small targets (such as pedestrians, bicycles, etc.), a higher overlap suppression threshold may mistakenly suppress the true target detection frame, resulting in missed detection problems, especially in surveillance areas with dense targets. Second, for large targets (such as vehicles, buildings, etc.), a lower overlap suppression threshold may retain too many overlapping detection frames, thereby increasing the probability of false detection and making the final detection result inaccurate.

[0004] In addition, the fixed overlap suppression threshold has poor adaptability in different monitoring scenarios and is difficult to maintain stable performance in various complex environments. For example, under different lighting, angles and background conditions, the detection frame of the target may change greatly, and the fixed overlap suppression threshold cannot effectively cope with these changes. Therefore, the traditional fixed overlap suppression threshold often leads to a decrease in detection accuracy and reliability in multi-target and multi-scale video surveillance scenarios, and cannot meet the requirements of high accuracy and low false detection rate in practical applications. Summary of the invention

[0005] The purpose of the present invention is to provide a multi-directional recognition video monitoring system to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a video surveillance system for multi-directional recognition, comprising a data acquisition module, a data analysis module and a result analysis module, wherein:

[0007] The data acquisition module acquires a monitoring area image;

[0008] The data analysis module detects the monitoring area picture using the target detection model to determine the detection frame of the target position in the monitoring area picture and the corresponding detection result, wherein the detection result includes the position, size, category and confidence level;

[0009] The result analysis module sorts all the detection frames detected in the monitoring area picture from high to low according to the confidence, and selects the detection frame with the highest confidence as the current best detection frame, and calculates a dynamic overlap suppression threshold according to the position, size and category in the current best detection frame; and calculates the intersection and union ratio of the remaining detection frames and the current best detection frame, and gradually adjusts the confidence of the remaining detection frames according to the calculated overlap suppression threshold and intersection and union ratio, reorders the adjusted detection frames, and repeats the above steps until all detection frames are processed.

[0010] As a further improvement of the technical solution, the data acquisition module uses a camera to acquire a picture of the monitoring area, and the picture of the monitoring area includes multiple targets, wherein the target represents a moving object.

[0011] As a further improvement of the technical solution, the process of the data analysis module determining the detection frame of the target position in the monitoring area picture specifically includes:

[0012] A real-time target detection model based on a convolutional neural network is used as the target detection model, and the monitoring area image is input into the target detection model;

[0013] Extract a multi-scale feature map of the image and generate multiple detection boxes on the multi-scale feature map. Each detection box has a default position, size and scale.

[0014] Perform bounding box regression on each detection box to adjust its position and size. The bounding box regression adjusts the position and size of the detection box by predicting the center point coordinates, width, and height.

[0015] Classify each detection box, determine the category of the target, and output the probability that each detection box belongs to each category;

[0016] Calculate the target existence confidence of each detection frame. The confidence score is calculated based on the classification probability and object existence probability of the detection frame. The classification probability indicates the probability that the target object in the detection frame belongs to a certain category, and the object existence probability indicates the probability of whether there is a target object in the detection frame. The confidence of the detection frame is obtained by multiplying the probability of the classification probability and the probability of the object existence.

[0017] Output the detection results of each detection box, including position, size, category and confidence.

[0018] As a further improvement of the technical solution, the result analysis module includes a dynamic threshold calculation unit, and the process of calculating the overlap suppression threshold by the dynamic threshold calculation unit specifically includes:

[0019] Sort all detection boxes by confidence from high to low. This step is to ensure that the most likely target detection box is processed first. The sorted detection box list is recorded as B1, B2, B3, ..., Bn, where B1 has the highest confidence.

[0020] Select the detection frame B1 with the highest confidence from the sorted detection frame list as the current best detection frame;

[0021] According to the category of the current best detection box B1, a dynamic overlap suppression threshold is calculated. Different categories and sizes require different overlap suppression thresholds, and the calculation method is based on the following rules:

[0022] If the category of the current best detection box B1 belongs to the small target category, select a threshold output function with an output value lower than 0.5, and output the overlap suppression threshold according to the size of the current best detection box;

[0023] If the category of the current best detection box B1 belongs to the large target category, a threshold output function with an output value higher than 0.5 is selected, and the overlap suppression threshold is output according to the size of the current best detection box.

[0024] As a further improvement of the technical solution, the result analysis module includes a confidence adjustment unit, and the confidence adjustment unit adjusts the confidence of the detection frame in a process specifically including:

[0025] For the current best detection frame B1, calculate the intersection-and-union ratio of the remaining detection frames B2, B3, ..., Bn with B1. The intersection-and-union ratio is a method to measure the degree of overlap between two detection frames. Its calculation formula is:

[0026] Where Bi represents the remaining detection boxes, i = 2, 3, ..., n;

[0027] According to the calculated intersection-over-union ratio IoU(Bi,B1) and the overlap suppression threshold, the confidence of the remaining detection boxes Bi is gradually adjusted. The adjustment formula is as follows:

[0028] Where exp represents the exponential function, confold(Bi) represents the confidence of the unadjusted detection box, confnew(Bi) represents the confidence of the adjusted detection box, and σ represents the adjustment parameter;

[0029] Re-sort the adjusted detection frames from high to low according to the confidence level to obtain a new detection frame list B1′, B′2, B′3, ..., B′n; from the re-sorted new detection frame list, select the detection frame with the highest new confidence level as the current best detection frame, and repeat the process of the dynamic threshold calculation unit and the confidence adjustment unit until all detection frames have been processed.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] This multi-directional recognition video surveillance system dynamically calculates the overlap suppression threshold. The video surveillance system can flexibly respond to the characteristics of different targets. For small targets (such as pedestrians, bicycles, etc.), the system selects a lower overlap suppression threshold to avoid erroneously suppressing the real target detection frame, thereby significantly reducing the problem of missed detection, especially in monitoring areas with dense targets. For large targets (such as vehicles, buildings, etc.), the system selects a higher overlap suppression threshold to effectively suppress excessive overlapping detection frames, reduce false detections, and make the final detection results more accurate.

[0032] In addition, the dynamic overlap suppression threshold method of the system is more adaptable in different monitoring scenarios and can maintain stable performance in various complex environments. For example, under different lighting, angle and background conditions, the detection frame of the target may change greatly, but the dynamic threshold calculation unit can flexibly adjust the overlap suppression threshold according to the category and size of the current optimal detection frame to ensure that the confidence adjustment of the detection frame is more reasonable. Therefore, the system not only improves the detection accuracy and reliability, but also meets the requirements of high accuracy and low false detection rate in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the overall module of the present invention;

[0034] Figure 2 It is a schematic diagram of the result analysis module unit of the present invention.

[0035] In the figure: 100, data acquisition module; 200, data analysis module; 300, result analysis module; 301, dynamic threshold calculation unit; 302, confidence adjustment unit. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0037] Next, see Figure 1-Figure 2 The present invention provides a technical solution: a multi-directional recognition video surveillance system, including a data acquisition module 100, a data analysis module 200 and a result analysis module 300.

[0038] The data acquisition module 100 acquires a picture of the monitoring area by setting a camera in the monitoring area, using the camera to acquire the picture of the monitoring area, and cropping the image according to the area to be monitored to remove irrelevant background areas; scaling the cropped image to the fixed size required by the target detection model; normalizing the pixel values ​​of the image to the range of [0,1] or [0,255] to meet the input requirements of the target detection model; wherein the picture of the monitoring area includes but is not limited to the background and the target, such as the background of the entrance road, sidewalk, signboard and green belt of the parking lot; the target represents a moving object, specifically a vehicle, pedestrian, bicycle, etc.

[0039] The data analysis module 200 detects the monitoring area image using the target detection model, determines the detection frame of the target position in the monitoring area image and the corresponding detection result, wherein the detection result includes the position, size, category and confidence, specifically including:

[0040] A real-time target detection model based on a convolutional neural network is used as the target detection model, and the monitoring area image is input into the target detection model;

[0041] Extract a multi-scale feature map of the image, which can capture objects of different sizes and details in the image;

[0042] Generate multiple detection boxes on the multi-scale feature map, each of which has a default position, size, and scale. The default value is obtained based on the target size and position statistics in the dataset during training.

[0043] Perform bounding box regression on each detection box to adjust its position and size so that it fits the target more accurately. Bounding box regression adjusts the position and size of the detection box by predicting the center point coordinates and width and height.

[0044] Classify each detection box, determine the category of the target, and output the probability that each detection box belongs to each category;

[0045] Calculate the target existence confidence of each detection frame. The confidence score combines the classification probability and object existence probability of the detection frame to finally get a comprehensive confidence score. The classification probability indicates the probability that the target object in the detection frame belongs to a certain category, and the object existence probability indicates the probability of whether there is a target object in the detection frame. The confidence of the detection frame is obtained by multiplying the probability of the classification probability and the probability of the object existence.

[0046] Output the detection results of each detection box, including position, size, category and confidence.

[0047] The dynamic threshold calculation unit 301 in the result analysis module 300 sorts all the detection frames detected in the monitoring area image from high to low according to the confidence, and selects the detection frame with the highest confidence as the current best detection frame, and calculates a dynamic overlap suppression threshold according to the position, size and category of the current best detection frame, specifically including:

[0048] Sort all detection boxes by confidence from high to low. This step is to ensure that the most likely target detection box is processed first. The sorted detection box list is recorded as B1, B2, B3, ..., Bn, where B1 has the highest confidence.

[0049] Select the detection frame B1 with the highest confidence from the sorted detection frame list as the current best detection frame;

[0050] According to the category of the current best detection box B1, a dynamic overlap suppression threshold is calculated. Different categories and sizes require different overlap suppression thresholds, and the calculation method is based on the following rules:

[0051] If the category of the current best detection box B1 belongs to the small target category, select a threshold output function with an output value lower than 0.5, and output the overlap suppression threshold according to the size of the current best detection box;

[0052] If the category of the current best detection box B1 belongs to the large target category, a threshold output function with an output value higher than 0.5 is selected, and the overlap suppression threshold is output according to the size of the current best detection box.

[0053] By dynamically calculating the overlap suppression threshold based on category and size, overlapping detection frames can be reduced more effectively to avoid repeated detection of the same target. For example, for small targets (such as pedestrians and bicycles), a lower overlap suppression threshold can retain multiple possible detection frames, while for large targets (such as vehicles), a higher overlap suppression threshold can more strictly suppress overlapping detection frames.

[0054] The confidence adjustment unit 302 in the result analysis module 300 calculates the intersection-over-union ratio of the remaining detection frames and the current best detection frame, gradually adjusts the confidence of the remaining detection frames according to the calculated overlap suppression threshold and the intersection-over-union ratio, reorders the adjusted detection frames, and repeats the above steps until all detection frames are processed, specifically including:

[0055] For the current best detection frame B1, calculate the intersection-and-union ratio of the remaining detection frames B2, B3, ..., Bn with B1. The intersection-and-union ratio is a method to measure the degree of overlap between two detection frames. Its calculation formula is:

[0056] Where Bi represents the remaining detection boxes, i = 2, 3, ..., n;

[0057] According to the calculated intersection-over-union ratio IoU(Bi,B1) and the overlap suppression threshold, the confidence of the remaining detection frames Bi is gradually adjusted. If the intersection-over-union ratio IoU(Bi,B1) is greater than the overlap suppression threshold, the current detection frame is adjusted; otherwise, the confidence remains unchanged; the adjustment formula is as follows:

[0058] Where exp represents the exponential function, confold(Bi) represents the confidence of the unadjusted detection box, confnew(Bi) represents the confidence of the adjusted detection box, and σ represents the adjustment parameter;

[0059] The adjusted detection frames are re-sorted from high to low according to the confidence level to obtain a new detection frame list; from the re-sorted new detection frame list, the detection frame with the highest new confidence level is selected as the current best detection frame, and the processes of the dynamic threshold calculation unit 301 and the confidence adjustment unit 302 are repeated until the intersection-over-union ratios of all detection frames are less than the overlap suppression threshold.

[0060] A confidence threshold is set, and all detection boxes below the confidence threshold are deleted, and detection boxes above the confidence threshold are retained; the adjustment, deletion, and increase of confidence allow those detection boxes with higher confidence to be retained, and those detection boxes with lower confidence to be suppressed. This not only reduces false detections, but also improves the accuracy of detection. For example, if a detection box has a high intersection-over-union ratio with the current best detection box, but has a low confidence, then its confidence will be further reduced, making it less likely to be retained. By gradually processing and adjusting the detection boxes, the result analysis module 300 can effectively reduce unnecessary calculations and processing, and improve the processing efficiency of the system.

[0061] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A video surveillance system with multi-directional recognition, characterized in that: It comprises a data acquisition module (100), a data analysis module (200) and a result analysis module (300), wherein: The data acquisition module (100) is used to acquire a monitoring area picture, wherein the monitoring area picture contains a target; At the same time, a detection frame that matches the target size is generated according to the target's position in the monitoring area image; The data analysis module (200) is used to determine the confidence level of the detection frame; The result analysis module (300) is used to analyze the overlap suppression threshold of the detection frame and adjust the confidence of the detection frame according to the overlap suppression threshold; Among them, after the confidence adjustment of all detection frames is completed, the detection frames with confidence higher than the confidence threshold are retained, and the detection frames with confidence lower than the confidence threshold are deleted.

2. The multi-directional video surveillance system according to claim 1, characterized in that: The data acquisition module (100) uses a camera to acquire a monitoring area picture, wherein the monitoring area picture includes a plurality of targets, wherein the targets represent moving objects.

3. The multi-directional video surveillance system according to claim 2, characterized in that: The process of the data analysis module (200) determining the detection frame of the target position in the monitoring area picture specifically includes: A real-time target detection model based on a convolutional neural network is used as the target detection model, and the monitoring area image is input into the target detection model; Extract a multi-scale feature map of the image and generate multiple detection boxes on the multi-scale feature map. Each detection box has a default position, size and scale. Perform bounding box regression on each detection box to adjust its position and size. The bounding box regression adjusts the position and size of the detection box by predicting the center point coordinates, width, and height. Classify each detection box, determine the category of the target, and output the probability that each detection box belongs to each category; Calculate the target existence confidence of each detection frame. The confidence score is calculated based on the classification probability and object existence probability of the detection frame. The classification probability indicates the probability that the target object in the detection frame belongs to a certain category, and the object existence probability indicates the probability of whether there is a target object in the detection frame. The confidence of the detection frame is obtained by multiplying the probability of the classification probability and the probability of the object existence. Output the detection results of each detection box, including position, size, category and confidence.

4. The multi-directional video surveillance system according to claim 3, characterized in that: The result analysis module (300) comprises a dynamic threshold calculation unit (301), and the process of calculating the overlap suppression threshold by the dynamic threshold calculation unit (301) specifically comprises: Sort all detection boxes by confidence from high to low. This step is to ensure that the most likely target detection box is processed first. The sorted detection box list is recorded as B1, B2, B3, ..., Bn, where B1 has the highest confidence. Select the detection frame B1 with the highest confidence from the sorted detection frame list as the current best detection frame; According to the category of the current best detection box B1, a dynamic overlap suppression threshold is calculated. Different categories and sizes require different overlap suppression thresholds, and the calculation method is based on the following rules: If the category of the current best detection box B1 belongs to the small target category, select a threshold output function with an output value lower than 0.5, and output the overlap suppression threshold according to the size of the current best detection box; If the category of the current best detection box B1 belongs to the large target category, a threshold output function with an output value higher than 0.5 is selected, and the overlap suppression threshold is output according to the size of the current best detection box.

5. The multi-directional video surveillance system according to claim 4, characterized in that: The result analysis module (300) comprises a confidence adjustment unit (302), and the confidence adjustment unit (302) adjusts the confidence of the detection frame in a process that specifically comprises: For the current best detection frame B1, calculate the intersection-and-union ratio of the remaining detection frames B2, B3, ..., Bn with B1. The intersection-and-union ratio is a method to measure the degree of overlap between two detection frames. Its calculation formula is: Where Bi represents the remaining detection boxes, i = 2, 3, ..., n; According to the calculated intersection-over-union ratio IoU(Bi,B1) and the overlap suppression threshold, the confidence of the remaining detection frames Bi is gradually adjusted. If the intersection-over-union ratio IoU(Bi,B1) is greater than the overlap suppression threshold, the current detection frame is adjusted; otherwise, the confidence remains unchanged; the adjustment formula is as follows: Where exp represents the exponential function, confold(Bi) represents the confidence of the unadjusted detection box, confnew(Bi) represents the confidence of the adjusted detection box, and σ represents the adjustment parameter; The adjusted detection frames are re-sorted from high to low according to the confidence level to obtain a new detection frame list; from the re-sorted new detection frame list, the detection frame with the highest new confidence level is selected as the current best detection frame, and the processes of the dynamic threshold calculation unit (301) and the confidence adjustment unit (302) are repeated until the intersection-over-union ratios of all detection frames are less than the overlap suppression threshold.

Images