A video surveillance system with multi-directional recognition

Abstract

The present invention relates to the field of video surveillance technology, and specifically to a multi-directional recognition video surveillance system, comprising a data acquisition module, a data analysis module, and a result analysis module. The data acquisition module acquires a picture of a surveillance area, and then uses a target detection model in the data analysis module to detect the picture to determine a detection frame of a target position and its detection results, including position, size, category, and confidence. The result analysis module sorts all detection frames from high to low according to confidence, selects the detection frame with the highest confidence as the current best detection frame, and calculates a dynamic overlap suppression threshold based on its position, size, and category. Subsequently, the intersection-over-union ratio of the remaining detection frames to the current best detection frame is calculated, and the confidence of the remaining detection frames is gradually adjusted based on the overlap suppression threshold and the intersection-over-union ratio. After re-sorting, the above steps are repeated until all detection frames have been processed.

Description

Technical Field

[0001] The present invention relates to the technical field of video surveillance, and in particular to a multi-directional recognition video surveillance system. Background Art

[0002] In traditional multi-directional recognition video surveillance systems, a fixed overlap suppression threshold is usually used in the target detection process. This method has obvious technical problems when dealing with targets of different sizes and categories.

[0003] First, a fixed overlap suppression threshold cannot flexibly adapt to the characteristics of different targets. For small targets (such as pedestrians and bicycles), a higher overlap suppression threshold may mistakenly suppress the true target detection frame, resulting in missed detections, especially in densely populated surveillance areas. Second, for large targets (such as vehicles and buildings), a lower overlap suppression threshold may retain too many overlapping detection frames, increasing the probability of false detections and making the final detection results less accurate.

[0004] Furthermore, fixed overlap suppression thresholds have poor adaptability across diverse surveillance scenarios, making it difficult to maintain stable performance in complex environments. For example, under varying lighting, angles, and background conditions, the target's detection bounds can vary significantly, and fixed overlap suppression thresholds cannot effectively address these variations. Consequently, traditional fixed overlap suppression thresholds often lead to reduced detection accuracy and reliability in multi-target, multi-scale video surveillance scenarios, failing to meet the high precision and low false detection rate requirements of practical applications. Summary of the Invention

[0005] The purpose of the present invention is to provide a multi-directional recognition video surveillance system to solve the problems raised in the above background technology.

[0006] To achieve the above-mentioned object, the present invention provides the following technical solution: a video surveillance system for multi-directional recognition, comprising a data acquisition module, a data analysis module and a result analysis module, wherein:

[0007] The data acquisition module acquires a picture of the monitoring area;

[0008] The data analysis module uses the target detection model to detect the monitoring area image, determine the detection frame of the target position in the monitoring area image and the corresponding detection result, wherein the detection result includes position, size, category and confidence level;

[0009] The result analysis module sorts all detection frames detected in the monitoring area image from high to low according to confidence, selects the detection frame with the highest confidence as the current best detection frame, calculates a dynamic overlap suppression threshold based on the position, size and category of the current best detection frame, and calculates the intersection-over-union ratio of the remaining detection frames to the current best detection frame. Based on the calculated overlap suppression threshold and intersection-over-union ratio, the confidence of the remaining detection frames is gradually adjusted, and the adjusted detection frames are reordered. The above steps are repeated until all detection frames are processed.

[0010] As a further improvement of the present technical solution, the data acquisition module uses a camera to acquire a picture of the monitoring area, and the picture of the monitoring area includes multiple targets, wherein the target represents a moving object.

[0011] As a further improvement of the present technical solution, the process of the data analysis module determining the detection frame of the target position in the monitoring area image specifically includes:

[0012] A real-time target detection model based on a convolutional neural network is used as the target detection model, and the monitoring area image is input into the target detection model;

[0013] Extract the multi-scale feature map of the image and generate multiple detection boxes on the multi-scale feature map. Each detection box has a default position, size and scale.

[0014] Perform bounding box regression on each detection box to adjust its position and size. The bounding box regression adjusts the position and size of the detection box by predicting the center point coordinates, width, and height.

[0015] Classify each detection box, determine the category of the target, and output the probability that each detection box belongs to each category;

[0016] Calculate the object existence confidence of each detection box. The confidence score is calculated based on the classification probability and object existence probability of the detection box. The classification probability represents the probability that the target object in the detection box belongs to a certain category, and the object existence probability represents the probability of whether the target object exists in the detection box. The confidence score of the detection box is obtained by multiplying the classification probability and the object existence probability.

[0017] Output the detection results of each detection box, including position, size, category and confidence.

[0018] As a further improvement of the present technical solution, the result analysis module includes a dynamic threshold calculation unit, and the process of calculating the overlap suppression threshold by the dynamic threshold calculation unit specifically includes:

[0019] Sort all detection boxes by confidence from high to low. This step is to ensure that the most likely target detection box is processed first. The sorted detection box list is recorded as B1, B2, B3, ..., Bn, where B1 has the highest confidence.

[0020] Select the detection frame B1 with the highest confidence from the sorted detection frame list as the current best detection frame;

[0021] According to the category of the current best detection box B1, a dynamic overlap suppression threshold is calculated. Different categories and sizes require different overlap suppression thresholds. The calculation method is based on the following rules:

[0022] If the category of the current best detection box B1 belongs to the small target category, select the threshold output function with an output value lower than 0.5, and output the overlap suppression threshold according to the size of the current best detection box;

[0023] If the category of the current best detection box B1 belongs to the large target category, a threshold output function with an output value higher than 0.5 is selected, and the overlap suppression threshold is output according to the size of the current best detection box.

[0024] As a further improvement of the present technical solution, the result analysis module includes a confidence adjustment unit, and the process of adjusting the confidence of the detection frame by the confidence adjustment unit specifically includes:

[0025] For the current best detection frame B1, calculate the intersection-over-union (IoU) of the remaining detection frames B2, B3, ..., Bn with B1. The IoU is a method to measure the degree of overlap between two detection frames. Its calculation formula is:

[0026] Where Bi represents the remaining detection boxes, i = 2, 3, ..., n;

[0027] According to the calculated intersection-over-union ratio IoU(Bi,B1) and the overlap suppression threshold, the confidence of the remaining detection boxes Bi is gradually adjusted. The adjustment formula is as follows:

[0028] Where exp represents the exponential function, confold(Bi) represents the confidence of the unadjusted detection box, confnew(Bi) represents the confidence of the adjusted detection box, and σ represents the adjustment parameter;

[0029] The adjusted detection frames are re-sorted from high to low according to confidence to obtain a new detection frame list B1′, B′2, B′3, ..., B′n; from the re-sorted new detection frame list, the detection frame with the highest new confidence is selected as the current best detection frame, and the process of the dynamic threshold calculation unit and the confidence adjustment unit is repeated until all detection frames have been processed.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] This multi-directional recognition video surveillance system dynamically calculates the overlap suppression threshold, which can flexibly respond to the characteristics of different targets. For small targets (such as pedestrians and bicycles), the system selects a lower overlap suppression threshold to avoid mistakenly suppressing the true target detection frame, thereby significantly reducing missed detection problems, especially in monitoring areas with dense targets. For large targets (such as vehicles and buildings), the system selects a higher overlap suppression threshold to effectively suppress excessive overlapping detection frames, reduce false detections, and make the final detection results more accurate.

[0032] In addition, the system's dynamic overlap suppression threshold method is more adaptable in different monitoring scenarios and can maintain stable performance in various complex environments. For example, under different lighting, angle and background conditions, the target detection frame may change significantly, but the dynamic threshold calculation unit can flexibly adjust the overlap suppression threshold according to the category and size of the current optimal detection frame to ensure that the confidence adjustment of the detection frame is more reasonable. Therefore, the system not only improves detection accuracy and reliability, but also meets the requirements of high precision and low false detection rate in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the overall module of the present invention;

[0034] Figure 2 Schematic diagram of the result analysis module unit of the present invention.

[0035] In the figure: 100, data acquisition module; 200, data analysis module; 300, result analysis module; 301, dynamic threshold calculation unit; 302, confidence adjustment unit. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0037] Next, see Figure 1-Figure 2 The present invention provides a technical solution: a multi-directional recognition video surveillance system, including a data acquisition module 100, a data analysis module 200 and a result analysis module 300.

[0038] The data acquisition module 100 obtains a picture of the monitoring area by setting a camera in the monitoring area, obtaining the picture of the monitoring area using the camera, and cropping the image according to the area to be monitored to remove irrelevant background areas; scaling the cropped image to the fixed size required by the target detection model; normalizing the pixel values of the image to the range of [0,1] or [0,255] to meet the input requirements of the target detection model; the picture of the monitoring area includes but is not limited to the background and the target, such as the background of the entrance road, sidewalk, signboard and green belt of the parking lot; the target represents a moving object, specifically a vehicle, pedestrian, bicycle, etc.

[0039] The data analysis module 200 uses the target detection model to detect the monitoring area image, determine the detection frame of the target position in the monitoring area image and the corresponding detection result, where the detection result includes the position, size, category and confidence level, specifically including:

[0040] A real-time target detection model based on a convolutional neural network is used as the target detection model, and the monitoring area image is input into the target detection model;

[0041] Extract a multi-scale feature map of the image, which can capture objects of different sizes and details in the image;

[0042] Generate multiple detection boxes on the multi-scale feature map, each with a default position, size, and scale. The default values are obtained during training based on the target size and position statistics in the dataset.

[0043] Perform bounding box regression on each detection box to adjust its position and size so that it fits the target more accurately. Bounding box regression adjusts the position and size of the detection box by predicting the center point coordinates, width, and height.

[0044] Classify each detection box, determine the category of the target, and output the probability that each detection box belongs to each category;

[0045] Calculate the object existence confidence of each detection box. The confidence score combines the classification probability and object existence probability of the detection box to obtain a comprehensive confidence score. The classification probability represents the probability that the target object in the detection box belongs to a certain category, and the object existence probability represents the probability of whether the target object exists in the detection box. The confidence score of the detection box is obtained by multiplying the classification probability and the object existence probability.

[0046] Output the detection results of each detection box, including position, size, category and confidence.

[0047] The dynamic threshold calculation unit 301 in the result analysis module 300 sorts all the detection frames detected in the monitoring area image from high to low according to their confidence, selects the detection frame with the highest confidence as the current best detection frame, and calculates a dynamic overlap suppression threshold based on the position, size, and category of the current best detection frame, specifically including:

[0048] Sort all detection boxes by confidence from high to low. This step is to ensure that the most likely target detection box is processed first. The sorted detection box list is recorded as B1, B2, B3, ..., Bn, where B1 has the highest confidence.

[0049] Select the detection frame B1 with the highest confidence from the sorted detection frame list as the current best detection frame;

[0050] According to the category of the current best detection box B1, a dynamic overlap suppression threshold is calculated. Different categories and sizes require different overlap suppression thresholds. The calculation method is based on the following rules:

[0051] If the category of the current best detection box B1 belongs to the small target category, select the threshold output function with an output value lower than 0.5, and output the overlap suppression threshold according to the size of the current best detection box;

[0052] If the category of the current best detection box B1 belongs to the large target category, a threshold output function with an output value higher than 0.5 is selected, and the overlap suppression threshold is output according to the size of the current best detection box.

[0053] By dynamically calculating the overlap suppression threshold based on category and size, overlapping detection frames can be reduced more effectively to avoid repeated detection of the same target. For example, for small targets (such as pedestrians and bicycles), a lower overlap suppression threshold can retain multiple possible detection frames, while for large targets (such as vehicles), a higher overlap suppression threshold can more strictly suppress overlapping detection frames.

[0054] The confidence adjustment unit 302 in the result analysis module 300 calculates the intersection-over-union (IoU) of the remaining detection frames and the current best detection frame, and gradually adjusts the confidences of the remaining detection frames based on the calculated overlap suppression threshold and IoU. The adjusted detection frames are reordered, and the above steps are repeated until all detection frames are processed. Specifically, the steps include:

[0055] For the current best detection frame B1, calculate the intersection-over-union (IoU) of the remaining detection frames B2, B3, ..., Bn with B1. The IoU is a method to measure the degree of overlap between two detection frames. Its calculation formula is:

[0056] Where Bi represents the remaining detection boxes, i = 2, 3, ..., n;

[0057] According to the calculated intersection-over-union ratio IoU(Bi,B1) and the overlap suppression threshold, the confidence of the remaining detection boxes Bi is gradually adjusted. If the intersection-over-union ratio IoU(Bi,B1) is greater than the overlap suppression threshold, the current detection box is adjusted; otherwise, the confidence remains unchanged. The adjustment formula is as follows:

[0058] Where exp represents the exponential function, confold(Bi) represents the confidence of the unadjusted detection box, confnew(Bi) represents the confidence of the adjusted detection box, and σ represents the adjustment parameter;

[0059] The adjusted detection frames are re-sorted from high to low according to the confidence level to obtain a new detection frame list; from the re-sorted new detection frame list, the detection frame with the highest new confidence level is selected as the current best detection frame, and the processes of the dynamic threshold calculation unit 301 and the confidence adjustment unit 302 are repeated until the intersection-over-union ratios of all detection frames are less than the overlap suppression threshold.

[0060] A confidence threshold is set, and all detection frames below the confidence threshold are deleted, while detection frames above the confidence threshold are retained; the adjustment, deletion, and increase of confidence levels allow detection frames with higher confidence levels to be retained, while detection frames with lower confidence levels are suppressed. This not only reduces false detections, but also improves the accuracy of detection. For example, if a detection frame has a high intersection-over-union ratio with the current best detection frame, but has a low confidence level, its confidence level will be further reduced, making it less likely to be retained. By gradually processing and adjusting the detection frames, the result analysis module 300 can effectively reduce unnecessary calculations and processing, thereby improving the processing efficiency of the system.

[0061] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-directional recognition video surveillance system, characterized in that: It includes a data acquisition module (100), a data analysis module (200) and a result analysis module (300), wherein: The data acquisition module (100) is used to acquire a monitoring area picture, wherein the monitoring area picture contains a target; At the same time, a detection frame that matches the target size is generated based on the target's position in the monitoring area image; The data analysis module (200) is used to determine the confidence level of the detection frame; The result analysis module (300) is used to analyze the overlap suppression threshold of the detection frame and adjust the confidence of the detection frame according to the overlap suppression threshold; Among them, after the confidence adjustment of all detection frames is completed, the detection frames with confidence higher than the confidence threshold are retained, and the detection frames with confidence lower than the confidence threshold are deleted; The result analysis module (300) includes a dynamic threshold calculation unit (301), and the process of calculating the overlap suppression threshold by the dynamic threshold calculation unit (301) specifically includes: Sort all detection boxes by confidence from high to low; the sorted detection box list is recorded as ,in The confidence level is the highest; Select the detection box with the highest confidence from the sorted list of detection boxes As the current best detection frame; Based on the current best detection box A dynamic overlap suppression threshold is calculated for each category and size. Different overlap suppression thresholds are required for different categories and sizes. The calculation method is based on the following rules: If the current best detection frame If the category belongs to the small target category, select the threshold output function with an output value lower than 0.5, and output the overlap suppression threshold according to the size of the current best detection box; If the current best detection frame If the category belongs to the large target category, the threshold output function with an output value higher than 0.5 is selected, and the overlap suppression threshold is output according to the size of the current best detection box.

2. The multi-directional recognition video surveillance system according to claim 1, characterized in that: The data acquisition module (100) uses a camera to acquire a monitoring area picture, wherein the monitoring area picture includes a plurality of targets, wherein the targets represent moving objects.

3. The multi-directional recognition video surveillance system according to claim 2, characterized in that: The process of the data analysis module (200) determining the detection frame of the target position in the monitoring area image specifically includes: A real-time target detection model based on a convolutional neural network is used as the target detection model, and the monitoring area image is input into the target detection model; Extract the multi-scale feature map of the image and generate multiple detection boxes on the multi-scale feature map. Each detection box has a default position, size and scale. Perform bounding box regression on each detection box to adjust its position and size. The bounding box regression adjusts the position and size of the detection box by predicting the center point coordinates, width, and height. Classify each detection box, determine the category of the target, and output the probability that each detection box belongs to each category; Calculate the object existence confidence of each detection box. The confidence score is calculated based on the classification probability and object existence probability of the detection box. The classification probability represents the probability that the target object in the detection box belongs to a certain category, and the object existence probability represents the probability of whether the target object exists in the detection box. The confidence score of the detection box is obtained by multiplying the classification probability and the object existence probability. Output the detection results of each detection box, including position, size, category and confidence.

4. The multi-directional recognition video surveillance system according to claim 1, characterized in that: The result analysis module (300) includes a confidence adjustment unit (302), and the process of adjusting the confidence of the detection frame by the confidence adjustment unit (302) specifically includes: For the current best detection box , calculate the remaining detection boxes and The intersection-over-union ratio is a method to measure the degree of overlap between two detection frames. Its calculation formula is: ,in Represents the remaining detection boxes, ; According to the calculated intersection-over-union ratio and overlap suppression threshold, gradually adjusting the remaining detection boxes The confidence level of If the confidence level is greater than the overlap suppression threshold, the current detection frame is adjusted; otherwise, the confidence level remains unchanged. The adjustment formula is as follows: ,in represents the exponential function, represents the confidence of the unadjusted detection box, represents the confidence of the adjusted detection box, Indicates adjustment parameters; The adjusted detection frames are re-sorted from high to low according to the confidence level to obtain a new detection frame list; from the re-sorted new detection frame list, the detection frame with the highest new confidence level is selected as the current best detection frame, and the processes of the dynamic threshold calculation unit (301) and the confidence adjustment unit (302) are repeated until the intersection-over-union ratios of all detection frames are less than the overlap suppression threshold.

Images