Intelligent visual monitoring alarm method and system
By employing a target detection box caching dictionary structure and intersection-union ratio calculation in the intelligent visual monitoring system, repeated alarms are suppressed, solving the problem of continuous false alarms in existing technologies and improving the user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- E SURFING VISION TECHNOLOGY CO LTD
- Filing Date
- 2022-11-28
- Publication Date
- 2026-05-05
AI Technical Summary
Existing intelligent visual monitoring alarm methods are prone to generating multiple false alarms for the same target, which seriously affects the user experience.
A target detection box caching dictionary structure is adopted. By calculating the intersection-union ratio and setting a suppression threshold, repeated alarms are suppressed. Target detection boxes whose intersection-union ratio does not reach the suppression threshold are added to the set of boxes that need alarms, ensuring that alarms only occur when necessary.
It effectively suppressed repeated false alarms, reduced continuous false alarms of targets in the background of video images by the intelligent visual monitoring system, and improved the user experience.
Smart Images
Figure CN115861925B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of machine vision monitoring technology, and in particular to an intelligent vision monitoring alarm method and system. Background Technology
[0002] Intelligent visual surveillance uses computer vision methods to automatically analyze image sequences captured by cameras without human intervention, enabling the location, identification, and tracking of targets in dynamic scenes. Based on this analysis, it can determine the target's behavior, thus achieving both routine management and timely response to abnormal situations.
[0003] Existing intelligent visual monitoring alarm methods have the problem of misidentifying objects in the image as targets, leading to false alarms. Furthermore, due to the characteristic that monitoring cameras remain in the same position for a long time, the background of continuous video images acquired within a certain period of time remains basically unchanged. If a false alarm occurs, multiple false alarms will be generated for the same target, which seriously affects the user experience. Summary of the Invention
[0004] This invention provides an intelligent visual monitoring alarm method and system to solve the technical problem that existing intelligent visual monitoring alarm methods tend to generate multiple false alarms for the same target, which seriously affects the user experience.
[0005] In view of the above, the first aspect of the present invention provides an intelligent visual monitoring and alarm method, comprising:
[0006] S1. Obtain several target detection boxes from the image captured by the camera;
[0007] S2. Classify each target detection box according to its label. Use the target detection box category as the key to find the set of cached target boxes corresponding to each target detection box in the target box cache dictionary structure. The value in the target box cache dictionary structure is a (y, z) structure, where y is the cached target box, z is the lifetime count, and z is an initial value of a constant m that is greater than 0.
[0008] S3. Extract any target detection box x from any class;
[0009] S4. Traverse the set of cached target boxes corresponding to the extracted target detection box x in the target box cache dictionary structure, and calculate the intersection-union ratio between the extracted target detection box x and the cached target box y in the cached target box set;
[0010] S5. Determine if there is an intersection-union ratio greater than the suppression threshold. If yes, suppress the alarm of the target detection box x, assign the target detection box x to the cached target box y in the cached target box set, reset the lifecycle count z to m+1, and end the traversal of the cached target box set. If no, mark the target detection box x as needing an alarm, and add (g,x) to the target detection box set T that needs an alarm.
[0011] S6. Determine whether all categories of target detection boxes have been extracted. If not, extract any target detection box x in the next category and return to step S4. If yes, execute step S7.
[0012] S7. Issue an alarm for all target detection boxes in the target detection box set T.
[0013] Optionally, step S7 may be followed by:
[0014] S8. Decrement the lifetime count of all values in the target bounding box cache dictionary structure by 1, delete the values with a lifetime count of 0, and add (x,z) to the target bounding box cache dictionary structure according to the target detection box set T.
[0015] Optionally, step S1 specifically includes:
[0016] Acquire images captured by the camera;
[0017] The YOLOX algorithm is used to perform object detection on the image, and several object detection boxes are extracted.
[0018] A second aspect of the present invention provides an intelligent visual monitoring and alarm system, comprising:
[0019] The target bounding box acquisition module is used to acquire several target detection boxes in the image captured by the camera;
[0020] The classification module is used to classify each target detection box according to its label. It uses the target detection box category as the key to look up the set of cached target boxes corresponding to each target detection box in the target box cache dictionary structure. The value in the target box cache dictionary structure is a (y, z) structure, where y is the cached target box, z is the lifecycle count, and z is a constant m with an initial value greater than 0.
[0021] The extraction module is used to extract any target detection box x from any class;
[0022] The intersection-union ratio (IUU) calculation module is used to traverse the set of cached target boxes corresponding to the extracted target detection box x in the target box cache dictionary structure, and calculate the IUU between the extracted target detection box x and the cached target box y in the cached target box set;
[0023] The first judgment module is used to determine whether there is an intersection-union ratio greater than the suppression threshold. If so, the target detection box x is suppressed to alarm, the target detection box x is assigned to the cached target box y in the cached target box set, the lifecycle count z is reset to m+1, and the traversal of the cached target box set ends. If not, the target detection box x is marked as needing to be alarmed, and (g,x) is added to the target detection box set T that needs to be alarmed.
[0024] The second judgment module is used to determine whether all categories of target detection boxes have been extracted. If not, it extracts any target detection box x in the next category and returns to the intersection-union calculation module. If yes, it executes the alarm module.
[0025] The alarm module is used to generate alarms for all target detection boxes in the target detection box set T.
[0026] Optionally, it also includes:
[0027] The dictionary update module is used to decrement the lifetime count of all values in the target bounding box cache dictionary structure by 1, delete values with a lifetime count of 0, and add (x,z) to the target bounding box cache dictionary structure according to the target detection box set T.
[0028] Optionally, the target bounding box acquisition module is specifically used for:
[0029] Acquire images captured by the camera;
[0030] The YOLOX algorithm is used to perform object detection on the image, and several object detection boxes are extracted.
[0031] As can be seen from the above technical solutions, the intelligent visual monitoring and alarm method and system provided by the present invention have the following advantages:
[0032] The intelligent visual monitoring alarm method provided by this invention caches target detection boxes in a target box cache dictionary structure. For the newly obtained target detection box, the intersection-union ratio (IU) is first calculated with the target detection boxes in the target box cache dictionary structure. Based on a set suppression threshold, it is determined whether to suppress the alarm. Target detection boxes whose IU does not reach the suppression threshold are added to the target detection box set T that needs to be alarmed. For all target detection boxes in the target detection box set T, an alarm is generated. This suppresses repeated false alarms and reduces the situation of continuous false alarms for targets in the background of video images in intelligent visual monitoring systems. It solves the technical problem that existing intelligent visual monitoring alarm methods are prone to generating multiple consecutive false alarms for the same target, which seriously affects the user experience.
[0033] The intelligent visual monitoring and alarm system provided by this invention is used to execute the intelligent visual monitoring and alarm method provided by this invention. Its principle and the technical effects achieved are the same as those of the intelligent visual monitoring and alarm method provided by this invention, and will not be repeated here. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0035] Figure 1 This is a flowchart illustrating an intelligent visual monitoring and alarm method provided in this invention.
[0036] Figure 2 This is a schematic diagram of the principle of an intelligent visual monitoring and alarm method provided in this invention;
[0037] Figure 3 This is a schematic diagram of the structure of an intelligent visual monitoring and alarm system provided in this invention. Detailed Implementation
[0038] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0039] For easier understanding, please refer to Figure 1 and Figure 2 This invention provides an embodiment of an intelligent visual monitoring alarm method, comprising:
[0040] Step 101: Obtain several target detection boxes from the image captured by the camera.
[0041] It should be noted that the process involves acquiring a single-frame image from the camera, using an object detection algorithm to identify all targets in the image, and outputting the corresponding object detection boxes. The YOLOX algorithm is an anchor-free object detection algorithm. Anchor-free detection algorithms have high detection speed, do not require pre-setting anchors, and only need to regress the target center points and dimensions of feature maps at different scales, greatly reducing time consumption and computational power required. YOLOX not only has higher operating efficiency but also improves accuracy compared to YOLOv5.
[0042] Step 102: Classify each target detection box according to its label. Use the target detection box category as the key to find the set of cached target boxes corresponding to each target detection box in the target box cache dictionary structure. The value in the target box cache dictionary structure is a (y, z) structure, where y is the cached target box, z is the lifecycle count, and z is an initial value of a constant m that is greater than 0.
[0043] It should be noted that the target detection boxes identified in the image are denoted as the target detection box set R. The target detection boxes in the target detection box set R are classified into G classes. The target box cache dictionary structure is denoted as d. The target box cache dictionary structure d uses the target detection box class g∈G as the key and a set of (y,z) structures as values, where y is the cached target box, z is the lifecycle count, and z is an initial constant m with a value greater than 0. The target box cache dictionary structure d caches the set of cached target boxes that have triggered alarms for each class of target detection boxes, and the target box cache dictionary structure d supports updates.
[0044] Step 103: Extract any target detection box x from any class.
[0045] It should be noted that in this embodiment of the invention, alarm detection is performed on each type of target detection box according to the target detection box category. Let k be the set of a certain g∈G type target detection box collected in step 101, and extract a target detection box x∈k from the g∈G type target detection box.
[0046] Step 104: Traverse the set of cached target boxes corresponding to the extracted target detection box x in the target box cache dictionary structure, and calculate the intersection-union ratio of the extracted target detection box x and the cached target box y in the cached target box set.
[0047] It should be noted that the cached target box set L corresponding to the target box cache dictionary structure d is found by using the target detection box g category as the key. The (y,z) structure in the cached target box set L is traversed, and the intersection-union ratio (IoU) of the target detection boxes x and y is calculated.
[0048] Step 105: Determine if there is an intersection-union ratio greater than the suppression threshold. If yes, suppress the alarm for target detection box x, assign target detection box x to cached target box y in the cached target box set, reset the lifetime count z to m+1, and end the traversal of the cached target box set. If no, mark target detection box x as needing an alarm, and add (g,x) to the target detection box set T that needs an alarm.
[0049] It should be noted that for any target detection box x, if the calculated Intersection over Union (IoU) is greater than the suppression threshold p, then the target detection box x is suppressed from alarming. The target detection box x is then assigned to a cached target box y within the cached target box set, and the lifecycle count z is reset to m+1. This means the target box cache dictionary structure is set to (y, z) in the 'd' structure, and y is set as the new target detection box x. The traversal of the cached target box set ends. If the calculated IoU of any target detection box x is not greater than the suppression threshold, the traversal of the cached target box set L continues, calculating the IoU of target detection box x with the next (y, z) structure, and so on. If the traversal of the cached target box set L is complete, and all IoU values obtained are not greater than the suppression threshold p, then the target detection box x is marked as requiring an alarm, and (g, x) is added to the target detection box set T that requires an alarm. The suppression threshold p is adjustable and can be applied after simple configuration according to the intelligent video surveillance scenario.
[0050] Step 106: Determine whether all object detection boxes of all categories have been extracted. If not, extract any object detection box x in the next category and return to step 104. If yes, proceed to step 107.
[0051] Step 107: Issue an alarm for all target detection boxes in the target detection box set T.
[0052] It should be noted that for each category of target detection boxes, steps 103 to 105 are executed until all categories of target detection boxes have been extracted, resulting in the final target detection box set T. An alarm is then generated for all target detection boxes in the target detection box set T.
[0053] The intelligent visual monitoring alarm method provided by this invention caches target detection boxes in a target box cache dictionary structure. For the newly obtained target detection box, the intersection-union ratio (IU) is first calculated with the target detection boxes in the target box cache dictionary structure. Based on a set suppression threshold, it is determined whether to suppress the alarm. Target detection boxes whose IU does not reach the suppression threshold are added to the target detection box set T that needs to be alarmed. For all target detection boxes in the target detection box set T, an alarm is generated. This suppresses repeated false alarms and reduces the situation of continuous false alarms for targets in the background of video images in intelligent visual monitoring systems. It solves the technical problem that existing intelligent visual monitoring alarm methods are prone to generating multiple consecutive false alarms for the same target, which seriously affects the user experience.
[0054] In one embodiment, after step 107, the method further includes:
[0055] Step 108: Decrement the lifetime count of all values in the target bounding box cache dictionary structure by 1, delete the values with a lifetime count of 0, and add (x,z) to the target bounding box cache dictionary structure according to the target detection box set T.
[0056] It should be noted that after all object detection boxes in the object detection box set T have been alerted, the set of values in the object detection box cache dictionary structure d is traversed. For all (y, z) structures, z = z - 1. If z = 0, the entry is deleted. Simultaneously, based on the object detection box set T, (x, z) is added to the object detection box cache dictionary structure d according to the object detection box category g. This allows the search for the corresponding cache set L of the object detection box for the next frame image to be performed from the object detection box cache dictionary structure d using g as the key.
[0057] For easier understanding, please refer to Figure 3 This invention provides an embodiment of an intelligent visual monitoring and alarm system, comprising:
[0058] The target bounding box acquisition module is used to acquire several target detection boxes in the image captured by the camera;
[0059] The classification module is used to classify each target detection box according to its label. It uses the target detection box category as the key to look up the set of cached target boxes corresponding to each target detection box in the target box cache dictionary structure. The value in the target box cache dictionary structure is a (y, z) structure, where y is the cached target box, z is the lifecycle count, and z is a constant m with an initial value greater than 0.
[0060] The extraction module is used to extract any target detection box x from any class;
[0061] The intersection-union ratio (IUU) calculation module is used to traverse the set of cached target boxes corresponding to the extracted target detection box x in the target box cache dictionary structure, and calculate the IUU between the extracted target detection box x and the cached target box y in the cached target box set;
[0062] The first judgment module is used to determine whether there is an intersection-union ratio greater than the suppression threshold. If so, the target detection box x is suppressed to alarm, the target detection box x is assigned to the cached target box y in the cached target box set, the lifecycle count z is reset to m+1, and the traversal of the cached target box set ends. If not, the target detection box x is marked as needing to be alarmed, and (g,x) is added to the target detection box set T that needs to be alarmed.
[0063] The second judgment module is used to determine whether all categories of target detection boxes have been extracted. If not, it extracts any target detection box x in the next category and returns to the intersection-union calculation module. If yes, it executes the alarm module.
[0064] The alarm module is used to generate alarms for all target detection boxes in the target detection box set T.
[0065] Also includes:
[0066] The dictionary update module is used to decrement the lifetime count of all values in the target bounding box cache dictionary structure by 1, delete values with a lifetime count of 0, and add (x,z) to the target bounding box cache dictionary structure according to the target detection box set T.
[0067] The target bounding box acquisition module is specifically used for:
[0068] Acquire images captured by the camera;
[0069] The YOLOX algorithm is used to perform object detection on the image, and several object detection boxes are extracted.
[0070] The intelligent visual monitoring alarm system provided by this invention caches target detection boxes in a target box cache dictionary structure. For the newly obtained target detection box, the intersection-union ratio (IUR) is first calculated with the target detection boxes in the target box cache dictionary structure. Based on a set suppression threshold, it is determined whether to suppress the alarm. Target detection boxes whose IUR does not reach the suppression threshold are added to the target detection box set T that needs to be alarmed. For all target detection boxes in the target detection box set T, an alarm is generated. This suppresses repeated false alarms and reduces the situation of continuous false alarms for targets in the background of video images in the intelligent visual monitoring system. It solves the technical problem that existing intelligent visual monitoring alarm methods are prone to generating multiple consecutive false alarms for the same target, which seriously affects the user experience.
[0071] The intelligent visual monitoring and alarm system provided by this invention is used to execute the intelligent visual monitoring and alarm method provided by this invention. Its principle and the technical effects achieved are the same as those of the intelligent visual monitoring and alarm method provided by this invention, and will not be repeated here.
[0072] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this specification are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0073] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An intelligent visual monitoring and alarm method, characterized in that, include: S1. Obtain several target detection boxes from the image captured by the camera; S2. Classify each target detection box according to its label. Use the target detection box category as the key to find the set of cached target boxes corresponding to each target detection box in the target box cache dictionary structure. The value in the target box cache dictionary structure is a (y, z) structure, where y is the cached target box, z is the lifetime count, and z is an initial value of a constant m that is greater than 0. S3. Extract any target detection box x from any class; S4. Traverse the set of cached target boxes corresponding to the extracted target detection box x in the target box cache dictionary structure, and calculate the intersection-union ratio of the extracted target detection box x and the cached target box y in the cached target box set; S5. Determine if there is an intersection-union ratio greater than the suppression threshold. If yes, suppress the alarm for target detection box x, assign target detection box x to the cached target box y in the cached target box set, reset the lifetime count z to m+1, and end the traversal of the cached target box set. If no, mark target detection box x as needing an alarm, and add (g,x) to the target detection box set T that needs an alarm; g is the target detection box category. S6. Determine whether all categories of target detection boxes have been extracted. If not, extract any target detection box x in the next category and return to step S4. If yes, execute step S7. S7. Issue an alarm for all target detection boxes in the target detection box set T.
2. The intelligent visual monitoring and alarm method according to claim 1, characterized in that, Step S7 is followed by: S8. Decrement the lifetime count of all values in the target bounding box cache dictionary structure by 1, delete the values with a lifetime count of 0, and add (x,z) to the target bounding box cache dictionary structure according to the target detection box set T.
3. The intelligent visual monitoring and alarm method according to claim 1, characterized in that, Step S1 specifically includes: Acquire images captured by the camera; The YOLOX algorithm is used to perform object detection on the image, and several object detection boxes are extracted.
4. An intelligent visual monitoring and alarm system, characterized in that, include: The target bounding box acquisition module is used to acquire several target detection boxes in the image captured by the camera; The classification module is used to classify each target detection box according to its label. It uses the target detection box category as the key to look up the set of cached target boxes corresponding to each target detection box in the target box cache dictionary structure. The value in the target box cache dictionary structure is a (y, z) structure, where y is the cached target box, z is the lifecycle count, and z is a constant m with an initial value greater than 0. The extraction module is used to extract any target detection box x from any class; The intersection-union ratio (IUU) calculation module is used to traverse the set of cached target boxes corresponding to the extracted target detection box x in the target box cache dictionary structure, and calculate the IUU between the extracted target detection box x and the cached target box y in the cached target box set; The first judgment module is used to determine whether there is an intersection-union ratio greater than the suppression threshold. If so, the alarm of the target detection box x is suppressed, the target detection box x is assigned to the cached target box y in the cached target box set, the lifecycle count z is reset to m+1, and the traversal of the cached target box set ends. If not, the target detection box x is marked as needing an alarm, and (g,x) is added to the target detection box set T that needs an alarm; g is the target detection box category. The second judgment module is used to determine whether all categories of target detection boxes have been extracted. If not, it extracts any target detection box x in the next category and returns to the intersection-union calculation module. If yes, it executes the alarm module. The alarm module is used to generate alarms for all target detection boxes in the target detection box set T.
5. The intelligent visual monitoring and alarm system according to claim 4, characterized in that, Also includes: The dictionary update module is used to decrement the lifetime count of all values in the target bounding box cache dictionary structure by 1, delete values with a lifetime count of 0, and add (x,z) to the target bounding box cache dictionary structure according to the target detection box set T.
6. The intelligent visual monitoring and alarm system according to claim 4, characterized in that, The target bounding box acquisition module is specifically used for: Acquire images captured by the camera; The YOLOX algorithm is used to perform object detection on the image, and several object detection boxes are extracted.
Citation Information
Patent Citations
Adaptive confidence threshold adjustment method for improving video detection continuity
CN115376038A
Object detection approach using generative sparse, hierarchical networks with top-down and lateral connections for combining texture / color detection and shape / contour detection
US20140072213A1