A regional warning method and system based on video intelligent recognition
By setting up multiple video surveillance devices in the intelligent video analysis area and fusing the picture to generate a clarity abnormality index, the problem of limited angles of a single video and abnormality of monitoring clarity is solved, and more comprehensive monitoring and more efficient risk perception are achieved.
Patent Information
- Application Number
- CN202410252405.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-06
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-03-06
AI Technical Summary
When the individual video surveillance system has limited angles and abnormal monitoring clarity, it is difficult to detect abnormalities in a comprehensive manner, resulting in an increase in potential risks and unable to cover the monitoring area normally.
By dividing the intelligent video analysis area into multiple sub-regions, each sub-region sets up multiple video monitoring devices to monitor and blend the picture at different angles, generate clarity abnormality index, and switch the backup device in time for image enhancement, ensuring continuous visual management Sex and stability.
It improves the perception of potential risks, enhances the monitoring vision, reduces monitoring blind spots, improves overall image quality and clarity, and ensures efficient and continuity of visual management.
Smart Images

Figure CN118038332B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of video recognition, and in particular to a regional warning method and system based on video intelligent recognition. Background Art
[0002] The intelligent video algorithm analysis system consists of two parts: the intelligent visual management system and the intelligent visual algorithm warehouse system. The intelligent visual management system mainly provides functions such as video monitoring, security event alarm, and event closed-loop processing. The intelligent visual algorithm warehouse system mainly provides the implementation and management of video analysis algorithms. The intelligent visual management system uses existing and newly built video surveillance equipment to intelligently identify people, behaviors, environments, devices, and other content in the video, realize active push of alarm information, realize data intelligence, empower scenarios, and realize business upgrades for digital and intelligent management. Through video intelligent analysis, automatic alarms for security management are realized, and rapid processing of alarm events and security event management are achieved.
[0003] The system can form a basic database by collecting various atomic data, forming a large-screen data display based on a map, and realize a visual display of device and event data. It can display the hardware distribution, alarm information, device data and other information of the corresponding account.
[0004] The intelligent visual algorithm warehouse supports the system to implement large-scale centralized view data analysis, and realizes the view data analysis and operation management requirements in large scenarios. It includes high-performance intelligent analysis service modules such as algorithm management, computing power scheduling, and algorithm analysis, and realizes video intelligent analysis services with high concurrency, high accuracy, and high computing speed. It mainly includes algorithm production, view data access, intelligent algorithm services, and supporting basic service capabilities, including algorithm annotation training, data warehouse, access adaptation, algorithm management, intelligent scheduling, operation and maintenance management, and other core functions, as follows: Supports a variety of video data linker protocols, including RTSP, GB / T28181, RTMP, HLS, etc. Supports video-based and image-based AI analysis capabilities to meet analysis requirements in different scenarios; supports unified algorithm operation and maintenance management; supports algorithm annotation and training management; provides standardized API interfaces, and supports standardized analysis data output.
[0005] The prior art has the following deficiencies:
[0006] When the existing technology uses video surveillance to perform visual management, most of the time, a video surveillance device is set up in an area to perform video sampling. First, when sampling through a single video, the sampling angle has certain limitations and cannot better detect abnormalities. Secondly, when the video surveillance device has a hidden danger of abnormal monitoring clarity, it may cause the monitored area to be unable to cover or monitor normally, thereby increasing the potential safety risks during management. Unmonitored areas may become the location where potential risks occur.
[0007] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention
[0008] The purpose of the present invention is to provide a regional warning method and system based on video intelligent recognition. By comprehensively analyzing video quality information and time parameter information, a clarity abnormality index is generated, and compared with a preset clarity abnormality index reference threshold, it can intelligently identify whether there is a clarity abnormality in the video surveillance device. In an abnormal situation, it is possible to switch to a backup video surveillance device in time and perform image enhancement to ensure the continuity and stability of visual management, so as to solve the problems in the above-mentioned background technology.
[0009] In order to achieve the above object, the present invention provides the following technical solution: a regional warning method based on video intelligent recognition, comprising the following steps:
[0010] S101, dividing the intelligent video analysis area into a plurality of sub-areas, sequentially numbering the plurality of sub-areas, and performing partition management on the intelligent video analysis area;
[0011] S102, setting a plurality of video surveillance devices for each sub-area, making the plurality of video surveillance devices maintain the same operating state and be associated with each other, and making the plurality of video surveillance devices monitor the same area from different monitoring angles respectively;
[0012] S103, before preparing to fuse the images monitored by several video surveillance devices at the same time point, perform clarity analysis on the several surveillance images to be fused, and select the surveillance image with the best clarity as a reference for fusion;
[0013] S104, fusing the images monitored by several video surveillance devices at the same time point to generate a fused image, and performing abnormal monitoring on the sub-area through the fused image;
[0014] S105. Perform picture clarity analysis on a number of video surveillance devices that maintain the same operating status and are interrelated, and identify video surveillance devices with abnormal video surveillance. During this period, video surveillance is performed through the normally operating video surveillance devices, and the clarity of the surveillance images is adjusted in real time using image enhancement algorithms to ensure the operation. When the clarity of the surveillance images cannot be adjusted, an early warning prompt is issued to inform relevant staff members.
[0015] Preferably, the monitoring picture with the best definition is selected as the basis for fusion, and the following steps are performed:
[0016] Obtain the surveillance images that need to be fused from several video surveillance devices, and ensure that these images are collected at the same time, that is, these images are from the same time point;
[0017] Perform clarity analysis on each monitoring screen;
[0018] According to the result of the clarity analysis, the surveillance picture with the best clarity is selected as the benchmark;
[0019] Marking the monitoring picture with the best definition to indicate that the marked monitoring picture is selected as a reference picture for fusion;
[0020] Perform preparatory work before fusion on the selected reference screen.
[0021] Preferably, the monitoring images from multiple different angles are fused by image stitching. Image stitching is a technology that fuses the monitoring images from multiple different angles into an image with a larger field of view. The specific steps are:
[0022] Step 1: For each monitoring device, perform camera calibration;
[0023] Step 2, extract feature points from each monitoring screen;
[0024] Step 3, match the feature points extracted from each monitoring screen;
[0025] Step 4: Calculate the transformation relationship between different cameras based on the matched feature points;
[0026] Step 5: Use the calculated transformation relationship to perform perspective transformation on each monitoring screen and map all monitoring screens onto the same plane;
[0027] Step 6: Splice the monitoring images after perspective transformation;
[0028] Step 7: Generate the stitched images to form a fused image.
[0029] Preferably, the video quality information of several video surveillance devices that maintain the same operating state and are interrelated includes SNR. After acquisition, the SNR is processed to generate an SNR deviation index; the time parameter information of several video surveillance devices that maintain the same operating state and are interrelated includes video frame rate and exposure time. After acquisition, the video frame rate and exposure time are processed to generate a time parameter abnormality concealment index.
[0030] Preferably, the logic of obtaining the SNR deviation index is as follows:
[0031] In a fixed time window, the real-time SNR of the video surveillance device during visual management is obtained, and the real-time SNR is represented by the function SNR(t) according to the time series;
[0032] The real-time SNR of the video surveillance device during visual management is compared with the SNR reference value, and the SNR deviation index is calculated. The calculation expression is: In the formula, SNR αα Indicates the SNR deviation index, SNR 参考 represents the SNR reference value, [t x , t y ] represents the period when the real-time SNR is lower than the SNR reference value when the video surveillance device performs visual management, t x <t y .
[0033] Preferably, the logic for obtaining the abnormal concealment index of the time parameter is as follows:
[0034] In a fixed time window, obtain the real-time video frame rate of the video surveillance device when performing visual management and the actual exposure time generated by several exposures, and calibrate the real-time video frame rate and the actual exposure time as Rate y and Expo k , y represents the number of the real-time video frame rate sorted in time sequence in a fixed time window when the video surveillance device performs visual management, y=1, 2, 3, 4, ..., n, n is a positive integer, k represents the number of the actual exposure time generated by performing several exposures in a fixed time window when the video surveillance device performs visual management, k=1, 2, 3, 4, ..., m, m is a positive integer;
[0035] Calculate the abnormal concealment index of the time parameter. The calculation expression is: Time ββ =h1*Rate 帧率 +h2*Expo 曝光 , where Time ββ Indicates the abnormal concealment index of time parameters, Rate 帧率 Indicates the video frame rate change coefficient, Expo 曝光 represents the exposure time variation coefficient, h1 and h2 are the video frame rate variation coefficients Rate 帧率 , Exposure time variation coefficient Expo 曝光 The weight factor of
[0036] Preferably, the SNR deviation index SNR generated when the video surveillance device performs visual management αα and time parameter abnormal concealment index Time ββPerform a formulaic analysis to generate the clarity anomaly index Φ 成像 , through the clarity anomaly index Φ 成像 Real-time monitoring of hidden dangers of abnormal monitoring clarity during visual management of video surveillance devices.
[0037] Preferably, the clarity anomaly index generated within a fixed time window when the video surveillance device performs visual management is compared and analyzed with a preset clarity anomaly index reference threshold, and the result of the comparison and analysis is as follows:
[0038] If the clarity abnormality index is greater than or equal to the clarity abnormality index reference threshold, an information fusion signal is generated, video monitoring is performed through a normally operating video monitoring device, and multi-frame fusion technology is used to fuse information of multiple consecutive frames;
[0039] If the clarity abnormality index is less than the clarity abnormality index reference threshold, a normal signal is generated and visual management is continued through the video monitoring device.
[0040] Preferably, when the video monitoring device performs visual management to generate an information fusion signal, video monitoring is performed by the normally operating video monitoring device, and the clarity of the monitoring picture is adjusted in real time using an image enhancement algorithm. The specific steps are as follows:
[0041] After receiving the information fusion signal, it immediately switches to the screen of the backup video monitoring device to ensure that real-time monitoring is not affected by abnormal clarity;
[0042] Performing preparation work before image enhancement on the monitoring screen of the video monitoring device after the information fusion signal is not received;
[0043] Use image enhancement algorithms to adjust monitoring images in real time;
[0044] Dynamically adjust the parameters of the image enhancement algorithm based on the real-time monitored picture quality;
[0045] During the image enhancement process, the clarity improvement effect is monitored in real time to ensure that the enhanced image maintains clarity as a whole and that abnormal areas are effectively enhanced. When the clarity of the surveillance image cannot be adjusted, an early warning is issued to notify relevant staff.
[0046] A regional warning system based on video intelligent recognition, including a video analysis area division module, a video monitoring module, a monitoring picture fusion preparation module, a monitoring picture fusion module, and an abnormality recognition and processing module;
[0047] The video analysis area division module divides the intelligent video analysis area into several sub-areas, numbers the several sub-areas in sequence, and performs partition management on the intelligent video analysis area;
[0048] The video monitoring module sets a plurality of video monitoring devices for each sub-area, so that the plurality of video monitoring devices maintain the same operating state and are related to each other, and at the same time, the plurality of video monitoring devices monitor the same area from different monitoring angles;
[0049] The monitoring screen fusion preparation module analyzes the clarity of the monitoring screens to be fused before preparing to fuse the screens monitored by several video monitoring devices at the same time point, and selects the monitoring screen with the best clarity as the basis for fusion;
[0050] The monitoring screen fusion module fuses the screens monitored by several video monitoring devices at the same time point to generate a fused image, and uses the fused image to monitor the sub-area for abnormalities;
[0051] The abnormality identification and processing module analyzes the picture clarity of several video surveillance devices that maintain the same operating status and are interrelated, and identifies the video surveillance devices with video surveillance abnormalities. During this period, video surveillance is carried out through the normally operating video surveillance devices, and the image enhancement algorithm is used to adjust the clarity of the monitoring picture in real time to ensure the operation. When the clarity of the monitoring picture cannot be adjusted, an early warning prompt is issued to notify the relevant staff.
[0052] In the above technical solution, the technical effects and advantages provided by the present invention are:
[0053] The present invention divides the intelligent video analysis area into multiple sub-areas, each of which has a corresponding video monitoring device, so that the entire area can be monitored more comprehensively, which effectively solves the problem of limited angles of a single video, reduces the risk of abnormalities not being detected, and improves the ability to perceive potential risks;
[0054] The present invention monitors the same area with different monitoring angles by using several video monitoring devices. The more monitoring angles there are, the easier it is to monitor anomalies from different angles, thereby achieving efficient anomaly monitoring. By acquiring the resolution of each monitoring screen to establish a data set, the advantage of selecting the monitoring screen with the highest resolution as a benchmark for image fusion is that the overall image quality and clarity can be improved. Selecting the monitoring screen with the highest resolution as a benchmark means that the image contains more details and higher image accuracy. By fusing other screens with it, the clarity of the overall image can be further improved while retaining the most information, effectively improving the monitoring system's ability to capture and present details, thereby improving the effectiveness of the visual management system.
[0055] The present invention integrates monitoring images from multiple different angles through image stitching, and can realize a seamlessly connected panoramic image that integrates multiple perspectives. The fused image provides a wider and more comprehensive monitoring field of view. The user can observe multiple monitoring images in one image at the same time, effectively reducing monitoring blind spots and improving the overall efficiency of the entire monitoring system. Through perspective transformation and other technologies, the smooth transition of different images at the splicing point is ensured, making the fused image more realistic and coherent, which helps to reduce the visual burden of the user, simplify monitoring operations, and improve the effect of abnormal monitoring, because the user only needs to focus on a specific area in one image without having to switch multiple monitoring images. The integrated image also reduces the complexity of the monitoring system, making the comprehensive management of the scene more convenient and efficient.
[0056] The present invention generates a clarity anomaly index by comprehensively analyzing video quality information and time parameter information, and compares it according to a preset clarity anomaly index reference threshold. It can intelligently identify whether a video surveillance device has a clarity anomaly. In an abnormal situation, it can promptly switch to a backup video surveillance device and perform image enhancement to ensure the continuity and stability of visual management. When the clarity of the monitoring image cannot be adjusted, an early warning prompt is issued to notify relevant staff. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0058] Figure 1 The present invention is a method flow chart of a regional warning method and system based on video intelligent recognition.
[0059] Figure 2 The present invention is a module schematic diagram of a method and system for regional warning based on video intelligent recognition. DETAILED DESCRIPTION
[0060] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art.
[0061] The present invention provides Figure 1 The method for regional warning based on video intelligent recognition shown includes the following steps:
[0062] S101, dividing the intelligent video analysis area into a plurality of sub-areas, sequentially numbering the plurality of sub-areas, and performing partition management on the intelligent video analysis area;
[0063] The division requirements of the intelligent video analysis area are not specifically limited here. All sub-area ranges can be divided into the same, or all sub-area ranges can be divided into different, and of course, it can also be a cross of the two.
[0064] S102, setting a plurality of video surveillance devices for each sub-area, making the plurality of video surveillance devices maintain the same operating state and be associated with each other, and making the plurality of video surveillance devices monitor the same area from different monitoring angles respectively;
[0065] Here, two or more video surveillance devices may be provided, and there is no specific limitation here. The number may be selected according to actual cost and demand. The optimal number is two. The more video surveillance devices are provided, the more viewing angles there are because several video surveillance devices monitor the same area from different monitoring angles, which facilitates monitoring of abnormalities from different angles, thereby achieving efficient abnormality monitoring.
[0066] S103, before preparing to fuse the images monitored by several video surveillance devices at the same time point, perform clarity analysis on the several surveillance images to be fused, and select the surveillance image with the best clarity as a reference for fusion;
[0067] Select the surveillance image with the best clarity as the basis for fusion, and follow the steps below:
[0068] Obtain the surveillance images that need to be fused from several video surveillance devices, and ensure that these images are collected at the same time, that is, these images are from the same time point;
[0069] Perform clarity analysis on each monitoring screen;
[0070] This can be done through visual inspection or using image quality analysis tools. Sharpness analysis may include checking aspects such as detail, contrast, color saturation, etc.
[0071] According to the result of the clarity analysis, the surveillance picture with the best clarity is selected as the benchmark;
[0072] This is usually the one that shows more detail, sharpness, and better contrast in the image.
[0073] Marking the monitoring picture with the best definition to indicate that the marked monitoring picture is selected as a reference picture for fusion;
[0074] Perform preparatory work before fusion on the selected reference screen;
[0075] This may involve image processing, adjusting picture parameters, etc. to ensure that different surveillance pictures can be coordinated and integrated;
[0076] The resolution of each surveillance image is obtained to establish a data set. The benefit of selecting the surveillance image with the highest resolution as the benchmark for image fusion is that the overall image quality and clarity can be improved. Selecting the surveillance image with the highest resolution as the benchmark means that the image contains more details and higher image accuracy. By fusing other images with it, the clarity of the overall image can be further improved while retaining the most information, effectively improving the surveillance system's ability to capture and present details, thereby improving the efficiency of the visual management system.
[0077] S104, fusing the images monitored by several video surveillance devices at the same time point to generate a fused image, and performing abnormal monitoring on the sub-area through the fused image;
[0078] Image stitching is used to fuse surveillance images from multiple different angles. Image stitching is a technology that fuses surveillance images from multiple different angles into an image with a larger field of view. The specific steps are as follows:
[0079] Step 1: For each monitoring device, perform camera calibration;
[0080] This includes determining the camera's intrinsic parameters (internal parameters such as focal length and principal point coordinates) and extrinsic parameters (external parameters such as the camera's position and orientation).
[0081] Step 2, extract feature points from each monitoring screen;
[0082] Feature points can be corner points, edges, and other features in an image that are easy to detect and match.
[0083] Step 3, match the feature points extracted from each monitoring screen;
[0084] This can use feature matching algorithms such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), etc.
[0085] Step 4: Calculate the transformation relationship between different cameras based on the matched feature points;
[0086] This may include transformations such as translation, rotation, scaling, etc.
[0087] Step 5: Use the calculated transformation relationship to perform perspective transformation on each monitoring screen and map all monitoring screens onto the same plane;
[0088] Perspective transformation can be achieved by calculating the isotropic matrix.
[0089] Step 6: Splice the monitoring images after perspective transformation;
[0090] This can be achieved by stacking, averaging, etc. In this step, it may be necessary to process the transition of image edges to ensure a natural fusion effect.
[0091] Step 7, generate the spliced images to form a fused image;
[0092] At this point, information from different surveillance images is integrated into one image.
[0093] By fusing surveillance images from multiple different angles through image stitching, a seamlessly connected panoramic image that integrates multiple perspectives can be achieved. This fused image provides a broader and more comprehensive surveillance field of view. Users can observe multiple surveillance images in one image at the same time, effectively reducing surveillance blind spots and improving the overall efficiency of the entire surveillance system. Through perspective transformation and other technologies, the smooth transition of different images at the stitching point is ensured, making the fused image more realistic and coherent, which helps to reduce the user's visual burden, simplify monitoring operations, and improve the effect of abnormal monitoring, because users only need to focus on specific areas in one image without having to switch multiple surveillance images. The integrated image also reduces the complexity of the surveillance system, making the comprehensive management of the scene more convenient and efficient.
[0094] S105, analyzing the clarity of images of several video surveillance devices that maintain the same operating state and are related to each other, identifying the video surveillance device with abnormal video surveillance, during which time, video surveillance is performed through the normally operating video surveillance device, and the clarity of the surveillance image is adjusted in real time using the image enhancement algorithm to ensure the operation is carried out, and when the clarity of the surveillance image cannot be adjusted, an early warning prompt is issued to inform relevant staff members;
[0095] The logic of analyzing the picture clarity of several video surveillance devices that maintain the same operating status and are related to each other is as follows:
[0096] The video quality information and time parameter information of a plurality of video surveillance devices that maintain the same operating state and are interrelated are obtained, and after the acquisition, the video quality information and the time parameter information are comprehensively analyzed to generate a clarity anomaly index.
[0097] The video quality information of several video surveillance devices that maintain the same operating state and are interrelated includes SNR. After acquisition, the SNR is processed to generate an SNR deviation index; the time parameter information of several video surveillance devices that maintain the same operating state and are interrelated includes video frame rate and exposure time. After acquisition, the video frame rate and exposure time are processed to generate a time parameter abnormality concealment index.
[0098] SNR (Signal-to-Noise Ratio): SNR is an important parameter for describing image clarity, indicating the ratio of useful signal to noise. A high SNR usually means a clearer image. MTF (Modulation Transfer Function): MTF is a performance parameter used to describe the optical system's ability to transmit image details. A high MTF value usually corresponds to better image clarity.
[0099] When visual management is performed through video surveillance, a low SNR when the video surveillance device is monitoring the screen usually means that the noise is relatively high, which may have a serious impact on the clarity of the monitoring screen. The following is a detailed explanation of the impact of low SNR on the clarity of the monitoring screen:
[0100] Reduced visibility of image details: A lower SNR means that the signal-to-noise ratio in the image is reduced and the noise is relatively increased. This makes it difficult to distinguish details and low-contrast information in the image, thereby reducing the visibility of image details and affecting the monitoring system's overall observation of the scene.
[0101] Reduced image contrast: The presence of noise will reduce the contrast of the image, making the edges and details of objects unclear. Low-contrast images are difficult to provide clear scene information, which has an adverse effect on the clarity of the monitoring screen.
[0102] Impact on target recognition and identification: When the SNR is low, the targets (such as people, vehicles, etc.) in the monitoring screen may be blurred due to noise, making it difficult to accurately identify and distinguish. This will affect the monitoring system's accurate judgment and response to events.
[0103] Reduced nighttime monitoring effect: The impact of SNR is more significant at night or in low-light conditions. Low SNR may increase noise in nighttime monitoring images, reduce image clarity, and make monitoring effects in low-light conditions poor.
[0104] Affecting the effect of digital image processing: Low SNR may have an adverse effect on subsequent digital image processing. For example, when performing image enhancement, edge detection and other processing, the presence of noise may make the processing results unstable and affect the final image quality.
[0105] Therefore, by monitoring the SNR of the video surveillance device in real time when visual management is performed on the video surveillance device, abnormal hidden dangers of low SNR of the video surveillance device can be intelligently perceived.
[0106] The logic of obtaining the SNR deviation index is as follows:
[0107] In a fixed time window, the real-time SNR of the video surveillance device during visual management is obtained, and the real-time SNR is represented by the function SNR(t) according to the time series;
[0108] It should be noted that the SNR of an image can be measured in real time using specialized video analysis software or tools provided by the surveillance system. These software usually have image processing and analysis functions, part of which may include real-time monitoring of SNR. For example, Milestone XProtect is a professional video management software that supports advanced analysis of videos, including image quality assessment, including SNR measurement, and is often used in large and organized video surveillance systems. Another example is Avigilon Control Center, a professional video management software that supports advanced video analysis and image quality assessment, and can be used to monitor SNR in real time.
[0109] The real-time SNR of the video surveillance device during visual management is compared with the SNR reference value, and the SNR deviation index is calculated. The calculation expression is: In the formula, SNR αα Indicates the SNR deviation index, SNR 参考 represents the SNR reference value, [t x , t y ] represents the period when the real-time SNR is lower than the SNR reference value when the video surveillance device performs visual management, t x <t y ;
[0110] The SNR reference value usually involves balancing the image quality and the needs of the actual application scenario. Different monitoring purposes may have different requirements for image quality. If the monitoring system is mainly used for security monitoring and event identification, the requirement for clarity may be higher, while a system used for general monitoring and recording may accept a lower SNR. Therefore, the setting of the SNR reference value is not specifically limited here and can be adjusted according to the needs of the actual application scenario.
[0111] It can be seen from the calculation expression of the SNR deviation index that within a fixed time window, the larger the performance value of the SNR deviation index generated when the video surveillance device performs visual management, the greater the risk of abnormal monitoring clarity when the video surveillance device performs visual management; conversely, the smaller the risk of abnormal monitoring clarity when the video surveillance device performs visual management.
[0112] When visual management is performed through video surveillance, the video frame rate refers to the number of frames displayed per second in the video, which directly affects the smoothness and clarity of the video image. When the video surveillance device monitors the image, the video frame rate changes greatly, which may have some impact on the clarity of the monitoring image. The following is a detailed explanation:
[0113] Dynamic object blur: When the video frame rate changes greatly, especially at a low frame rate, fast-moving objects may have a large spatial displacement between adjacent frames, causing the objects to be blurred in the image and reducing the clarity of the objects.
[0114] Motion artifacts: When the video frame rate changes greatly, fast-moving objects may produce motion artifacts between adjacent frames. Such artifacts may appear as blurred, stretched, or ghosting object outlines, reducing the clarity of the surveillance image.
[0115] Distortion and discontinuity: When the video frame rate is unstable, some parts of the picture may be distorted or discontinuous because the transition between different frames is not smooth enough. This distortion and discontinuity will reduce the overall clarity of the monitoring picture.
[0116] Difficulty in capturing details: A low frame rate may result in the inability to accurately capture events that occur quickly in the video, especially some events that occur instantly, which will reduce the ability to capture details and affect the effectiveness of the surveillance system.
[0117] When visual management is performed through video surveillance, exposure time refers to the exposure time of the video surveillance device to light, which directly affects the brightness and clarity of the video image. Large changes in the exposure time of the video surveillance device when monitoring the image may have some impact on the clarity of the monitoring image. The following is a detailed explanation:
[0118] Unstable brightness: Changes in exposure time cause the brightness of the image to fluctuate greatly at different time points, which may cause objects in the image to switch between light and dark, affecting the overall clarity of the image.
[0119] Loss of details: Longer exposure times may result in overexposure, where details are submerged in overly bright areas, while shorter exposure times may result in overexposure, where details are lost in dark areas. Both situations affect the clarity and information acquisition of the surveillance image.
[0120] Dynamic range problem: Changes in exposure time may affect the dynamic range of the image, that is, it is difficult to maintain appropriate contrast between areas with higher and lower brightness, resulting in reduced image clarity.
[0121] Moving object blur: When the exposure time is long, moving objects may leave blurred tracks in the image because they have a large displacement between adjacent frames. This will affect the clarity of moving objects in the surveillance picture.
[0122] Therefore, when visual management is performed on the video surveillance device, the video frame rate and exposure time of the video surveillance device are monitored in real time, and abnormal hidden dangers of large changes in the video frame rate and exposure time of the video surveillance device can be intelligently perceived.
[0123] The logic for obtaining the time parameter anomaly concealment index is as follows:
[0124] In a fixed time window, obtain the real-time video frame rate of the video surveillance device when performing visual management and the actual exposure time generated by several exposures, and calibrate the real-time video frame rate and the actual exposure time as Rate y and Expo k , y represents the number of the real-time video frame rate sorted in time sequence in a fixed time window when the video surveillance device performs visual management, y=1, 2, 3, 4, ..., n, n is a positive integer, k represents the number of the actual exposure time generated by performing several exposures in a fixed time window when the video surveillance device performs visual management, k=1, 2, 3, 4, ..., m, m is a positive integer;
[0125] It should be noted that Milestone XProtect is a professional video management software used to monitor and manage video cameras, which can provide a real-time monitoring interface, including displaying performance parameters such as video frame rate. Secondly, GenetecSecurity Center is an integrated security management software used for video monitoring, access control, etc. It can provide real-time monitoring images and display information such as video frame rate;
[0126] Calculate the abnormal concealment index of the time parameter. The calculation expression is: Time ββ =h1*Rate 帧率 +h2*Expo 曝光 , where Time ββ Indicates the abnormal concealment index of time parameters, Rate 帧率 Indicates the video frame rate change coefficient, Expo 曝光 represents the exposure time variation coefficient, h1 and h2 are the video frame rate variation coefficients Rate 帧率 , Exposure time variation coefficient Expo 曝光 The weight factor is used to balance the proportion of each data in the formula, thereby promoting the accuracy of the calculation results.
[0127] It can be seen from the calculation expression of the time parameter anomaly concealment index that within a fixed time window, the larger the performance value of the time parameter anomaly concealment index generated when the video surveillance device performs visual management, the greater the hidden danger of abnormal monitoring clarity when the video surveillance device performs visual management; conversely, the smaller the hidden danger of abnormal monitoring clarity when the video surveillance device performs visual management.
[0128] The SNR deviation index SNR generated when the video surveillance device performs visual management αα and time parameter abnormal concealment index Time ββ Perform a formulaic analysis to generate the clarity anomaly index Φ 成像 , based on the formula:
[0129]
[0130] , where μ1 and μ2 are SNR deviation index SNR αα and time parameter abnormal concealment index Time ββ The preset proportional coefficient, and μ1 and μ2 are both greater than 0.
[0131] From the calculation formula, we can know that within a fixed time window, the SNR deviation index SNR generated by the video surveillance device when performing visual management is αα The larger the value, the more hidden the time parameter abnormality index Time ββ The larger the value, the larger the clarity anomaly index Φ generated by the video surveillance device during visual management. 成像 The larger the performance value, the greater the hidden danger of abnormal monitoring clarity when the video surveillance device performs visual management. Conversely, the smaller the hidden danger of abnormal monitoring clarity when the video surveillance device performs visual management.
[0132] The clarity anomaly index generated by the video surveillance device in a fixed time window during visual management is compared and analyzed with the preset clarity anomaly index reference threshold. The results of the comparison and analysis are as follows:
[0133] If the clarity abnormality index is greater than or equal to the clarity abnormality index reference threshold, an information fusion signal is generated, video monitoring is performed through a normally operating video monitoring device, and multi-frame fusion technology is used to fuse information of multiple consecutive frames;
[0134] If the clarity abnormality index is less than the clarity abnormality index reference threshold, a normal signal is generated and visual management continues through the video surveillance device.
[0135] When the video surveillance device generates information fusion signals for visual management, video surveillance is performed through the normally operating video surveillance device, and the clarity of the surveillance image is adjusted in real time using the image enhancement algorithm. The specific steps are as follows:
[0136] After receiving the information fusion signal, it immediately switches to the screen of the backup video monitoring device to ensure that real-time monitoring is not affected by abnormal clarity;
[0137] Performing preparation work before image enhancement on the monitoring screen of the video monitoring device after the information fusion signal is not received;
[0138] This may include selecting a higher resolution frame as a baseline, setting parameters for the image enhancement algorithm, and determining areas for enhancement.
[0139] Use image enhancement algorithms to adjust monitoring images in real time;
[0140] This can include increasing sharpness, adjusting contrast, and removing noise to improve clarity in areas of abnormality.
[0141] Dynamically adjust the parameters of the image enhancement algorithm based on the real-time monitored picture quality;
[0142] This allows the clarity to be adaptively adjusted based on monitoring requirements under different conditions.
[0143] During the image enhancement process, the clarity improvement effect is monitored in real time to ensure that the enhanced image maintains clarity as a whole and that abnormal areas are effectively enhanced. When the clarity of the surveillance image cannot be adjusted, an early warning is issued to notify relevant staff.
[0144] The present invention provides Figure 2 The regional warning system based on video intelligent recognition shown includes a video analysis area division module, a video monitoring module, a monitoring picture fusion preparation module, a monitoring picture fusion module, and an abnormality recognition and processing module;
[0145] The video analysis area division module divides the intelligent video analysis area into several sub-areas, numbers the several sub-areas in sequence, and performs partition management on the intelligent video analysis area;
[0146] The video monitoring module sets a plurality of video monitoring devices for each sub-area, so that the plurality of video monitoring devices maintain the same operating state and are related to each other, and at the same time, the plurality of video monitoring devices monitor the same area from different monitoring angles;
[0147] The monitoring screen fusion preparation module analyzes the clarity of the monitoring screens to be fused before preparing to fuse the screens monitored by several video monitoring devices at the same time point, and selects the monitoring screen with the best clarity as the basis for fusion;
[0148] The monitoring screen fusion module fuses the screens monitored by several video monitoring devices at the same time point to generate a fused image, and uses the fused image to monitor the sub-area for abnormalities;
[0149] The abnormality identification and processing module analyzes the picture clarity of several video surveillance devices that maintain the same operating status and are related to each other, and identifies the video surveillance device with abnormal video surveillance. During this period, video surveillance is performed through the normally operating video surveillance device, and the clarity of the surveillance picture is adjusted in real time using the image enhancement algorithm to ensure the operation. When the clarity of the surveillance picture cannot be adjusted, an early warning prompt is issued to inform the relevant staff;
[0150] An area warning method based on video intelligent recognition provided by an embodiment of the present invention is implemented by the above-mentioned area warning system based on video intelligent recognition. The specific method and process of an area warning system based on video intelligent recognition are detailed in the embodiment of the above-mentioned area warning method based on video intelligent recognition, which will not be repeated here.
[0151] The present invention divides the intelligent video analysis area into multiple sub-areas, each of which has a corresponding video monitoring device, so that the entire area can be monitored more comprehensively, which effectively solves the problem of limited angles of a single video, reduces the risk of abnormalities not being detected, and improves the ability to perceive potential risks;
[0152] The present invention monitors the same area with different monitoring angles by using several video monitoring devices. The more monitoring angles there are, the easier it is to monitor anomalies from different angles, thereby achieving efficient anomaly monitoring. By acquiring the resolution of each monitoring screen to establish a data set, the advantage of selecting the monitoring screen with the highest resolution as a benchmark for image fusion is that the overall image quality and clarity can be improved. Selecting the monitoring screen with the highest resolution as a benchmark means that the image contains more details and higher image accuracy. By fusing other screens with it, the clarity of the overall image can be further improved while retaining the most information, effectively improving the monitoring system's ability to capture and present details, thereby improving the effectiveness of the visual management system.
[0153] The present invention integrates monitoring images from multiple different angles through image stitching, and can realize a seamlessly connected panoramic image that integrates multiple perspectives. The fused image provides a wider and more comprehensive monitoring field of view. The user can observe multiple monitoring images in one image at the same time, effectively reducing monitoring blind spots and improving the overall efficiency of the entire monitoring system. Through perspective transformation and other technologies, the smooth transition of different images at the splicing point is ensured, making the fused image more realistic and coherent, which helps to reduce the visual burden of the user, simplify monitoring operations, and improve the effect of abnormal monitoring, because the user only needs to focus on a specific area in one image without having to switch multiple monitoring images. The integrated image also reduces the complexity of the monitoring system, making the comprehensive management of the scene more convenient and efficient.
[0154] The present invention generates a clarity anomaly index by comprehensively analyzing video quality information and time parameter information, and compares it according to a preset clarity anomaly index reference threshold. It can intelligently identify whether a video surveillance device has a clarity anomaly. In an abnormal situation, it can promptly switch to a backup video surveillance device and perform image enhancement to ensure the continuity and stability of visual management. When the clarity of the monitoring image cannot be adjusted, an early warning prompt is issued to notify relevant staff.
[0155] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.
[0156] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center containing one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.
[0157] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0158] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0159] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0160] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0161] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0162] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A regional warning method based on video intelligent recognition, characterized in that: The following steps are involved: S101, dividing the intelligent video analysis area into a plurality of sub-areas, sequentially numbering the plurality of sub-areas, and performing partition management on the intelligent video analysis area; S102, setting a plurality of video surveillance devices for each sub-area, making the plurality of video surveillance devices maintain the same operating state and be associated with each other, and making the plurality of video surveillance devices monitor the same area from different monitoring angles respectively; S103, before preparing to fuse the images monitored by several video surveillance devices at the same time point, perform clarity analysis on the several surveillance images to be fused, and select the surveillance image with the best clarity as a reference for fusion; S104, fusing the images monitored by several video surveillance devices at the same time point to generate a fused image, and performing abnormal monitoring on the sub-area through the fused image; Image stitching is used to fuse surveillance images from multiple different angles. Image stitching is a technology that fuses surveillance images from multiple different angles into an image with a larger field of view. The specific steps are as follows: Step 1: For each monitoring device, perform camera calibration; Step 2, extract feature points from each monitoring screen; Step 3, match the feature points extracted from each monitoring screen; Step 4: Calculate the transformation relationship between different cameras based on the matched feature points; Step 5: Use the calculated transformation relationship to perform perspective transformation on each monitoring screen and map all monitoring screens onto the same plane; Step 6: Splice the monitoring images after perspective transformation; Step 7, generate the spliced images to form a fused image; S105. Perform picture clarity analysis on a number of video surveillance devices that maintain the same operating status and are interrelated, and identify video surveillance devices with abnormal video surveillance. During this period, video surveillance is performed through the normally operating video surveillance devices, and the clarity of the surveillance images is adjusted in real time using image enhancement algorithms to ensure the operation. When the clarity of the surveillance images cannot be adjusted, an early warning prompt is issued to inform relevant staff members.
2. According to claim 1, a method for regional warning based on video intelligent recognition is characterized in that: Select the surveillance image with the best clarity as the basis for fusion, and follow the steps below: Obtain the surveillance images that need to be fused from several video surveillance devices, and ensure that these images are collected at the same time, that is, these images are from the same time point; Perform clarity analysis on each monitoring screen; According to the result of the clarity analysis, the surveillance picture with the best clarity is selected as the benchmark; Marking the monitoring picture with the best definition to indicate that the marked monitoring picture is selected as a reference picture for fusion; Perform preparatory work before fusion on the selected reference screen.
3. The method for regional warning based on video intelligent recognition according to claim 1 is characterized in that: The video quality information of several video surveillance devices that maintain the same operating state and are interrelated includes SNR. After acquisition, the SNR is processed to generate an SNR deviation index; the time parameter information of several video surveillance devices that maintain the same operating state and are interrelated includes video frame rate and exposure time. After acquisition, the video frame rate and exposure time are processed to generate a time parameter abnormality concealment index.
4. The method for regional warning based on video intelligent recognition according to claim 3 is characterized in that: The logic of obtaining the SNR deviation index is as follows: In a fixed time window, the real-time SNR of the video surveillance device during visual management is obtained, and the real-time SNR is represented by the function SNR(t) according to the time series; The real-time SNR of the video surveillance device during visual management is compared with the SNR reference value, and the SNR deviation index is calculated. The calculation expression is: In the formula, SNR αα Indicates the SNR deviation index, SNR 参考 represents the SNR reference value, [t x , t y ] represents the period when the real-time SNR is lower than the SNR reference value when the video surveillance device performs visual management, t x <t y .
5. The method for regional warning based on video intelligent recognition according to claim 4 is characterized in that: The logic for obtaining the time parameter anomaly concealment index is as follows: In a fixed time window, obtain the real-time video frame rate of the video surveillance device when performing visual management and the actual exposure time generated by several exposures, and calibrate the real-time video frame rate and the actual exposure time as Rate y and Expo k , y represents the number of the real-time video frame rate sorted in time sequence in a fixed time window when the video surveillance device performs visual management, y=1, 2, 3, 4, ..., n, n is a positive integer, k represents the number of the actual exposure time generated by performing several exposures in a fixed time window when the video surveillance device performs visual management, k=1, 2, 3, 4, ..., m, m is a positive integer; Calculate the abnormal concealment index of the time parameter. The calculation expression is: Time ββ =h1*Rate 帧率 +h2*Expo 曝光 , where Time ββ Indicates the abnormal concealment index of time parameters, Rate 帧率 Indicates the video frame rate change coefficient, Expo 曝光 represents the exposure time variation coefficient, h1 and h2 are the video frame rate variation coefficients Rate 帧率 , Exposure time variation coefficient Expo 曝光 The weight factor of 6. The method for regional warning based on video intelligent recognition according to claim 5 is characterized in that: The SNR deviation index SNR generated when the video surveillance device performs visual management αα and time parameter abnormal concealment index Time ββ Perform a formulaic analysis to generate the clarity anomaly index Φ 成像 , through the clarity anomaly index Φ 成像 Real-time monitoring of hidden dangers of abnormal monitoring clarity during visual management of video surveillance devices.
7. The method for regional warning based on video intelligent recognition according to claim 6 is characterized in that: The clarity anomaly index generated by the video surveillance device in a fixed time window during visual management is compared and analyzed with the preset clarity anomaly index reference threshold. The results of the comparison and analysis are as follows: If the clarity abnormality index is greater than or equal to the clarity abnormality index reference threshold, an information fusion signal is generated, video monitoring is performed through a normally operating video monitoring device, and multi-frame fusion technology is used to fuse information of multiple consecutive frames; If the clarity abnormality index is less than the clarity abnormality index reference threshold, a normal signal is generated and visual management is continued through the video monitoring device.
8. The method for regional warning based on video intelligent recognition according to claim 7 is characterized in that: When the video surveillance device generates information fusion signals for visual management, video surveillance is performed through the normally operating video surveillance device, and the clarity of the surveillance image is adjusted in real time using the image enhancement algorithm. The specific steps are as follows: After receiving the information fusion signal, it will immediately switch to the screen of the backup video monitoring device to ensure that the real-time monitoring is not affected by the abnormal clarity; Performing preparation work before image enhancement on the monitoring screen of the video monitoring device after the information fusion signal is not received; Use image enhancement algorithms to adjust monitoring images in real time; Dynamically adjust the parameters of the image enhancement algorithm based on the real-time monitored picture quality; During the image enhancement process, the clarity improvement effect is monitored in real time to ensure that the enhanced image maintains clarity as a whole and that abnormal areas are effectively enhanced. When the clarity of the surveillance image cannot be adjusted, an early warning is issued to notify relevant staff.
9. A regional warning system based on video intelligent recognition, used to implement a regional warning method based on video intelligent recognition as described in any one of claims 1 to 8, characterized in that: It includes video analysis area division module, video monitoring module, monitoring screen fusion preparation module, monitoring screen fusion module, and abnormality identification and processing module; The video analysis area division module divides the intelligent video analysis area into several sub-areas, sequentially numbers the several sub-areas, and performs partition management on the intelligent video analysis area; The video monitoring module sets a plurality of video monitoring devices for each sub-area, so that the plurality of video monitoring devices maintain the same operating state and are related to each other, and at the same time, the plurality of video monitoring devices monitor the same area from different monitoring angles; The monitoring screen fusion preparation module analyzes the clarity of the monitoring screens to be fused before preparing to fuse the screens monitored by several video monitoring devices at the same time point, and selects the monitoring screen with the best clarity as the basis for fusion; The monitoring screen fusion module fuses the screens monitored by several video monitoring devices at the same time point to generate a fused image, and uses the fused image to monitor the sub-area for abnormalities; The abnormality identification and processing module analyzes the picture clarity of several video surveillance devices that maintain the same operating status and are interrelated, and identifies the video surveillance devices with video surveillance abnormalities. During this period, video surveillance is carried out through the normally operating video surveillance devices, and the image enhancement algorithm is used to adjust the clarity of the monitoring picture in real time to ensure the operation. When the clarity of the monitoring picture cannot be adjusted, an early warning prompt is issued to notify the relevant staff.
