Intelligent analysis and early warning platform and method for security
By using an intelligent analysis and early warning platform to perform real-time security detection on surveillance videos, and by utilizing AI inference models, the challenges of rapid analysis and early warning in security systems have been solved. This enables the timely detection and handling of potential threats, thereby improving the security and efficiency of security systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA SCI & TECH WESTERN RES INST OF COMPUTING TECH
- Filing Date
- 2023-03-07
- Publication Date
- 2026-07-10
AI Technical Summary
Existing security monitoring systems struggle to perform rapid and accurate security analysis and early warning when faced with large amounts of surveillance video data, resulting in the inability to detect and address potential security threats in a timely manner.
An intelligent analysis and early warning platform is adopted. It acquires monitoring videos through a data acquisition module, configures AI inference models and detection areas, and uses facial recognition, behavior recognition and scene recognition models to perform real-time security detection, generate alarm events and manage them.
It enables rapid analysis and processing of surveillance videos, allowing for timely detection and handling of security threats, preventing major security incidents, and improving the security and efficiency of the security system.
Smart Images
Figure CN116246416B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of security technology, and in particular to an intelligent analysis and early warning platform and method for security applications. Background Technology
[0002] Various sectors of society have placed higher demands on security and prevention in practical applications. Digital network monitoring technology, as an effective security and automated management method, has been widely adopted by security monitoring systems across various industries. On the one hand, digital network monitoring technology enables management departments to obtain information about important locations, thus strengthening security management within the organization; on the other hand, it improves security efficiency.
[0003] However, in order to fully cover the monitoring area, a large number of sensors need to be installed. These sensors will generate a lot of data in real time. The system needs to effectively analyze the large amount of data in order to reflect all the events that are happening on site in real time, understand the actual operational status of the personnel on site, and deal with emergencies in a timely manner.
[0004] Therefore, there is a need for an intelligent analysis and early warning platform and method for security purposes that can quickly analyze and process surveillance videos. Summary of the Invention
[0005] One of the objectives of this invention is to provide an intelligent analysis and early warning platform for security purposes, capable of rapidly analyzing and processing surveillance videos.
[0006] To solve the above-mentioned technical problems, this application provides the following technical solution:
[0007] An intelligent analysis and early warning platform for security applications includes:
[0008] The data acquisition module is used to connect to the camera and acquire surveillance video from the camera;
[0009] The device management module is used to configure the AI inference model, the detection area, and the detection time for each camera.
[0010] The inference module is used to acquire surveillance video from the data acquisition module. Within the detection time limit, it uses the configured AI inference model to perform security detection on the detection area of the surveillance video to determine whether there is an alarm event. If there is an alarm event, it sends the alarm event to the alarm event management module.
[0011] The alarm event management module is used to receive alarm events from the inference module and add the received alarm events to a preset alarm event list.
[0012] The basic principles and beneficial effects of the scheme are as follows:
[0013] This solution allows for the selection of appropriate AI inference models to analyze surveillance videos based on the needs of industrial plants. This enables dynamic monitoring of overall safety risks within the plant, timely detection of alarm events, and effective prevention of major safety accidents, safeguarding the company's safe production. By configuring different AI inference models, it can support the needs of various industrial scenarios, including access control, attendance tracking, personnel safety in industrial plants, production safety monitoring, and detection of personnel violations.
[0014] By configuring the detection area and setting the detection time, the content of the surveillance video to be analyzed can be precisely controlled according to the detection requirements, thereby improving the accuracy of the analysis.
[0015] In summary, this solution effectively prevents major safety incidents by rapidly analyzing and processing surveillance video in real time and providing early warnings of potential situations.
[0016] Furthermore, it also includes a personnel management module, used to input personnel information and classify the lists to which personnel information belongs; the lists include blacklists and whitelists, and the personnel information includes basic personnel information and corresponding front-facing headshot information.
[0017] Furthermore, the AI inference model includes a face recognition model, a behavior recognition model, and a scene recognition model.
[0018] Furthermore, when the configured AI inference model is a face recognition model, the face recognition model is used to detect faces from video frames of the surveillance video to obtain face images; align the face images, extract face features from the face images, compare the face features with the face features in the person's frontal head image information to determine whether they match, if they match, determine the corresponding person's basic information, determine whether they belong to a blacklisted person, if they do, generate an alarm event and send it to the alarm event management module.
[0019] Furthermore, when the configured AI inference model is a behavior recognition model, the behavior recognition model is used to detect all human postures and actions from the video frames of the surveillance video, extract features from the detected human postures and actions, convert the human postures and actions into a set of feature vectors, classify the feature vectors, determine whether the behavior is a preset dangerous behavior, and if so, generate an alarm event and send it to the alarm event management module.
[0020] Furthermore, when the configured AI inference model is a scene recognition model, the scene recognition model is used to detect all objects from the video frames of the monitoring video, extract features from the detected objects, convert them into feature vectors, classify the feature vectors, determine whether they are preset scenes, and if so, generate an alarm event and send it to the alarm event management module.
[0021] Furthermore, the alarm event management module is also used to receive handling information and mark the handling type of alarm events based on the handling information. The handling type includes normal events and false alarm events.
[0022] Furthermore, it also includes a system management module, which is used to generate statistical data based on alarm events, and to issue alarm reminders after receiving alarm events in the alarm event management system.
[0023] The second objective of this invention is to provide an intelligent analysis and early warning method for security applications, comprising the following steps:
[0024] S1. Configure an AI inference model, detection area, and detection time for each camera;
[0025] S2. Within the detection timeframe, the inference module uses the configured AI inference model to perform security detection on the detection area of the surveillance video to determine whether there is an alarm event. If there is an alarm event, the alarm event is sent to the alarm event management module. The AI inference model includes a face recognition model, a behavior recognition model, and a scene recognition model.
[0026] S3. Receive alarm events from the inference module, add the received alarm events to the preset alarm event list, and issue an alarm reminder;
[0027] S4. Generate statistical data based on alarm events.
[0028] Furthermore, step S2 specifically includes:
[0029] When the configured AI inference model is a face recognition model, face detection is performed on the video frames of the surveillance video to obtain face images; face features are extracted from the face images, and the face features are compared with the face features in the person's frontal head image information to determine whether they match. If they match, the basic information of the corresponding person is determined, and it is determined whether they belong to the blacklist. If they do, an alarm event is generated.
[0030] When the configured AI inference model is a behavior recognition model, all human postures and actions are detected from the video frames of the surveillance video; features are extracted from the detected human postures and actions, and the human postures and actions are converted into a set of feature vectors. The feature vectors are classified to determine whether the behavior is a preset dangerous behavior. If so, an alarm event is generated and sent to the alarm event management module.
[0031] When the configured AI inference model is a scene recognition model, all objects are detected from the video frames of the monitoring video. The detected objects are then feature-extracted and converted into feature vectors. The feature vectors are then classified to determine whether they belong to a preset scene. If so, an alarm event is generated and sent to the alarm event management module. Attached Figure Description
[0032] Figure 1 This is a logic block diagram of an embodiment of an intelligent analysis and early warning platform for security. Detailed Implementation
[0033] The following detailed description illustrates the specific implementation method:
[0034] Example 1
[0035] like Figure 1 As shown in the figure, an intelligent analysis and early warning platform for security in this embodiment includes: a camera, a data acquisition module, an inference module, a personnel management module, a system management module, a device management module, and an alarm event module.
[0036] The data acquisition module is used to connect to the camera and acquire surveillance video from the camera.
[0037] The personnel management module is used to input personnel information and categorize personnel into lists, including blacklists and whitelists. The information collected in this solution has been approved by the individuals whose information was collected.
[0038] It is also used to modify and delete personnel information; personnel information includes basic personnel information and corresponding frontal headshot information; basic personnel information includes name, position, department, etc.
[0039] The device management module is used to configure an AI inference model for each camera, set the calculation threshold of the AI inference model, and adjust the detection sensitivity of the AI inference model. Generally, the higher the calculation threshold, the more stringent the detection results, and the higher the false negative rate; the higher the sensitivity, the wider the range of objects detected by the AI inference model, but at the same time, it will also increase the false positive rate. In this embodiment, the AI inference model includes a face recognition model, a behavior recognition model, and a scene recognition model.
[0040] The device management module is also used to configure the detection area and set the detection time limit for each camera. For example, when configuring the detection area, it supports the drawing of custom quadrilaterals, and the four corners of the quadrilateral can be dragged arbitrarily. In this embodiment, a maximum of three quadrilaterals are supported in one frame to achieve coverage when detecting special terrain in the scene. After setting the detection time limit, the detection function will only be effective within the set time range.
[0041] The device management module is also used to query information about connected cameras and to manage connected cameras. Management functions include adding, editing, and deleting basic camera information, video stream addresses, etc.
[0042] The inference module is used to acquire surveillance video from the data acquisition module. Within the detection time limit, it uses the configured AI inference model to perform security detection on the detection area of the surveillance video to determine whether there is an alarm event. If there is an alarm event, it sends the alarm event to the alarm event management module.
[0043] Specifically,
[0044] When the configured AI inference model is a face recognition model, the face recognition model is used to detect faces from video frames of the surveillance video to obtain face images, that is, to find the face positions in the video frames; to align the face images, that is, to adjust the detected faces to the same size, angle, position, etc., for subsequent processing; then to extract face features from the face images, and compare the face features with the face features in the person's frontal head image information to determine whether they match. If they match, the basic information of the corresponding person is determined, and it is determined whether they belong to a blacklisted person. If they do, an alarm event is generated and sent to the alarm event management module.
[0045] When the configured AI inference model is a behavior recognition model, the behavior recognition model is used to detect all human postures and actions, such as the position of the hands, from the video frames of the surveillance video; to extract features from the detected human postures and actions, converting them into a set of feature vectors, classifying the feature vectors, and determining whether the behavior is a preset dangerous behavior. If so, an alarm event is generated and sent to the alarm event management module; preset dangerous behaviors include making and receiving phone calls, smoking, etc.
[0046] When the configured AI inference model is a scene recognition model, the scene recognition model is used to detect all objects from the video frames of the surveillance video, extract features from the detected objects, convert them into feature vectors, classify the feature vectors, and determine whether they are preset scenes. If so, an alarm event is generated and sent to the alarm event management module. Preset scenes include fire, water accumulation, oil leak, etc.
[0047] In other embodiments, behavioral detection can also be performed on whether a safety helmet, walkie-talkie, or oxygen cylinder is being worn.
[0048] The alarm event includes event information and event screenshots. The event information includes the time, type, and corresponding camera information.
[0049] The alarm event management module is used to receive alarm events from the inference module and add the received alarm events to a preset alarm event list; it also supports monitoring and management personnel to query alarm events by time, event type, and camera.
[0050] The alarm event management module is also used to receive handling information and mark the handling type of alarm events based on the handling information. The handling type includes normal events and false alarm events. In this embodiment, the handling information is the type label and remarks of the alarm events marked by the monitoring and management personnel. The alarm event management module is also used to record the remarks of false alarm events.
[0051] The alarm event management module is also used to receive manually entered alarm events. After the monitoring manager discovers a problem in the monitoring room, he or she manually records it into the system.
[0052] The system management module is used to generate statistical data based on alarm events, such as real-time statistics of alarm events generated on the same day, categorized by total number of events, total number of responses, event type, and response type.
[0053] The system management module is also used to issue alarm notifications after receiving alarm events in the alarm event management system. In this embodiment, the alarm notification is an alarm ringtone, which can be manually turned off or completely disabled in the configuration of the system management module.
[0054] Based on the above platform, this embodiment also provides an intelligent analysis and early warning method for security, including the following:
[0055] S1. Configure an AI inference model, detection area, and detection time for each camera;
[0056] S2. Within the detection timeframe, the inference module uses the configured AI inference model to perform security detection on the detection area of the surveillance video to determine whether there is an alarm event. If there is an alarm event, the alarm event is sent to the alarm event management module.
[0057] Specifically, AI inference models include facial recognition models, behavior recognition models, and scene recognition models;
[0058] When the configured AI inference model is a face recognition model, face detection is performed on the video frames of the surveillance video to obtain face images; the face images are aligned; then the face features are extracted from the face images, and the face features are compared with the face features in the person's frontal head image information to determine whether they match. If they match, the basic information of the corresponding person is determined, and it is determined whether they belong to the blacklist. If they do, an alarm event is generated.
[0059] When the configured AI inference model is a behavior recognition model, all human postures and actions are detected from the video frames of the surveillance video; features are extracted from the detected human postures and actions, and the human postures and actions are converted into a set of feature vectors. The feature vectors are classified to determine whether the behavior is a preset dangerous behavior. If so, an alarm event is generated and sent to the alarm event management module; preset dangerous behaviors include making and receiving phone calls, smoking, etc.
[0060] When the configured AI inference model is a scene recognition model, all objects are detected from the video frames of the monitoring video. Features of the detected objects are extracted and converted into feature vectors. The feature vectors are then classified to determine whether they belong to a preset scene. If so, an alarm event is generated and sent to the alarm event management module. Preset scenes include fire, water accumulation, oil leak, etc.
[0061] S3. Receive alarm events from the inference module, add the received alarm events to the preset alarm event list, and issue an alarm reminder;
[0062] S4. Generate statistical data based on alarm events.
[0063] This solution's platform supports the needs of industrial scenarios such as access control, attendance, personnel safety in industrial plants, production safety monitoring, and personnel violation detection. It can utilize surveillance videos collected by existing (or additional) cameras in the plant area, and then analyze the surveillance videos through AI inference models to achieve dynamic monitoring of overall safety production risks in the plant area, timely detection of alarm events, and effective prevention of major safety accidents, thus safeguarding the enterprise's safe production.
[0064] Example 2
[0065] The difference between this embodiment and Embodiment 1 is that the platform in this embodiment also includes a mobile terminal. The mobile terminal is used to receive push alarm events from the alarm event management module and to notify safety production management personnel in a timely manner. It is also used to input handling information and view statistical data.
[0066] Example 3
[0067] The difference between this embodiment and embodiment one is that, in this embodiment, the inference module is also used to re-perform security detection on the entire area of the video frame corresponding to the false alarm event when the handling type of the alarm event is a false alarm event, to determine whether there is an alarm event; if there is an alarm event, generate abnormal information of the detection area and send it to the device management module.
[0068] If no alarm event occurs, the inference module also adds the corresponding video frame to a preset training image set; records the alarm type, the camera to which the false alarm event belongs, and the video frame region where the detected target is located; the detected target is, for example, an object or a human body. The video frame region where the target is located is, for example, the configured detection area is divided into 4 small regions by a cross, and the small region where the target is located is the video frame region where the target is located.
[0069] The inference module is also used to determine whether the video frame region of the camera and the detected target corresponding to the false alarm event is the same as that of the camera and the detected target corresponding to the false alarm event when the same alarm event occurs again after detection. If they are different, the alarm event is sent to the alarm event management module; if they are the same, a preset number of other video frames are selected within the time range of the video frame corresponding to the alarm event. In this embodiment, 5 frames are selected within 1 second. The inference module is also used to estimate the processing time of the other selected video frames based on the current load and determine whether the processing time is less than a preset time, which is 1-5 seconds. If it is less than the preset time, the configured AI inference model is used to process the other selected video frames. Security checks are performed on video frames. When at least one detection result indicates an alarm event, the alarm event is sent to the alarm event management module. If the alarm time is greater than or equal to a preset time, it is determined whether the processing time is greater than the average response time. If it is, the alarm event is sent to the alarm event management module. If the processing time is less than the average response time, it is determined whether the number of characters in the annotation information of the false alarm event exceeds a preset value. If it does not exceed the preset value, the alarm event is sent to the alarm event management module, and the configured AI inference model is used to perform security checks on other video frames. The sent alarm events are annotated based on the detection results of other video frames. If the alarm event exceeds the preset value, the alarm event is sent to the alarm event management module, and no further detection is performed.
[0070] In this embodiment, after a false alarm occurs, if subsequent alarm events are similar in timing (i.e., the alarm type, the camera associated with it, and the video frame region where the detected target is located are the same), security checks are performed on other video frames to increase the number of detection samples and improve detection accuracy if the processing time for re-detection is short (within a preset time, minimizing the impact on alarm timeliness). If the processing time for re-detection is long (exceeding the preset time), a decision is made based on the average response time (the interval between receiving an alarm event and receiving the handling information is the response time, and the average response time is the average of the response times). If the processing time exceeds the average response time, there is a high probability that the monitoring management personnel have already viewed the alarm event. Re-detection is only performed when the cause of the false alarm is unclear (the number of characters in the remarks information does not exceed a preset value), which saves computing power and allows for thorough analysis of the alarm event, reducing the workload of the monitoring management personnel.
[0071] The above are merely embodiments of the present invention. The invention is not limited to the fields covered by these embodiments. Commonly known structures and characteristics in the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are able to access all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.
Claims
1. An intelligent analysis and early warning platform for security applications, characterized in that, include: The data acquisition module is used to connect to the camera and acquire surveillance video from the camera; The device management module is used to configure the AI inference model, the detection area, and the detection time for each camera. The inference module is used to acquire surveillance video from the data acquisition module. Within the detection time limit, it uses the configured AI inference model to perform security detection on the detection area of the surveillance video to determine whether there is an alarm event. If there is an alarm event, it sends the alarm event to the alarm event management module. The alarm event management module is used to receive alarm events from the inference module and add the received alarm events to a preset alarm event list. The alarm event management module is also used to receive handling information and mark the handling type of alarm events based on the handling information. The handling type includes normal events and false alarm events. The inference module is also used to re-perform security checks on the entire area of the video frame corresponding to the false alarm event when the handling type of the alarm event is a false alarm event, to determine whether there is an alarm event. If an alarm event occurs, generate abnormal information for the detection area and send it to the device management module; If no alarm event occurs, the inference module is also used to add the corresponding video frame to the preset training image set; and to record the alarm type, the camera to which the false alarm event belongs, and the video frame region where the detected target is located. The inference module is also used to determine whether the video frame region of the camera and the detected target corresponding to the alarm event is the same as that of the camera and the detected target corresponding to the false alarm event when the same alarm event is generated again after detection. If they are different, the alarm event is sent to the alarm event management module; if they are the same, a preset number of other video frames are selected within the time range of the video frame corresponding to the alarm event. The inference module is also used to estimate the processing time of other reselected video frames based on the current load and to determine whether the processing time is less than the preset time. If the time is less than the preset time, the configured AI inference model is used to perform security checks on other video frames. When at least one detection result has an alarm event, the alarm event is sent to the alarm event management module. If the time is greater than or equal to the preset time, determine whether the processing time is greater than the average response time. If it is greater than or equal to the average response time, send the alarm event to the alarm event management module. If the response time is less than the average response time, determine whether the number of characters in the annotation information of the false alarm event exceeds a preset value. If it does not exceed the preset value, send the alarm event to the alarm event management module and use the configured AI inference model to perform security detection on other video frames. Add annotations to the sent alarm events based on the detection results of other video frames. If it exceeds the preset value, send the alarm event to the alarm event management module.
2. The intelligent analysis and early warning platform for security as described in claim 1, characterized in that: It also includes a personnel management module, which is used to input personnel information and classify the lists to which personnel information belongs; the lists include blacklists and whitelists, and the personnel information includes basic personnel information and corresponding front-facing headshot information.
3. The intelligent analysis and early warning platform for security as described in claim 2, characterized in that: The AI inference model includes a face recognition model, a behavior recognition model, and a scene recognition model.
4. The intelligent analysis and early warning platform for security as described in claim 3, characterized in that: When the configured AI inference model is a face recognition model, the face recognition model is used to detect faces from video frames of the surveillance video to obtain face images; align the face images, extract face features from the face images, compare the face features with the face features in the person's frontal head image information to determine whether they match, if they match, determine the corresponding person's basic information, determine whether they belong to a blacklist person, if they do, generate an alarm event and send it to the alarm event management module.
5. The intelligent analysis and early warning platform for security according to claim 4, characterized in that: When the configured AI inference model is a behavior recognition model, the behavior recognition model is used to detect all human postures and actions from the video frames of the surveillance video, extract features from the detected human postures and actions, convert the human postures and actions into a set of feature vectors, classify the feature vectors, determine whether the behavior is a preset dangerous behavior, and if so, generate an alarm event and send it to the alarm event management module.
6. The intelligent analysis and early warning platform for security as described in claim 5, characterized in that: When the configured AI inference model is a scene recognition model, the scene recognition model is used to detect all objects from the video frames of the monitoring video, extract features from the detected objects, convert them into feature vectors, classify the feature vectors, determine whether they are preset scenes, and if so, generate an alarm event and send it to the alarm event management module.
7. The intelligent analysis and early warning platform for security as described in claim 6, characterized in that: It also includes a system management module, which is used to generate statistical data based on alarm events, and to issue alarm reminders after receiving alarm events in the alarm event management.
8. A method for intelligent analysis and early warning in security, using the intelligent analysis and early warning platform for security as described in any one of claims 1-7, characterized in that, Includes the following steps: S1. Configure an AI inference model, detection area, and detection time for each camera; S2. Within the detection timeframe, the inference module uses the configured AI inference model to perform security detection on the detection area of the surveillance video to determine whether there is an alarm event. If there is an alarm event, the alarm event is sent to the alarm event management module. The AI inference model includes a face recognition model, a behavior recognition model, and a scene recognition model. S3. Receive alarm events from the inference module, add the received alarm events to the preset alarm event list, and issue an alarm reminder; S4. Generate statistical data based on alarm events.
9. The intelligent analysis and early warning method for security according to claim 8, characterized in that: Step S2 specifically includes: When the configured AI inference model is a face recognition model, face detection is performed on the video frames of the surveillance video to obtain face images; face features are extracted from the face images, and the face features are compared with the face features in the person's frontal head image information to determine whether they match. If they match, the basic information of the corresponding person is determined, and it is determined whether they belong to the blacklist. If they do, an alarm event is generated. When the configured AI inference model is a behavior recognition model, all human postures and actions are detected from the video frames of the surveillance video; features are extracted from the detected human postures and actions, and the human postures and actions are converted into a set of feature vectors. The feature vectors are classified to determine whether the behavior is a preset dangerous behavior. If so, an alarm event is generated and sent to the alarm event management module. When the configured AI inference model is a scene recognition model, all objects are detected from the video frames of the monitoring video. The detected objects are then feature-extracted and converted into feature vectors. The feature vectors are then classified to determine whether they belong to a preset scene. If so, an alarm event is generated and sent to the alarm event management module.