Video surveillance system, abnormal event detection method

Abstract

The present invention relates to the field of security detection technology, and in particular provides a video monitoring system and an abnormal event detection method, the method comprising: collecting information of multiple categories of data sets according to a preset ratio and marking them respectively to the data sets, dividing them according to the preset ratio to obtain multiple training sets and verification sets; inputting the multiple training sets into the training model for model training, inputting the verification sets corresponding to the training sets into the training module for verification, and embedding the trained models into the corresponding target detection units respectively; determining the electronic fence layout area according to the construction area and dividing the area, setting cross-area rules and abnormal event level determination rules; calling up the monitoring screen according to the detection results of the preset detection nodes and preprocessing them; using the target detection module to obtain the target detection results of the monitoring screen after preprocessing; and obtaining the type and level of the abnormal event according to the target detection results. Its purpose is to solve the technical problems of missed detection and difficult detection of abnormal events in construction under plateau environment.

Description

Technical Field

[0001] The present invention relates to the field of security detection technology, and in particular to a video monitoring system and an abnormal event detection method. Background Art

[0002] The construction scale of plateau power projects is large and the system is complex. Local abnormal events will affect the normal operation of the entire project. In order to ensure construction safety and coordination, it is of great significance to the construction of power projects in plateau environments if abnormal events can be accurately warned and identified.

[0003] At present, real-time detection of abnormal events in plateau power construction based on video surveillance has become a hot research direction. However, abnormal event detection based on video surveillance in the complex environment of the plateau is very challenging. Traditional video surveillance systems generally use high-definition cameras to monitor key nodes of the construction site in real time, and identify abnormal events in the construction site based on video surveillance. This type of monitoring system usually uses a passive monitoring module. Generally, after an abnormal event occurs, the monitoring personnel obtain abnormal event information by retrieving the monitoring screen or other means, and further determine the type of abnormal event. In actual application, the above scheme relies on manual operation to monitor the scene and discover abnormal events by watching the video, which often results in false detection and missed detection.

[0004] Especially when serious abnormal events such as fire, typhoon, lightning and other natural disasters occur in the construction site, the existing methods are difficult to correctly detect the actual accident type based on image recognition; on the other hand, due to the complexity of the plateau environment, the image acquisition equipment can only collect limited areas. When the accident occurs in the detection blind area and cannot be detected, it will cause the problem of missed detection and difficulty in detection of abnormal events. In order to effectively improve the accuracy and stability of traditional video abnormal event detection, we propose a video monitoring system and abnormal event detection method. Summary of the invention

[0005] The purpose of the present invention is to provide a video monitoring system and an abnormal event detection method to solve the technical problems of missed detection and difficult detection of abnormal events during construction in a plateau environment.

[0006] The first technical solution of the present invention provides a video surveillance system, including a server and at least one electronic fence arranged in a surveillance area, wherein the server is communicatively connected to the electronic fence, and the electronic fence includes:

[0007] At least one data collection device, used to collect environmental data within the monitoring area;

[0008] At least one detection node, used to detect the change trend of detection data in the monitoring area and predict abnormal events;

[0009] At least one monitoring device, used to obtain video image information within the monitoring area;

[0010] at least one sensing component for identifying intrusion into or out of the electronic fence;

[0011] Display screen, used to display video image monitoring data and danger alarm data;

[0012] The main control module is configured with a target detection module, and the target detection module is configured to perform target detection based on the image data collected by the monitoring device to determine the target display area where the target detection object exists.

[0013] Furthermore, an angle adjustment component is installed on the monitoring device, and the angle adjustment component is configured to adjust an angle parameter of the monitoring device according to a detection result of the target detection module.

[0014] Furthermore, the system also includes:

[0015] The fire sensor is arranged in the monitoring area and is connected to the monitoring device for communication. When a fire is detected, the angle of the monitoring device is adjusted by the angle adjustment component to monitor the fire.

[0016] Furthermore, the plurality of target detection modules include:

[0017] A first target detection unit is configured to perform a full-image scan on the image data to detect whether there is any foreign object intruding into the monitoring area;

[0018] A second target detection unit is configured to perform a full-image scan on the image data to detect whether a natural disaster occurs in the monitoring area;

[0019] A third target detection unit is configured to perform a full-image scan on the image data to detect whether the construction workers in the monitoring area are wearing protective gear;

[0020] The fourth target detection unit is configured to perform a full-image scan on the image data to detect whether there are any abnormal traffic events in the monitoring area.

[0021] Furthermore, the server adopts a cloud server, and the cloud server is configured to receive image information transmitted by the target detection module, decompress it, and then store it in a database in partitions.

[0022] Furthermore, the environmental data information collected by the data collection device includes at least: temperature and humidity, oxygen concentration, weather, ultraviolet radiation, and atmospheric pressure.

[0023] Furthermore, the system also includes: an early warning notification module, which is used to issue early warning information based on whether there is an abnormal event in the monitoring area. If there is an abnormal event, the corresponding early warning information is obtained, and the early warning information at least includes: time, geographic location coordinates and abnormal event type.

[0024] The second technical solution of the present invention provides an abnormal event detection method, which is applied to the video surveillance system described in any one of the technical solutions of the first technical solution of the present invention, and comprises the following steps:

[0025] Collect information of multiple categories of data sets according to preset ratios and annotate them respectively, and divide them into multiple training sets and validation sets according to preset ratios;

[0026] Multiple training sets are input into the training model for model training, and the verification sets corresponding to the training sets are input into the training module for verification, and the trained models are embedded into the corresponding target detection units.

[0027] Determine the layout area of ​​the electronic fence according to the construction area, divide the area of ​​the electronic fence and set cross-region rules and rules for determining the level of abnormal events;

[0028] Retrieving monitoring images according to the detection results of the preset detection nodes and preprocessing the monitoring images;

[0029] Use the target detection module to obtain the target detection results of the pre-processed monitoring screen;

[0030] Obtain the type and level of abnormal events based on the target detection results.

[0031] Furthermore, monitoring screen preprocessing includes:

[0032] Split the monitoring screen into frame images, input them into multiple target detection units to detect abnormal events, and if a certain category is detected, output the detected category and mark the warning information;

[0033] If there is no abnormal event, the no warning information will be output directly.

[0034] Furthermore, the method of obtaining the target detection result also includes: inputting the video clip into the spatial stream convolutional neural network and the temporal stream convolutional neural network of the M-I3D module for feature extraction, and using a multi-instance ranking model to output anomaly scores according to the features to determine whether an abnormal event occurs.

[0035] The abnormal event detection method provided by the second aspect of the technical solution of the present invention is applied to the video surveillance system described in any one of the technical solutions of the first aspect of the present invention, and thus it has all the beneficial effects of the video surveillance system described in the first aspect of the technical solution of the present application, which will not be repeated here.

[0036] The beneficial effects of the present invention include:

[0037] 1. The video monitoring system provided by the present invention improves the problem of real-time monitoring of only key nodes in the construction video monitoring system under plateau environment through the coordinated use of electronic fences, monitoring equipment, data acquisition devices, detection nodes and target detection modules. It adopts an active monitoring mode to issue early warnings before abnormal events occur, without relying on manual operation to monitor the scene, and further strengthens the safety management of the monitoring area;

[0038] 2. The abnormal event detection method provided by the present invention is based on a video surveillance system. It determines whether there is an abnormal event in the construction area by comparing the feature matching degree of multiple positive and negative samples in the target detection module, thereby avoiding false detection and missed detection, and effectively improving the accuracy of abnormal events. On the other hand, it divides abnormal events into levels to achieve different processing methods for different levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments of the present invention are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A schematic diagram of the structure of a video surveillance system provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the structure of an electronic fence provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of the steps of the abnormal event detection method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0044] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0045] The embodiments of the present application take a power construction project in a plateau environment as an example to introduce the technical solution proposed in the present application, but the usage scenario is not limited to the power construction project. Any scenario in which the present application can be used is within the scope of protection required by the present application.

[0046] First, the monitoring equipment preferably has a camera with wide-angle overall scene shooting and high zoom capability, so that the shooting range of each monitoring device is large and the feature identification of the target can be achieved; in the construction area, the layout of the monitoring equipment is based on covering the construction area as much as possible.

[0047] See also Figure 1 and Figure 2 As shown, the first technical solution of the present application provides a video surveillance system, including a server and at least one electronic fence arranged in a surveillance area, wherein the server is communicatively connected to the electronic fence, and the electronic fence includes:

[0048] Multiple data acquisition devices for collecting environmental data in the monitoring area; the environmental data information collected by the data acquisition devices at least includes: temperature and humidity, oxygen concentration, weather, ultraviolet radiation, and atmospheric pressure. Considering the complex environment of the plateau, its environmental factors are also one of the main causes of abnormal events. The data acquisition devices may include but are not limited to one or more of temperature and humidity sensors, barometers, wind flow meters, oxygen concentration monitors, ultraviolet detectors, and meteorological detectors;

[0049] At least one detection node is used to detect the change trend of detection data in the monitoring area and predict abnormal events; the detection data includes but is not limited to the environmental data collected by the above-mentioned data acquisition device and the video image information collected by the monitoring device, and the abnormal events occurring in the monitoring area are predicted based on the detection data;

[0050] At least one monitoring device for acquiring video image information; the monitoring device may be arranged in the monitoring area by assuming a high tower, and its communication is preferably wireless communication to upload the collected image information to the server; due to the difficulty of power supply in the plateau environment, its power module is preferably a photovoltaic panel + battery to realize power supply; its arrangement point is not limited to one or more of the entrance and exit of the monitoring area, intersection, power equipment, construction equipment, archive room, and financial information room;

[0051] At least one sensing component is used to identify intrusion into or out of the electronic fence; the sensing component is configured to continuously scan the monitoring area through radar, and if the sensing component detects illegal intrusion, it outputs the location information of the suspicious target. At this time, the monitoring device performs target detection on the illegally intruding foreign object to obtain the type of abnormal event.

[0052] The display screen is used to display video image monitoring data and danger alarm data; the main control module is equipped with a target detection module, and the target detection module is configured to perform target detection based on the image data collected by the monitoring device to determine the target display area where the target object exists. In this embodiment, a dispatch center is provided for use with the display screen and the main control module. The dispatch center is used to divide the entire construction project into regions. Compared with the prior art, it only needs to be equipped with one display screen to complete the monitoring of abnormal events in the entire construction area, without the need for manual auxiliary judgment, thereby solving the problem of difficult detection and missed detection.

[0053] Among them, an angle adjustment component is installed on the monitoring equipment, and a plurality of fire sensors that are communicatively connected to the monitoring equipment are arranged in the construction area. The angle adjustment component is configured to adjust the angle parameters of the monitoring equipment according to the target detection module; wherein the angle adjustment component can be configured to change the monitoring range of the monitoring equipment through a motor. When the fire sensor detects the occurrence of a fire, the angle of the monitoring equipment is adjusted by the angle adjustment component to monitor the fire scene; the specific structure and working principle of the angle adjustment component can be referred to the invention patent with publication number CN112911224B, a high-definition camera module for a video surveillance system, which will not be repeated here.

[0054] Among them, the multiple target detection modules include:

[0055] The first target detection unit is configured to perform a full-image scan on the image data to detect whether there is any foreign object intruding in the monitoring area; the triggering condition of the first target detection unit matches the sensing component in the above embodiment; the second target detection unit is configured to perform a full-image scan on the image data to detect whether there is any natural disaster in the monitoring area; the triggering condition of the second detection unit matches the detection node in the above embodiment; the third target detection unit is configured to perform a full-image scan on the image data to detect whether the construction personnel in the monitoring area wear protective gear; the fourth target detection unit is configured to perform a full-image scan on the image data to detect whether there is any abnormal traffic event in the monitoring area;

[0056] In this embodiment, the first target detection unit and the second target detection unit are used to realize early prediction of abnormal events in the monitoring area, that is, there are factors that trigger abnormal events in the monitoring area but no abnormal events actually occur. Compared with the existing technology, this method belongs to active monitoring, which is more conducive to preventing abnormal events from occurring during the construction process; combined with the third target detection unit and the fourth target detection unit, the detection accuracy of abnormal events is effectively improved to avoid missed detections.

[0057] The server is a cloud server, and the cloud server is configured to receive image information transmitted by the target detection module, decompress it, and store it in a database in partitions.

[0058] The system also includes: an early warning notification module for identifying whether there is an abnormal event in the monitoring area. If there is an abnormal event, the corresponding early warning information is obtained, and the early warning information at least includes: time, geographic location coordinates and abnormal event type.

[0059] See also Figure 3 As shown, the second technical solution of the present invention provides an abnormal event detection method, which is applied to the video surveillance system described in any one of the technical solutions of the first technical solution of the present invention, comprising:

[0060] Step S1, collecting multiple categories of data set information according to a preset ratio and annotating them to the data set respectively, and dividing them according to the preset ratio to obtain multiple training sets and verification sets;

[0061] In step S1, the multiple categories of data set information include: data set information related to foreign object intrusion, data set information related to natural disasters, data set information related to wearing of protective gear, and data set information related to traffic anomalies;

[0062] Step S2, inputting multiple training sets into the training model for model training, inputting the verification sets corresponding to the training sets into the training module for verification, and embedding the trained models into the corresponding target detection units respectively;

[0063] In step S2, the training process of the first target detection unit includes: inputting the image information of the illegal intrusion characteristics in the foreign object intrusion data set information into the training model for iterative training, and after the training is completed, inputting the illegal intrusion feature image in the verification data into the model for verification and obtaining the verification result; the training process of the second target detection unit includes: inputting the image information of the natural disaster characteristics in the natural disaster data set information into the training model for iterative training, and after the training is completed, the natural disaster feature image in the processed verification data is input into the model for verification and obtaining the verification result; the model training process of the third target detection unit includes: inputting the image information of the construction personnel wearing construction equipment in the data set into the training model for iterative training, and after the training is completed, the construction personnel feature image in the verification data is input into the model for verification and obtaining the verification result; the training process using the fourth target detection unit includes: inputting the image information of the traffic abnormality characteristics in the data set into the training model for iterative training, and after the training is completed, the traffic abnormality feature image in the verification data is input into the model for verification and obtaining the verification result;

[0064] Step S3, determining the layout area of ​​the electronic fence according to the construction area, dividing the electronic fence layout area and setting cross-region rules and abnormal event level determination rules;

[0065] In step S3, the division of the cross-regional rules specifically includes: establishing a first rule editor, corresponding to appropriate cross-regional rules according to the regional division in the construction area, for example, in a plateau environment, different cross-regional rules are formulated in consideration of factors such as the natural environment and personnel protection during the construction process; the division of the abnormal event level rules specifically includes: establishing a second rule editor, corresponding to appropriate abnormal event levels according to different abnormal events in the above-mentioned multiple data set information; it should be noted that in this embodiment, the cross-regional rules and abnormal event level determination rules should be determined according to the specific conditions of the monitoring area and the construction area, and the present disclosure does not limit the cross-regional rules and abnormal event level determination rules;

[0066] Step S4, retrieve the monitoring picture according to the detection result of the preset detection node and pre-process the monitoring picture;

[0067] Among them, monitoring screen preprocessing includes:

[0068] Split the surveillance screen into frame images, and unify the image size to 416×416 to reduce the impact of different size inputs on the reduced target detection capability. Input them into multiple target detection units to detect abnormal events. If a certain category is detected, the detection category is output and the warning information is marked.

[0069] If there is no abnormal event, no warning information will be output directly.

[0070] Step S5, using the target detection module to obtain the target detection result of the pre-processed monitoring image;

[0071] Step S6, obtaining the type and level of the abnormal event according to the target detection result.

[0072] Among them, the obtaining of target detection results also includes: inputting the video clip into the spatial stream convolutional neural network and the temporal stream convolutional neural network of the M-I3D module for feature extraction, and using a multi-example sorting model to output anomaly scores according to the features to determine whether an abnormal event has occurred. The target monitoring method in this embodiment is mainly applied to the third target detection unit in the above embodiment to realize the behavior recognition of construction personnel and the behavior recognition of construction personnel in a relatively complex construction environment. The specific recognition method can refer to the invention patent with publication number CN114724236A, a behavior recognition method and system based on 3D convolutional neural network, which will not be repeated here.

[0073] The abnormal event detection method provided by the second aspect of the technical solution of the present invention is applied to the video surveillance system described in any one of the technical solutions of the first aspect of the present invention, and thus it has all the beneficial effects of the video surveillance system described in the first aspect of the technical solution of the present application, which will not be repeated here.

[0074] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. Video surveillance system, characterized in that: The system comprises a server and at least one electronic fence arranged in a monitoring area, wherein the server is in communication connection with the electronic fence, and the electronic fence comprises: At least one data collection device, used to collect environmental data within the monitoring area; At least one detection node, used to detect the change trend of detection data in the monitoring area and predict abnormal events; At least one monitoring device, used to obtain video image information within the monitoring area; at least one sensing component for identifying intrusion into or out of the electronic fence; Display screen, used to display video image monitoring data and danger alarm data; A main control module, which is equipped with a target detection module, wherein the target detection module is configured to perform target detection based on the image data collected by the monitoring device to determine a target display area where a target detection object exists; The abnormal event detection method of the video surveillance system comprises the following steps: Collect information of multiple categories of data sets according to preset ratios and annotate them respectively, and divide them into multiple training sets and validation sets according to preset ratios; Multiple training sets are input into the training model for model training, and the verification sets corresponding to the training sets are input into the training module for verification, and the trained models are embedded into the corresponding target detection units respectively; Determine the layout area of ​​the electronic fence according to the construction area, divide the area of ​​the electronic fence and set cross-region rules and rules for determining the level of abnormal events; According to the detection results of the preset detection nodes, the monitoring screen is retrieved and preprocessed; the monitoring screen preprocessing includes: Split the monitoring screen into frame images, input them into multiple target detection units to detect abnormal events, and if a certain category is detected, output the detected category and mark the warning information; If there is no abnormal event, directly output no warning information; The target detection module is used to obtain the target detection results of the preprocessed surveillance images, and the video clips are input into the spatial stream convolutional neural network and temporal stream convolutional neural network of the M-I3D module for feature extraction. The multi-instance ranking model is used to output anomaly scores based on the features to determine whether an abnormal event has occurred. Obtain the type and level of abnormal events based on the target detection results.

2. The video surveillance system according to claim 1, characterized in that: The monitoring device is equipped with an angle adjustment component, and the angle adjustment component is configured to adjust the angle parameter of the monitoring device according to the detection result of the target detection module.

3. The video surveillance system according to claim 2, characterized in that: The system also includes: The fire sensor is arranged in the monitoring area and is connected to the monitoring device for communication. When a fire is detected, the angle of the monitoring device is adjusted by the angle adjustment component to monitor the fire.

4. The video surveillance system according to claim 1, characterized in that: The plurality of target detection modules include: A first target detection unit is configured to perform a full-image scan on the image data to detect whether there is any foreign object intruding into the monitoring area; A second target detection unit is configured to perform a full-image scan on the image data to detect whether a natural disaster occurs in the monitoring area; A third target detection unit is configured to perform a full-image scan on the image data to detect whether the construction workers in the monitoring area are wearing protective gear; The fourth target detection unit is configured to perform a full-image scan on the image data to detect whether there are any abnormal traffic events in the monitoring area.

5. The video surveillance system according to claim 1, characterized in that: The server adopts a cloud server, and the cloud server is configured to receive image information transmitted by the target detection module, decompress it, and then partition and store it in a database.

6. The video surveillance system according to claim 1, characterized in that: The environmental data information collected by the data collection device at least includes: temperature and humidity, oxygen concentration, weather, ultraviolet radiation, and atmospheric pressure.

7. The video surveillance system according to claim 1, characterized in that: The system also includes: an early warning notification module, which is used to issue early warning information based on whether there is an abnormal event in the monitoring area. If there is an abnormal event, the corresponding early warning information is obtained, and the early warning information at least includes: time, geographical location coordinates and abnormal event type.

Images