Cabinet abnormal state detection system and method based on smart city
Through the box abnormality detection system of smart cities, camera video data and neural network model are used to combine operation logs and face recognition, the rapid identification and labeling of box abnormal door opening is achieved, and the monitoring efficiency in the existing technology is solved. The problem of artificial dependence improves the accuracy and efficiency of monitoring.
Patent Information
- Application Number
- CN202210095820.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-26
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2042-01-26
AI Technical Summary
The existing box abnormal door opening monitoring solution cannot achieve real-time analysis, and requires manual processing, low work efficiency, and the door magnetic sensor is easy to be damaged or cannot upload data, so it cannot identify normal and abnormal door opening scenarios.
The box abnormal state detection system based on smart city, obtains video data in real time through the camera, processes it in frames and inputs it to the neural network model for judgment, combines the operation log to determine whether it is abnormally opened, and marks the preset logo on the picture, giving priority to processing high Priority video data reduces the pressure on neural network models, and uses face recognition and weather information to optimize monitoring.
It realizes the rapid determination of whether the box door is abnormally opened, improves monitoring efficiency, reduces manual processing requirements, and ensures accurate identification and timely response to abnormal door opening.
Smart Images

Figure CN114500950B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of video surveillance technology, and in particular to a system and method for detecting abnormal state of a cabinet based on a smart city. Background Art
[0002] In the construction of smart cities, ordinary people are not allowed to enter some dangerous areas and important places, such as box-type transformer stations with high voltage electricity, computer rooms where important equipment is deployed, etc. In order to avoid personal safety accidents or loss of public equipment caused by abnormal opening of the box door, it is necessary to monitor and alarm the abnormal opening status of the box door in real time.
[0003] The commonly used solution for monitoring abnormal door opening of cabinets at home and abroad is to use door magnetic sensors. This monitoring solution can only monitor the opening and closing status of the cabinet door, but cannot realize real-time analysis of the door opening scene, which has great limitations. For example, the cabinet may not have the conditions for door magnetic installation, or the door magnetic data cannot be uploaded to the remote control center; it cannot intelligently identify the scene, and cannot identify whether it is a normal opening or an abnormal opening; the door magnetic element is easily damaged, resulting in the inability to monitor the cabinet door opening normally, etc.
[0004] In other solutions, video equipment linkage is carried out, and video surveillance equipment is installed to monitor the box 24 hours a day. When the door magnetic sensor detects that the box is abnormally opened, an alarm message is pushed, and then the management personnel turn on the corresponding video surveillance equipment to browse in real time or play back historical video data. However, this monitoring method requires manual processing and has low work efficiency.
[0005] To this end, a smart city-based cabinet abnormal state detection system and method is needed that can quickly determine whether the cabinet door is abnormally opened. Summary of the invention
[0006] The present invention provides a box abnormal state detection system and method based on smart city, which can quickly determine whether the box door is abnormally opened.
[0007] In order to solve the above technical problems, this application provides the following technical solutions:
[0008] The abnormal state detection system of the cabinet based on smart city includes:
[0009] The docking module is used to obtain the video data collected by the camera in real time and determine whether the camera belongs to the preset area;
[0010] A framing module, used to perform framing processing on the corresponding video data to generate a number of pictures when the camera belongs to a preset area;
[0011] The analysis module is used to input the image into the neural network model for judgment and obtain the judgment result; when the judgment result is that the door is opened, it is judged whether it is an abnormal door opening based on the operation log;
[0012] The identification module is used to mark a preset identification on the picture when the result is judged as abnormal door opening.
[0013] The basic scheme principles and beneficial effects are as follows:
[0014] In this solution, after acquiring video data in real time and determining whether the camera belongs to the preset area, the subsequent processing measures for the video data corresponding to the camera can be determined. For example, if it does not belong to the preset area, it indicates that the importance of the video data corresponding to the camera is low, and manual monitoring can continue to be used. When the camera belongs to the preset area, the corresponding video data is processed by frame division, and the video data is converted into frame-by-frame pictures, and then the pictures are input into the neural network model for judgment, and the judgment result of whether the door of the box is open can be obtained. When the judgment result is that the door is open, it can be accurately determined whether it is an abnormal door opening through the operation log. Finally, the preset mark is marked on the picture of the abnormal door opening. When the user is reviewing, it is convenient for the user to quickly determine the corresponding picture when the door opening abnormality occurs.
[0015] In summary, this solution can quickly determine whether the door is opened abnormally.
[0016] Furthermore, it also includes a recognition module for judging whether there are people in the monitoring area based on the video data collected by the camera in the preset area; the analysis module is used to extract pictures at preset time intervals when there are no people in the monitoring area, and input the pictures into the neural network model for judgment.
[0017] When there are no people in the monitoring area, the possibility of the door opening abnormally is small. At preset time intervals, pictures are extracted and input into the neural network model for judgment, rather than judging the pictures in real time, which can reduce the processing pressure of the neural network model.
[0018] Furthermore, the recognition module is also used to perform face recognition when there are people in the monitoring area to determine whether the people are staff members. If they are not staff members, the corresponding video data is marked as high-priority video data; if they are staff members, the corresponding video data is marked as low-priority video data;
[0019] The framing module is also used to give priority to framing high-priority video data corresponding to cameras in a preset area.
[0020] By collecting the staff's face data in advance, the staff can be accurately identified through face recognition. When the staff appears in the monitoring area, that is, near the box, the possibility of maintenance or inspection is high, and the corresponding video data is marked as low-priority video data. When the frame processing module is busy, it does not need to be processed immediately.
[0021] When non-staff appear near the box, the possibility of abnormal door opening is high. The corresponding video data will be marked as high-priority video data. When the framing module is busy, it can be guaranteed to be processed first, which is convenient for subsequent rapid identification and judgment.
[0022] Furthermore, the analysis module is also used to preferentially input images corresponding to high-priority video data into the neural network model for judgment.
[0023] The current face recognition solution is relatively mature. You can directly purchase existing equipment or connect to existing face recognition platforms. For example, existing face recognition platforms usually serve many customers, and can set up more servers for face recognition processing and handle large amounts of data in parallel. However, the neural network model is developed separately and is only used for door opening recognition. It has a limited scope of use and a limited budget, and cannot be configured with too much hardware. Therefore, performance bottlenecks are prone to occur. By giving priority to inputting the pictures corresponding to high-priority video data into the neural network model for judgment, when the neural network model is busy, the judgment result of whether to open the door can still be quickly obtained.
[0024] Further, the analysis module determines the preset number of pictures input into the neural network model based on the high priority video data and the low priority video data, wherein the preset number of pictures corresponding to the high priority video data is greater than the preset number of pictures corresponding to the low priority data.
[0025] This ensures that when non-staff appear near the box, adequate identification and judgment can be made.
[0026] Furthermore, the frame segmentation module is also used to sequentially number the pictures;
[0027] The analysis module extracts a picture at a preset number of intervals and inputs it into the neural network model until the number of input pictures equals the preset number of pictures or the judgment result of the neural network model is that the door is open.
[0028] Furthermore, it also includes a meteorological processing module for obtaining weather information and determining whether the current weather belongs to a preset weather type based on the weather information; the analysis module is also used to shorten the preset time for extracting pictures when there are no people in the monitoring area and the current weather belongs to the preset weather type.
[0029] For example, the preset weather type is that the wind force level exceeds level 5. In this case, the possibility of the door being blown open by the wind increases because it is not closed properly. Shortening the preset time for extracting pictures can increase the judgment interval and facilitate early detection of abnormal opening of the door.
[0030] Furthermore, it also includes a verification module, which is used to determine whether the operation log records the door opening plan at the current time when the judgment result of the neural network model is that the door is not opened. If so, an early warning message is generated.
[0031] The operation log records the door opening plan at the current time, but the neural network model determines that the door is not opened. There may be two situations: one is that the staff opened the door normally but the neural network model did not recognize it correctly, and the other is that the neural network model recognized it correctly but the staff did not open the door according to the door opening plan. By generating early warning information, it is convenient for relevant management personnel to verify and handle it.
[0032] The abnormal state detection method of the cabinet based on the smart city uses the above system to detect the abnormal state of the cabinet. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 This is Example 1, a cabinet abnormal state detection system based on smart city. DETAILED DESCRIPTION
[0034] The following is further described in detail through specific implementation methods:
[0035] Embodiment 1
[0036] like Figure 1 As shown, the smart city-based cabinet abnormal state detection system of this embodiment includes a docking module, a framing module, an analysis module, an identification module and a recognition module.
[0037] The docking module is used to obtain the video data collected by the camera in real time and determine whether the camera belongs to the preset area. In this embodiment, the preset area is set manually, for example, the area that needs to be monitored is set as the preset area.
[0038] The recognition module is used to determine whether there are people in the monitoring area based on the video data collected by the camera in the preset area;
[0039] The recognition module is also used to perform face recognition when there are people in the monitoring area to determine whether the person is registered. If not, the corresponding video data is marked as high-priority video data; if already registered, the corresponding video data is marked as low-priority video data.
[0040] The framing module is used to perform framing processing on the corresponding video data when the camera belongs to a preset area to generate a plurality of pictures. The framing module is also used to sequentially number the pictures. In this embodiment, Arabic numerals are used for sequential numbering. When the framing module performs framing processing, the high-priority video data corresponding to the camera in the preset area is preferentially processed.
[0041] The analysis module is used to extract pictures at preset time intervals when there are no people in the monitoring area, and input the pictures into the neural network model for judgment. In this embodiment, one picture is extracted.
[0042] The analysis module is also used to determine the preset number of pictures input into the neural network model based on the high-priority video data and the low-priority video data when there are people in the monitoring area, and preferentially input the pictures corresponding to the high-priority video data into the neural network model for judgment. Among them, the preset number of pictures corresponding to the high-priority video data is greater than the preset number of pictures corresponding to the low-priority data. In this embodiment, when the analysis module inputs the pictures into the neural network model, one picture is extracted every preset number of pictures and input into the neural network model until the number of pictures input is equal to the preset number of pictures or the judgment result of the neural network model is to open the door. For example, the preset number of pictures corresponding to the high-priority video data is 20, and one picture is extracted every 5 pictures and input into the neural network model.
[0043] The analysis module is also used to obtain the judgment result from the neural network model; when the judgment result is that the door is opened, it is judged whether it is an abnormal door opening based on the operation log; in this embodiment, before the staff conducts inspections, maintenance and other operations, there will be a corresponding inspection plan or maintenance plan, in which the content related to entering the box can be extracted as a door opening plan and added to the operation log as a basis for judgment. The time of door opening can be obtained through video data, and it is judged whether there is a door opening plan for the corresponding time recorded in the operation log. If not, it is judged as abnormal door opening.
[0044] The identification module is used to mark a preset identification on the picture when the result of the judgment is abnormal door opening. In this embodiment, the annotation parameters can also be set, and the preset identification is marked on the picture based on the annotation parameters, wherein the annotation parameters include the size of the identification, the position of the identification and the transparency of the identification. The size of the identification is the length of the identification * the width of the identification. The position of the identification is the distance of the identification from the left edge of the picture and the distance from the upper edge of the picture. The transparency of the identification is expressed as a percentage, for example, the transparency is 100%, that is, the identification is completely invisible. The identification is made of, for example, a picture formed by rasterizing the text "Abnormal Door Opening". In this embodiment, the preset identification is marked on the picture by FFmpeg.
[0045] This embodiment also provides a method for detecting abnormal state of a cabinet based on a smart city, and uses the above system to detect abnormal state of a cabinet.
[0046] In order to use the neural network model for judgment, this embodiment also provides a training method for the neural network model, including the following steps:
[0047] S1. Obtain video data and generate several pictures by dividing the frames. In this embodiment, the frame division uses the segmentation component in OpenCV.
[0048] S2. Classify and label the images to build a training image set. Labels include open door and not open door.
[0049] S3, input the training picture set into the neural network model for training; the neural network model is a convolutional neural network model. In this embodiment, the convolutional neural network model uses the convolutional neural network model component in OpenCV.
[0050] Embodiment 2
[0051] The difference between this embodiment and the first embodiment is that the system of this embodiment further includes a meteorological processing module. The meteorological processing module is used to obtain weather information and determine whether the current weather belongs to a preset weather type based on the weather information; the analysis module is also used to shorten the preset time for extracting pictures when there are no people in the monitoring area and the current weather belongs to the preset weather type. In this embodiment, the preset weather type is a wind force level exceeding level 5. For example, the preset time is 60 seconds, and when the wind force level exceeds level 5, it is shortened to 30 seconds.
[0052] Embodiment 3
[0053] The difference between this embodiment and the first embodiment is that the system of this embodiment further includes a verification module. The verification module is used to determine whether there is a door opening plan at the current time recorded in the operation log when the neural network model determines that the door is not opened, and if so, generate warning information.
[0054] Embodiment 4
[0055] The difference between this embodiment and the first embodiment is that the system of this embodiment further includes a slicing module, a correction module and a storage module.
[0056] The slicing module is used to slice the video data and generate a description file and a plurality of media segments; the description file is used to record the shooting date, total duration, and number and duration of each media segment of the video data. In this embodiment, the video data is sliced in milliseconds.
[0057] In this embodiment, the description file uses an m3u8 file and the media segment uses a ts file. For example, the total duration of the video data is 10 seconds, the slices are 10 ts files, the duration of a single ts file is 1 second, and the ts files are numbered from 001 to 010. The shooting date is, for example, 2021-7-15-12:01:00:001.
[0058] The recognition module is also used to perform binarization processing on the media segment, and determine whether the binarized media segment contains a preset identification object. If the preset identification object is contained, the correction module processes it, and if the preset identification object is not contained, the storage module processes it. In this embodiment, each frame of the media segment is binarized. The preset identification objects include people, animals, etc.
[0059] The correction module is used to perform noise reduction on the media segment. In this embodiment, the media segment is firstly grayed and then subjected to Gaussian filtering for noise reduction.
[0060] The storage module is used to store the description file and the media segment after the noise reduction process.
[0061] This embodiment can store the video data after processing, which is convenient for subsequent calls. Since the video data is composed of frames of pictures, it is impossible to completely separate the continuous frames when slicing, that is, between two continuous media segments, several frames may be lost, resulting in loss of picture quality. By performing noise reduction processing on the sliced media segments, the quality of the picture can be effectively improved. However, if all media segments are subjected to noise reduction processing, a large amount of computing resources will be consumed. Moreover, when no person or animal passes through the monitoring range of the camera, the picture is static, and the difference between each frame is small. Even if several frames are lost in the middle, it will not cause too much impact on the picture quality. In this embodiment, the preset identification object can be a person or an animal according to the monitored object. For example, the preset identification object is an animal. When the media segment contains an animal, the media segment is subjected to noise reduction processing. Since animals appear in the picture, the difference between each frame becomes larger. If several frames are lost in the middle, the impact on the picture quality becomes larger. At this time, noise reduction processing is performed to improve the clarity of the picture to offset the impact of the possible loss of several frames and reduce the loss of picture quality. Moreover, if animals appear in the picture, the media clip is more likely to be called up and viewed by staff later, and the viewing experience can be improved after noise reduction processing.
[0062] When the recognition module is processing, if a preset identifier is included, the type of the preset identifier is also determined; based on the type of the preset identifier, a first preset quantity value is determined, and then the recognition module processing is omitted for the first preset quantity of media segments after the media segment, and the correction module is directly processed.
[0063] Since the monitoring range of the camera is certain, under normal circumstances, the time that people and animals in the tunnel pass under the camera is also within a certain range (due to the fast speed of animals, the time will be shorter than the time that people pass). When an animal or a person appears within the monitoring range of the camera, it is reflected in the media clip, that is, it contains a preset identification object. It takes a certain amount of time for animals or people to leave the monitoring range. The possibility of containing animals or people in the media clips during this period of time is high. Therefore, omitting the processing of the recognition module and directly processing it by the correction module can further simplify the recognition process and save computing resources. In this embodiment, the first preset number is determined comprehensively based on the estimated average speed of the preset identification object, the duration of the media clip and the monitoring range of the camera. The estimated average speed can be determined based on the installation location of the camera.
[0064] Furthermore, in this embodiment, the actual average speed of the preset identification object is calculated based on the moving distance of the preset identification object in the media segment, and it is determined whether the absolute value of the difference between the actual average speed and the estimated average speed and the ratio of the estimated average speed are greater than a threshold, that is, |V1-V2| / V2, where V1 is the actual average speed and V2 is the estimated average speed.
[0065] If it is greater than the threshold, the first preset number is determined based on the estimated average speed of the preset identification object, the duration of the media segment, and the monitoring range of the camera. If it is less than or equal to the threshold, the first preset number is determined based on the actual average speed of the preset identification object, the duration of the media segment, and the monitoring range of the camera. In this embodiment, the threshold is 20%.
[0066] The above are only embodiments of the present invention. The invention is not limited to the fields involved in this implementation case. The common sense such as the known specific structures and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to explain the content of the claims.
Claims
1. Box abnormal state detection system based on smart city, It is characterized in that include: The docking module is used to obtain the video data collected by the camera in real time and determine whether the camera belongs to the preset area; A framing module is used to perform framing processing on the corresponding video data when the camera belongs to a preset area to generate a number of pictures, and is also used to give priority to framing processing on the high-priority video data corresponding to the camera in the preset area; The recognition module is used to determine whether there are people in the monitoring area based on the video data collected by the camera in the preset area. If there are people in the monitoring area, face recognition is performed to determine whether the people are staff members. If they are not staff members, the corresponding video data is marked as high-priority video data. If they are staff members, the corresponding video data is marked as low-priority video data; The identification module is used to mark the image with a preset identification when the result is abnormal door opening; A weather processing module is used to obtain weather information and determine whether the current weather belongs to a preset weather type based on the weather information; The analysis module is used to extract the content entering the box from the inspection plan or maintenance plan, and add it to the operation log as a door opening plan; input the picture into the neural network model for judgment to obtain a judgment result; when the judgment result is that the door is opened, judge whether it is an abnormal door opening based on the operation log; when there is no person in the monitoring area, extract the picture at a preset time interval, and input the picture into the neural network model for judgment; when there is no person in the monitoring area and the current weather belongs to the preset weather type, shorten the preset time for extracting the picture.
2. According to the smart city-based cabinet abnormal state detection system according to claim 1, Features: The analysis module is also used to preferentially input the pictures corresponding to the high priority video data into the neural network model for judgment.
3. According to the smart city-based cabinet abnormal state detection system according to claim 2, Features: The analysis module determines the preset number of pictures input into the neural network model based on the high-priority video data and the low-priority video data, wherein the preset number of pictures corresponding to the high-priority video data is greater than the preset number of pictures corresponding to the low-priority data.
4. According to the smart city-based cabinet abnormal state detection system according to claim 3, Features: The frame segmentation module is also used to sequentially number the pictures; The analysis module extracts a picture at a preset number of intervals and inputs it into the neural network model until the number of input pictures equals the preset number of pictures or the judgment result of the neural network model is that the door is open.
5. According to the smart city-based cabinet abnormal state detection system according to claim 4, Features: It also includes a verification module, which is used to determine whether the operation log records a door opening plan for the current time when the neural network model determines that the door is not opened. If so, an early warning message is generated.
6. Abnormal state detection method of cabinet based on smart city, It is characterized in that The steps include: S1, obtaining video data collected by the camera, and determining whether the camera belongs to a preset area; S2. Obtain weather information, and determine whether the current weather belongs to a preset weather type based on the weather information; S3, judging whether there are people in the monitoring area according to the video data collected by the camera in the preset area, and if there are people in the monitoring area, performing face recognition to judge whether the people are staff members, if they are not staff members, marking the corresponding video data as high priority video data, and if they are staff members, marking the corresponding video data as low priority video data; S4, performing frame processing on the video data corresponding to the camera in the preset area to generate a plurality of pictures, and giving priority to performing frame processing on the high-priority video data corresponding to the camera in the preset area; S5. Extract the contents entering the box from the inspection schedule or maintenance schedule and add them to the operation log as a door opening schedule; S6, input the picture into the neural network model for judgment, and obtain the judgment result; when the judgment result is that the door is opened, determine whether it is an abnormal door opening based on the operation log; when there is no person in the monitoring area, extract the picture at a preset time interval, and input the picture into the neural network model for judgment; when there is no person in the monitoring area and the current weather belongs to the preset weather type, shorten the preset time for extracting the picture; S7. When the result of the judgment is abnormal door opening, a preset mark is marked on the picture.
Citation Information
Patent Citations
Video image feature recognition method and system
CN112507824A
Anti-theft alarm evidence obtaining method and system for intelligent lamp pole
CN113393625A
Box door different opening detection system and method based on video analysis
CN113923418A
