Abnormal recognition result processing method applied to smart city box detection

Through real-time video data processing and neural network model judgment, combined with operation log and labeling technology, the problem of real-time analysis and intelligent identification of box abnormal door monitoring in the existing technology is solved, and rapid retrieval and labeling is realized, improving work efficiency and identification are improved. accuracy.

CN114429617BActive Publication Date: 2024-11-01CHINA COAL SCI & IND GRP CHONGQING SMART CITY SCI & TECH RES INST CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210095823.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-26
Publication Date
2024-11-01
Estimated Expiration
2042-01-26

AI Technical Summary

Technical Problem

The existing box abnormal door opening monitoring solution cannot achieve real-time analysis and intelligent identification, and requires manual processing, resulting in low work efficiency and the inability to quickly retrieve pictures or videos of abnormal door opening.

Method used

By obtaining video data in real time, processing it in frames and inputting the neural network model for judgment, obtaining the door opening status of the box, combining the operation log to determine whether it is an abnormal door opening, and marking the preset logo on the picture to quickly retrieve and review the abnormal situation.

Benefits of technology

It realizes rapid identification and labeling of abnormal door opening of the box, improves the efficiency of users during review, and ensures accurate identification and rapid positioning of abnormal door opening.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114429617B_ABST
    Figure CN114429617B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of video surveillance technology, and specifically discloses an abnormal recognition result processing method applied to smart city box detection, comprising the following steps: S1, real-time acquisition of video data, frame processing of the video data, and generation of a number of pictures; S2, inputting the pictures into a neural network model for judgment, and obtaining a judgment result; S3, when the judgment result is a door opening, judging whether it is an abnormal door opening based on an operation log; S4, if it is an abnormal door opening, marking a preset mark on the picture. The technical solution of the present invention can quickly retrieve abnormal situations that have occurred.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and in particular to a method for processing abnormal recognition results applied to smart city box detection. Background Art

[0002] In the construction of smart cities, ordinary people are not allowed to enter some dangerous areas and important places, such as box-type transformer stations with high voltage electricity, computer rooms where important equipment is deployed, etc. In order to avoid personal safety accidents or loss of public equipment caused by abnormal opening of the box door, it is necessary to monitor and alarm the abnormal opening status of the box door in real time.

[0003] The commonly used solution for monitoring abnormal door opening of cabinets at home and abroad is to use door magnetic sensors. This monitoring solution can only monitor the opening and closing status of the cabinet door, but cannot realize real-time analysis of the door opening scene, which has great limitations. For example, the cabinet may not have the conditions for door magnetic installation, or the door magnetic data cannot be uploaded to the remote control center; it cannot intelligently identify the scene, and cannot distinguish between normal opening and abnormal opening; the door magnetic element is easy to damage, resulting in the inability to monitor the cabinet door opening normally, etc.

[0004] In other solutions, video equipment linkage is carried out, and video surveillance equipment is installed to monitor the box 24 hours a day. When the door magnetic sensor detects that the box is abnormally opened, an alarm message is pushed, and then the management personnel turn on the corresponding video surveillance equipment to browse in real time or play back historical video data. However, this monitoring method requires manual processing and has low work efficiency.

[0005] To this end, the applicant applied for a computer vision-based method for identifying abnormal cabinet states on the same day, which automatically identified the door opening state by training a neural network model. However, when users need to review, they cannot intuitively and quickly find the corresponding pictures or videos when the abnormal door opening occurs from a large number of pictures or videos.

[0006] Therefore, there is a need for a method for processing abnormal identification results that can be quickly retrieved and applied to smart city box detection. Summary of the invention

[0007] The present invention provides an abnormality recognition result processing method applied to smart city box detection, which can quickly retrieve abnormal situations that have occurred.

[0008] In order to solve the above technical problems, this application provides the following technical solutions:

[0009] The method for processing abnormal recognition results applied to smart city box detection includes the following steps.

[0010] S1, real-time acquisition of video data, frame processing of the video data, and generation of several pictures;

[0011] S2, input the picture into the neural network model for judgment, and obtain the judgment result;

[0012] S3. When the result of the judgment is that the door is opened, judging whether it is an abnormal door opening based on the operation log;

[0013] S4. If the door is opened abnormally, a preset mark will be placed on the picture.

[0014] The basic scheme principles and beneficial effects are as follows:

[0015] In this solution, after acquiring video data in real time, frame processing is performed to convert the video data into pictures frame by frame, and then the pictures are input into the neural network model for judgment, so as to obtain the judgment result of whether the door of the cabinet is open. When the judgment result is that the door is open, the operation log can accurately determine whether it is an abnormal door opening. Finally, a preset mark is marked on the picture of abnormal door opening. When the user is reviewing, it is convenient for the user to quickly search and determine the corresponding picture when the door opening abnormality occurs.

[0016] Furthermore, in S2, the position information of the door is also obtained from the neural network model;

[0017] In S4, if the door is opened abnormally, the door is marked on the picture based on the position information of the door.

[0018] The door of the box is marked to facilitate users to quickly determine the location of the door, and at the same time, to verify whether the recognition of the neural network model is accurate.

[0019] Furthermore, the method further includes S5, reintegrating the labeled pictures and the unlabeled pictures into a video stream.

[0020] When the user is watching the video again, even if the video is dragged, it is not easy to miss the abnormal door opening situation.

[0021] Furthermore, S2 is also used to preprocess the image and input the preprocessed image into the neural network model.

[0022] By preprocessing the images, it is easier for the neural network model to recognize them.

[0023] Furthermore, in S3, the door opening time is also obtained to determine whether the door opening plan for the corresponding time is recorded in the operation log. If not, it is determined to be an abnormal door opening.

[0024] During normal inspection or maintenance by staff, the door opening plan will be recorded in the operation log. When there is no door opening plan at the corresponding time in the operation log, it indicates that it is not a normal operation by the staff, so it can be judged as abnormal door opening.

[0025] Furthermore, in S4, annotation parameters are obtained, and a preset mark is marked on the image based on the annotation parameters, wherein the annotation parameters include the size of the mark, the position of the mark, and the transparency of the mark.

[0026] It is convenient to adjust the logo.

[0027] Furthermore, in S4, the box is marked on the picture based on the position information of the box.

[0028] Furthermore, in S4, the door and the body are marked by selecting the door and the body by a rectangular frame.

[0029] Furthermore, in S4, a preset mark is marked on the picture through FFmpeg. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flow chart of a method for processing abnormal identification results applied to smart city cabinet detection according to Example 1. DETAILED DESCRIPTION

[0031] The following is further described in detail through specific implementation methods:

[0032] Embodiment 1

[0033] like Figure 1 As shown, the abnormal recognition result processing method applied to smart city box detection in this embodiment includes the following steps:

[0034] S1. Acquire video data in real time, perform frame processing on the video data, and generate a number of pictures.

[0035] In this embodiment, video data in .264 format is obtained and decoded by FFMpeg. The decoded video data is processed by the segmentation component in OpenCV.

[0036] S2. Preprocess the image, input the preprocessed image into the neural network model for judgment, and obtain the judgment result, the position information of the door and the position information of the box body. In this embodiment, during preprocessing, the image is converted into a grayscale image, the grayscale image is decomposed into eight planes, and one of the plane images is extracted and input into the neural network model for judgment.

[0037] S3. When the result of the judgment is that the door is opened, the time of the door opening is also obtained to judge whether there is a door opening plan at the corresponding time recorded in the operation log. If not, it is judged as abnormal door opening.

[0038] In this embodiment, before the staff conducts inspections, maintenance and other operations, there will be a corresponding inspection plan or maintenance plan, in which the content related to entering the box can be extracted as a door opening plan and added to the operation log as a basis for judgment.

[0039] S4. If the door is opened abnormally, the door is marked on the picture based on the position information of the door; the box body is marked on the picture based on the position information of the box body. In this embodiment, the door and the box body are marked by selecting the door and the box body by a rectangular frame. A marking parameter is also obtained, and a preset mark is marked on the picture based on the marking parameter, wherein the marking parameter includes the mark size, the position of the mark, and the transparency of the mark.

[0040] In this embodiment, the size of the logo is the length of the logo*the width of the logo. The position of the logo is the distance of the logo from the left edge of the image and the distance from the upper edge of the image. The transparency of the logo is expressed as a percentage, for example, 100% transparency means that the logo is completely invisible. The logo is made of, for example, a picture formed by rasterizing the text "door opening abnormality".

[0041] S5, reintegrate the marked and unmarked pictures into a video stream. In this embodiment, the door, the box body and the preset logo are marked on the picture by FFmpeg. The marked and unmarked pictures are also reintegrated into a video stream by FFmpeg.

[0042] In order to use the neural network model for judgment, this embodiment also provides a training method for the neural network model, including the following steps:

[0043] A1. Obtain video data in a preset format and decode it. In this embodiment, the preset format is H.264 format, which is decoded by FFMpeg. In order to reduce the size of the video and reduce the bandwidth occupied during transmission, the video will be encoded and compressed to reduce the size. In the H.264 format, the encoding algorithm is intra-frame compression and inter-frame compression. Intra-frame compression is an algorithm for generating I frames, and inter-frame compression is an algorithm for generating B frames and P frames. Among them, I frames are key frames, and the information recorded by P frames is the difference between the frame and the previous key frame (or P frame). B frames are bidirectional difference frames, and the recorded information is the difference between the frame and the previous and next frames. Therefore, decoding is required to restore each frame in the video data.

[0044] A2. Frame the decoded video data according to the first preset rule to generate several pictures. In this embodiment, the frame division adopts the segmentation component in OpenCV. In this embodiment, the first preset rule includes the interval of processing frames, the size of the picture and the storage path of the picture. The interval of processing frames, that is, how many frames to extract a picture. For example, if the frame rate is 25fps, the interval of processing frames is 25, that is, 1 picture is generated in 1 second.

[0045] A3. Preprocess the images, classify and label them, and build a training image set. Labels include open and closed doors. They also include box doors and box bodies.

[0046] In the preprocessing, the image is converted into a grayscale image, the grayscale image is decomposed into eight planes, and one of the plane image classification marks is extracted. A grayscale image is an 8-bit 256-color image. By extracting each bit of the image separately, one image can be decomposed into 8 images.

[0047] In this embodiment, during preprocessing, after decomposing the grayscale image into eight bit planes, a number of bit plane images are discarded according to a preset standard, and one of the bit plane images is randomly extracted from the remaining bit plane images for classification and marking. The preset standard is to discard the bit plane image that does not contain valid information. For example, the 8-bit plane images formed by the decomposition are respectively recorded as bit planes Figure 1 , bit plane 2, bit plane 3, bit plane 4, ... bit plane 8. According to experience, bit plane Figure 1 The bit plane image 1 and the bit plane image 2 usually contain sufficient valid information for training, so the preset standard is to discard the bit plane images 3 to 8 and randomly extract the bit plane image 1 or the bit plane image 2 for classification and marking.

[0048] A4. Input the training picture set into the neural network model for training; the neural network model is a convolutional neural network model. In this embodiment, the convolutional neural network model uses the convolutional neural network model component in OpenCV.

[0049] Embodiment 2

[0050] The difference between this embodiment and the first embodiment is that in S1 of this embodiment, a face recognition module is also used to determine whether there are people in the monitoring area; when there are people in the monitoring area, face recognition is performed to determine whether the people are registered. If they are not registered, the corresponding video data is marked as high-priority video data; if they are registered, the corresponding video data is marked as low-priority video data. During frame processing, high-priority video data corresponding to the camera in the preset area is preferentially framed.

[0051] When there are no personnel in the monitoring area, the possibility of abnormal opening of the box door is small. At preset intervals, pictures are extracted and input into the neural network model for judgment, rather than judging the pictures in real time, which can reduce the pressure of neural network model processing. By collecting the facial data of the staff in advance, the staff can be accurately identified through face recognition. When the staff appears in the monitoring area, that is, near the box, the possibility of maintenance or inspection is high, and the corresponding video data is marked as low-priority video data. When the frame processing module is busy, it does not need to be processed immediately.

[0052] Embodiment 3

[0053] The difference between this embodiment and the first embodiment is that, in this embodiment, S1 further includes:

[0054] S101, slicing step: Slice the video data to generate a description file and several media segments; the description file is used to record the shooting date, total duration, and number and duration of each media segment of the video data. In this embodiment, the video data is sliced ​​in milliseconds. In this embodiment, the description file uses an m3u8 file and the media segment uses a ts file. For example, the total duration of the video data is 10 seconds, and the slices are 10 ts files. The duration of a single ts file is 1 second, and the ts files are numbered from 001 to 010. The shooting date is, for example, 2021-7-15-12:01:00:001.

[0055] S102, identification step: binarization processing is performed on the media segment, and it is determined whether the binarized media segment contains a preset identification object. If it contains the preset identification object, it jumps to the correction step; if it does not contain the preset identification object, it jumps to the storage step. In this embodiment, binarization processing is performed on each frame of the media segment. The preset identification objects include people, animals, etc.

[0056] S103, correction step: perform noise reduction on the media segment; in this embodiment, the media segment is firstly grayscaled and then subjected to Gaussian filtering for noise reduction.

[0057] S104, storage step: storing the description file and the media segment after the noise reduction processing.

[0058] This embodiment can store the video data after processing, which is convenient for subsequent calls. Since the video data is composed of frames of pictures, it is impossible to completely separate the continuous frames when slicing, that is, between two continuous media segments, several frames may be lost, resulting in loss of picture quality. By performing noise reduction processing on the sliced ​​media segments, the quality of the picture can be effectively improved. However, if all media segments are subjected to noise reduction processing, a large amount of computing resources will be consumed. Moreover, when no person or animal passes through the monitoring range of the camera, the picture is static, and the difference between each frame is small. Even if several frames are lost in the middle, it will not cause too much impact on the picture quality. In this embodiment, the preset identification object can be a person or an animal according to the monitored object. For example, the preset identification object is an animal. When the media segment contains an animal, the media segment is subjected to noise reduction processing. Since animals appear in the picture, the difference between each frame becomes larger. If several frames are lost in the middle, the impact on the picture quality becomes larger. At this time, noise reduction processing is performed to improve the clarity of the picture to offset the impact of the possible loss of several frames and reduce the loss of picture quality. Moreover, if animals appear in the picture, the media clip is more likely to be called up and viewed by staff later, and the viewing experience can be improved after noise reduction processing.

[0059] In the S102 recognition step of this embodiment, if a preset identifier is included, the type of the preset identifier is also determined; based on the type of the preset identifier, a first preset quantity value is determined, and then the recognition step is omitted for the first preset number of media segments after the media segment, and the correction step is directly performed.

[0060] Since the monitoring range of the camera is certain, under normal circumstances, the time that people and animals pass under the camera is also within a certain range (animals usually move faster, so the time they spend under the camera is less than the time it takes for people to pass). When an animal or a person appears within the monitoring range of the camera, it is reflected in the media clip, that is, it contains a preset identification object. It takes a certain amount of time for an animal or a person to leave the monitoring range, and there is a high possibility that the media clip during this period contains an animal or a person. Therefore, omitting the identification step and directly performing the correction step can further simplify the identification process and save computing resources. In this embodiment, the first preset number is determined based on the estimated average speed of the preset identification object, the duration of the media clip, and the monitoring range of the camera. The estimated average speed can be determined based on the installation location of the camera.

[0061] Furthermore, in this embodiment, the actual average speed of the preset identification object is calculated based on the moving distance of the preset identification object in the media segment, and it is determined whether the absolute value of the difference between the actual average speed and the estimated average speed and the ratio of the estimated average speed are greater than a threshold, that is, |V1-V2| / V2, where V1 is the actual average speed and V2 is the estimated average speed.

[0062] If it is greater than the threshold, the first preset number is determined based on the estimated average speed of the preset identification object, the duration of the media segment, and the monitoring range of the camera. If it is less than or equal to the threshold, the first preset number is determined based on the actual average speed of the preset identification object, the duration of the media segment, and the monitoring range of the camera. In this embodiment, the threshold is 20%.

[0063] The above are only embodiments of the present invention. The invention is not limited to the fields involved in this implementation case. The common sense such as the known specific structures and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to explain the content of the claims.

Claims

1. The abnormal recognition result processing method applied to smart city box detection is characterized by: The steps include: S1, obtain video data in real time to determine whether there are people in the monitoring area; When there are people in the monitoring area, face recognition is performed to determine whether the person has been registered. If not, the corresponding video data is marked as high-priority video data; if it has been registered, the corresponding video data is marked as low-priority video data; Performing frame processing on the video data to generate a plurality of images, wherein the frame processing on the video data includes preferentially performing frame processing on high-priority video data corresponding to cameras in a preset area; Slice the video data to generate a description file and several media segments. The description file is used to record the shooting date, total duration of the video data, and the number and duration of each media segment; Binarize the media segment, determine whether the binarized media segment contains a preset identification object, and if so, grayscale the media segment and perform Gaussian filtering to reduce noise, wherein the preset identification object includes a person and an animal; Store the description file and the media clip after noise reduction processing; S2, input the picture into the neural network model for judgment, and obtain the judgment result; S3. When the result of the judgment is that the door is opened, judging whether it is an abnormal door opening based on the operation log; S4. If the door is opened abnormally, a preset mark will be placed on the picture.

2. The abnormal identification result processing method for smart city box detection according to claim 1 is characterized in that: In S2, the position information of the door is also obtained from the neural network model; In S4, if the door is opened abnormally, the door is marked on the picture based on the position information of the door.

3. The abnormal identification result processing method for smart city box detection according to claim 2 is characterized in that: It also includes S5, reintegrating the labeled pictures and the unlabeled pictures into a video stream.

4. The abnormal identification result processing method for smart city box detection according to claim 3 is characterized in that: The S2 is also used to preprocess the image and input the preprocessed image into the neural network model.

5. The abnormal identification result processing method for smart city box detection according to claim 4 is characterized in that: In S3, the door opening time is also obtained to determine whether the door opening plan for the corresponding time is recorded in the operation log. If not, it is determined to be an abnormal door opening.

6. The abnormal identification result processing method for smart city box detection according to claim 5 is characterized in that: In S4, annotation parameters are also obtained, and a preset mark is marked on the picture based on the annotation parameters, wherein the annotation parameters include the mark size, the mark position and the mark transparency.

7. The method for processing abnormal identification results for smart city box detection according to claim 6 is characterized in that: In the above S4, the box is also marked on the picture based on the position information of the box.

8. The abnormal identification result processing method for smart city box detection according to claim 7 is characterized in that: In S4, the door and the body are marked by selecting the door and the body by a rectangular frame.

9. The abnormal identification result processing method for smart city box detection according to claim 1 is characterized in that: In S4, a preset mark is marked on the picture through FFmpeg.