Dangerous area and important place monitoring platform for smart cities

Abstract

The present invention relates to the field of video surveillance technology, and specifically discloses a monitoring platform for dangerous areas and important places applied to smart cities, including a monitoring terminal for collecting video data of a preset area, and also including a server and a user terminal; the server is used to obtain the video data collected by the monitoring terminal in real time and perform frame processing to generate a number of pictures; the pictures are finally input into a neural network model for judgment to obtain a judgment result; when the judgment result is a door opening, it is judged based on an operation log whether it is an abnormal door opening, and if it is an abnormal door opening, a preset mark is marked on the picture and re-integrated into a video stream; the server is also used to push the video stream to a preset user terminal. The technical solution of the present invention can facilitate management personnel to know the on-site situation in a timely manner.

Description

Technical Field

[0001] The present invention relates to the technical field of video surveillance, and particularly to a monitoring platform for dangerous areas and important places applied to smart cities. Background Art

[0002] In the construction of smart cities, ordinary personnel are not allowed to enter some dangerous areas and important places. For example, box-type substations with high voltage electricity, microcomputer rooms where important equipment is deployed, etc. In order to avoid personal safety accidents or loss of public equipment caused by abnormal opening of the box body, it is necessary to monitor and alarm the abnormal opening state of the box body in real time.

[0003] The commonly used existing schemes for monitoring abnormal opening of box bodies at home and abroad are to use door magnetic sensors. This monitoring scheme can only monitor the opening and closing state of the box body door and cannot perform real-time analysis of the opening scenario, which has great limitations. For example, the box body may not have the conditions for installing door magnets, or the door magnetic data cannot be uploaded to the remote control center; it cannot perform intelligent identification of the scenario and cannot identify normal opening and abnormal opening; the door magnetic element is easily damaged, resulting in the inability to normally monitor the opening of the box body, etc.

[0004] In other schemes, video device linkage will be carried out, and video surveillance devices will be installed at the same time to continuously monitor the box body for 24 hours. When the door magnetic sensor detects abnormal opening of the box body, an alarm message will be pushed, and then the management personnel will turn on the corresponding video surveillance device to view the real-time or historical video data, but this monitoring method requires manual processing and has low work efficiency.

[0005] Therefore, a monitoring platform for dangerous areas and important places applied to smart cities that can facilitate management personnel to timely know the on-site situation is needed. Summary of the Invention

[0006] The present invention provides a monitoring platform for dangerous areas and important places applied to smart cities, which can facilitate management personnel to timely know the on-site situation.

[0007] In order to solve the above technical problems, the present application provides the following technical solutions:

[0008] A monitoring platform for dangerous areas and important places applied to smart cities includes a monitoring terminal for collecting video data of a preset area, and also includes a server and a user terminal;

[0009] The server is used to obtain the video data collected by the monitoring terminal in real time and perform frame splitting processing to generate a number of pictures; it is also used to input the pictures into a neural network model for judgment to obtain a judgment result; when the judgment result is that the door is open, it is judged whether it is an abnormal opening based on the operation log. If it is an abnormal opening, a preset identifier is marked on the picture, and it is re-integrated into a video stream;

[0010] The server is also used to push the video stream to a preset user terminal.

[0011] The principle and beneficial effects of the basic solution are as follows:

[0012] In this solution, through the monitoring terminal, video data of preset areas such as box-type substations with high voltage electricity and microcomputer rooms where important equipment is deployed can be collected in real time. The server performs frame-by-frame processing on the video data, converts the video data into one frame of pictures, and then inputs the pictures into the neural network model for judgment, and the judgment result of whether the box door is opened can be obtained. When the judgment result is that the door is opened, it can be accurately judged whether it is an abnormal door opening through the operation log. Finally, a preset identifier is marked on the picture of the abnormal door opening. The re-integrated video stream is sent to the preset user terminal. The preset user terminal can be the user terminal used by the management personnel. It is convenient for the management personnel to know the on-site situation in a timely manner. Through the preset identifier, when the management personnel conduct a review, they can also quickly determine the corresponding position when the abnormal door opening occurs.

[0013] Further, the server includes an identification module and a push module. The identification module is used to judge whether there are people in the preset area according to the video data. When there are people in the preset area, face recognition is performed to judge whether the people belong to the staff. If they belong to the staff, the push module is used to obtain the judgment result of whether it is an abnormal door opening. If it is an abnormal door opening, the re-integrated video stream is pushed to the user terminal of the corresponding staff.

[0014] It can remind the corresponding staff that his current door opening behavior is not recorded in the operation log.

[0015] Further, the user terminal is also used to receive a verification request and perform identity verification. After the identity verification is successful, an identification pattern is generated;

[0016] The server also includes a verification module. The verification module is used to extract the identification pattern from the video data. If the identification pattern is successfully extracted, the picture containing the identification pattern is saved. The push module is also used to push the picture containing the identification pattern and the video stream containing the corresponding staff to the preset user terminal.

[0017] After seeing the video stream, the staff can input a verification request to the user terminal and then conduct identity verification. After passing the identity verification, the user terminal confirms that it is the operation of the staff himself / herself and generates an identification pattern. The staff can show the identification pattern to the monitoring terminal. The monitoring terminal can capture the scene where the staff shows the identification pattern. For normal inspection or maintenance operations on the box body, it should be registered in the operation log first. However, in actual operations, there are many emergencies, and it may occur that the registration is not carried out in advance. At this time, if the staff is conducting normal inspection or maintenance, the method of identity verification plus showing the identification pattern can be used for double verification, leaving a record of the presence of the staff for filing and future reference. Compared with only conducting identity verification, it can strengthen the impression of the staff and at the same time make the staff realize that the current behavior is under monitoring.

[0018] Furthermore, the user terminal is also used to send pattern generation information to the server after generating the identification pattern;

[0019] The verification module is also used to extract the identification pattern from the video data after receiving the pattern generation information.

[0020] After receiving the pattern generation information, the verification module extracts the identification pattern. Compared with real-time extraction, the data processing volume is smaller, which can reduce the system energy consumption.

[0021] Furthermore, if the recognition module determines that it does not belong to the staff, the push module is used to obtain the judgment result of whether it is an abnormal door opening. If it is an abnormal door opening, the video stream containing non-staff is pushed to a preset user terminal; the push module is also used to send a timed viewing message to the preset user terminal;

[0022] The preset user terminal is used to determine whether the video stream containing non-staff is viewed within the first preset time after receiving the timed viewing message. If it is not viewed, a reminder message is generated.

[0023] When a non-staff appears near the box body, the risk of abnormal door opening is high and requires timely handling by the management personnel. By sending a timed viewing message to the preset user terminal, the management personnel can be urged to view it in time.

[0024] Furthermore, if the verification module does not receive the pattern generation information within the second preset time, the push module is also used to push the video stream containing the corresponding staff to the preset user terminal and send a timed viewing message to the preset user terminal.

[0025] If the staff does not register in the operation log in advance and does not conduct verification, the possibility of risk is high. Adopting the same handling measures as non-staff can urge the management personnel to handle it in time.

[0026] Furthermore, the identification pattern includes several digits or letters.

[0027] Adopt the method of numbers or letters, with clear features and easy to identify. At the same time, existing license plate recognition and other solutions can be utilized to ensure the success rate of recognition.

[0028] Furthermore, the first preset time is 30 - 120 seconds. Description of the Drawings

[0029] Figure 1 It is a logic block diagram of the monitoring platform for dangerous areas and important places in the smart city applied in the first embodiment. Detailed Implementation Modes

[0030] The following is a further detailed description through specific implementation modes:

[0031] First Embodiment

[0032] As Figure 1 shown, the monitoring platform for dangerous areas and important places in the smart city of this embodiment includes a server, several monitoring terminals and several user terminals.

[0033] The monitoring terminals are installed in preset areas for collecting video data of the preset areas. The preset areas are, for example, box-type substations with high voltage electricity, dangerous areas or important places such as microcomputer rooms where important equipment is deployed. In this embodiment, the monitoring terminals adopt cameras.

[0034] The server includes a docking module, a frame splitting module, an analysis module, an identification module, an integration module, an identification module, a pushing module, and a verification module.

[0035] The docking module is used to obtain the video data collected by the cameras in real time;

[0036] The identification module is used to judge whether there are people in the preset area according to the video data collected by the cameras in the preset area;

[0037] The identification module is also used to perform face recognition when there are people in the preset area, judge whether the people belong to the staff. If they do not belong to the staff, mark the corresponding video data as high-priority video data. If they belong to the staff, mark the corresponding video data as low-priority video data.

[0038] The frame splitting module is used to perform frame splitting on the video data to generate several pictures. The frame splitting module is also used to sequentially number the pictures. In this embodiment, sequential numbering is adopted using Arabic numerals. Among them, when the frame splitting module performs frame splitting, it preferentially performs frame splitting on the high-priority video data.

[0039] The analysis module is used to extract pictures at preset intervals when there are no people in the monitoring area, and input the pictures into the neural network model for judgment. In this embodiment, 1 picture is extracted.

[0040] The analysis module is also used to determine the preset number of pictures input into the neural network model based on the high-priority video data and the low-priority video data when there are people in the monitoring area, and preferentially input the pictures corresponding to the high-priority video data into the neural network model for judgment. Among them, the preset number of pictures corresponding to the high-priority video data is greater than the preset number of pictures corresponding to the low-priority video data. In this embodiment, when the analysis module inputs pictures into the neural network model, one picture is extracted at intervals of a preset number of pictures and input into the neural network model until the number of input pictures is equal to the preset number of pictures or the judgment result of the neural network model is to open the door.

[0041] For example, the preset number of pictures corresponding to the high-priority video data is 20, and one picture is extracted at intervals of 5 pictures and input into the neural network model.

[0042] The analysis module is also used to obtain the judgment result from the neural network model; when the judgment result is to open the door, it is judged whether it is an abnormal door opening based on the operation log; in this embodiment, before the staff conducts inspections, repairs and other operations, there will be corresponding inspection plans or repair plans, and the content related to entering the box can be extracted and added to the operation log as the door opening plan as the basis for judgment. The time of opening the door can be obtained through the video data, and it is judged whether there is a door opening plan corresponding to the time in the operation log. If not, it is judged as an abnormal door opening.

[0043] The identification module is used to mark a preset identification on the picture when the judgment result is an abnormal door opening. In this embodiment, annotation parameters can also be set, and a preset identification is marked on the picture based on the annotation parameters, where the annotation parameters include the size of the identification, the position of the identification, and the transparency of the identification. The size of the identification is the length * width of the identification. The position of the identification is the distance from the left edge of the picture and the distance from the upper edge of the picture. The transparency of the identification is expressed as a percentage. For example, a transparency of 100% means the identification is completely invisible. The identification is made of a picture formed by rasterizing the text "abnormal door opening", for example. In this embodiment, a preset identification is marked on the picture through FFmpeg.

[0044] The integration module is used to re-integrate the marked and unmarked pictures into a video stream.

[0045] The push module is used to push the video stream to a preset user terminal. In this embodiment, the preset user terminal is the user terminal used by the management personnel. The user terminal can be a mobile phone or a tablet computer, and a mobile phone is used in this embodiment.

[0046] Specifically, if the result of face recognition belongs to a staff member and the judgment result of the neural network model is abnormal door opening, the pushing module is used to push the re-integrated video stream to the user terminal of the corresponding staff member. In this embodiment, the user terminal of the corresponding staff member is the user terminal currently registered by the staff member.

[0047] The user terminal of the corresponding staff member is further used to receive a verification request and perform identity verification. After successful identity verification, an identification pattern is generated. The identity verification can be one of face recognition, fingerprint recognition, or password input. The identification pattern contains several digits or letters. In this embodiment, it includes two digits, and the digits fill the entire screen display.

[0048] The user terminal is also used to send pattern generation information to the server after generating the identification pattern;

[0049] The verification module is further used to extract the identification pattern from the video data after receiving the pattern generation information. If the identification pattern is successfully extracted, a picture containing the identification pattern is saved. The pushing module is also used to push the picture containing the identification pattern and the video stream containing the corresponding staff member to a preset user terminal.

[0050] When the verification module does not receive the pattern generation information within the second preset time, the pushing module is also used to push the video stream containing the corresponding staff member to a preset user terminal and send a timed viewing message to the preset user terminal.

[0051] If the result of face recognition does not belong to a staff member and the judgment result of the neural network model is abnormal door opening, the pushing module pushes the video stream containing the non-staff member to a preset user terminal and sends a timed viewing message to the preset user terminal.

[0052] The preset user terminal is used to determine whether the video stream containing the non-staff member is viewed within the first preset time after receiving the timed viewing message. If it is not viewed, a reminder message is generated. The first preset time is 30 - 120 seconds. In this embodiment, it is 30 seconds.

[0053] In order to make a judgment using the neural network model, this embodiment also provides a training method for the neural network model, including the following steps:

[0054] S1. Obtain video data and perform frame splitting to generate a number of pictures. In this embodiment, the frame splitting uses the splitting component in OpenCV.

[0055] S2. Classify and label the pictures to construct a training picture set. The labels include door opening and non-door opening.

[0056] S3. Input the training image set into the neural network model for training; the neural network model is a convolutional neural network model. In this embodiment, the convolutional neural network model used is the convolutional neural network model component in OpenCV.

[0057] Embodiment 2

[0058] The difference between this embodiment and Embodiment 1 is that the server in this embodiment further includes a slicing module, a correction module, and a storage module.

[0059] The slicing module is used to slice the video data to generate a description file and several media segments; the description file is used to record the shooting date, total duration of the video data, and the numbers and durations of each media segment. In this embodiment, the video data is sliced in milliseconds.

[0060] In this embodiment, the description file uses an m3u8 file, and the media segments use ts files. For example, the total duration of the video data is 10 seconds, sliced into 10 ts files, the duration of a single ts file is 1 second, and the numbers of the ts files range from 001 to 010. The shooting date is, for example, 2021-7-15-12:01:00:001.

[0061] The recognition module is further used to perform binarization processing on the media segments, and determine whether the binarized media segments contain a preset recognition object. If they contain a preset recognition object, they are processed by the correction module; if they do not contain a preset recognition object, they are processed by the storage module. In this embodiment, binarization processing is performed on each frame of the media segment. The preset recognition objects include people, animals, etc.

[0062] The correction module is used to perform noise reduction processing on the media segments; in this embodiment, the media segments are first grayscale processed and then Gaussian filtered for noise reduction.

[0063] The storage module is used to store the description file and the media segments after noise reduction processing.

[0064] This embodiment can process and store video data for convenient subsequent calling. Since video data is composed of frames, during slicing, it is impossible to exactly separate at the boundaries between consecutive frames. That is, between two consecutive media segments, there may be a loss of several frames, resulting in a loss of picture quality. By performing noise reduction processing on the sliced media segments, the picture quality can be effectively improved. However, if noise reduction processing is performed on all media segments, it will consume a large amount of computing resources. Moreover, when there are no people or animals passing within the monitoring range of the camera, the picture is static, and the difference between frames is small. Even if several frames are lost in the middle, it will not have too much impact on the picture quality. In this embodiment, the preset recognition object can be a person or an animal, etc., according to the monitored object. For example, when the preset recognition object is an animal, the media segment is subjected to noise reduction processing when the media segment contains an animal. Since there is an animal in the picture, the difference between frames becomes larger. If several frames are lost in the middle, the impact on the picture quality becomes greater. At this time, noise reduction processing is performed to improve the picture clarity to offset the possible impact of the loss of several frames and reduce the loss of picture quality. Moreover, when there is an animal in the picture, the possibility that the media segment will be subsequently called and viewed by the staff is higher. After noise reduction processing, the viewing experience can also be improved.

[0065] During the processing of the recognition module, if the preset recognition object is included, the type of the preset recognition object is also determined; based on the type of the preset recognition object, a first preset quantity value is determined, and then the processing of the recognition module for the first preset number of media segments after this media segment is omitted, and the correction module directly processes them.

[0066] Since the monitoring range of the camera is fixed, under normal circumstances, the time for people and animals in the pipe gallery to pass under the camera is also within a certain range (the time for animals is less than that for people because of their faster speed). When an animal or a person appears within the monitoring range of the camera, it is reflected in the media segment, that is, the preset recognition object is included. It takes a certain amount of time for an animal or a person to leave the monitoring range. The media segments during this period are highly likely to contain the animal or the person. Therefore, omitting the processing of the recognition module and directly processing by the correction module can further simplify the recognition process and save computing resources. In this embodiment, the first preset quantity is comprehensively determined according to the estimated average speed of the preset recognition object, the duration of the media segment, and the monitoring range of the camera. The estimated average speed can be determined according to the installation position of the camera.

[0067] Furthermore, in this embodiment, the actual average speed of the preset recognition object is also calculated based on the moving distance of the preset recognition object in the media segment, and the absolute value of the difference between the actual average speed and the estimated average speed is judged to see if the ratio to the estimated average speed is greater than the threshold. That is: |V1 - V2| / V2, where V1 is the actual average speed and V2 is the estimated average speed.

[0068] If it is greater than the threshold value, the first preset quantity is determined comprehensively according to the estimated average speed of the preset recognition object, the duration of the media segment, and the monitoring range of the camera. If it is less than or equal to the threshold value, the first preset quantity is determined comprehensively according to the actual average speed of the preset recognition object, the duration of the media segment, and the monitoring range of the camera. In this embodiment, the threshold value is 20%.

[0069] The above are only embodiments of the present invention. The invention is not limited to the fields involved in this embodiment case. Common knowledge such as the specific structures and characteristics known in the art are not described in detail here. Those of ordinary skill in the art know all the common general technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can know all the prior arts in this field, and have the ability to apply the conventional experimental means before this date. Those of ordinary skill in the art can, under the inspiration given in this application, combine their own abilities to improve and implement this solution. Some typical well-known structures or well-known methods should not become obstacles for those of ordinary skill in the art to implement this application. It should be noted that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can also be made, and these should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent. The protection scope required by this application should be based on the content of its claims, and the specific implementation manners described in the specification can be used to interpret the content of the claims.

Claims

1. A monitoring platform for dangerous areas and important places in smart cities, including a monitoring terminal for collecting video data of a preset area, characterized in that: Also includes servers and user terminals; The server is used to obtain the video data collected by the monitoring terminal in real time and perform frame processing to generate a number of pictures; it is also used to input the pictures into the neural network model for judgment to obtain the judgment result; when the judgment result is that the door is opened, it is judged based on the operation log whether it is an abnormal door opening. If it is an abnormal door opening, a preset mark is marked on the picture and it is reintegrated into a video stream; The server includes a slicing module, a correction module, an identification module, an analysis module, a storage module and a push module, wherein: The slicing module is used to slice the video data and generate a description file and a plurality of media segments. The description file is used to record the shooting date, total duration of the video data, and the number and duration of each media segment; A correction module for noise reduction of media clips; The recognition module is used to determine whether there are people in the preset area according to the video data. If there are people in the preset area, face recognition is performed to determine whether the people are staff members. If they are not staff members, the corresponding video data is marked as high-priority video data. If they are staff members, the corresponding video data is marked as low-priority video data; The recognition module is further used to perform binarization processing on the media segment, determine whether the binarized media segment contains a preset identification object, and if so, determine the type of the preset identification object, determine a first preset number value based on the type of the preset identification object, and then omit the recognition module processing for the first preset number of media segments after the media segment, and process them by the correction module; if the preset identification object is not contained, the storage module processes it, and the preset identification object includes a person and an animal; An analysis module is used to extract pictures at preset intervals when there are no people in the monitoring area, and input the pictures into the neural network model for judgment; when there are people in the monitoring area, determine the preset number of pictures input into the neural network model according to the high-priority video data and the low-priority video data, and preferentially input the pictures corresponding to the high-priority video data into the neural network model for judgment, wherein the preset number of pictures corresponding to the high-priority video data is greater than the preset number of pictures corresponding to the low-priority video data; A storage module, used for storing description files and media segments after noise reduction processing; The push module is used to obtain the judgment result of whether it is an abnormal door opening. If it is an abnormal door opening, the reintegrated video stream will be pushed to the user terminal of the corresponding staff.

2. The dangerous area and important place monitoring platform for smart city according to claim 1 is characterized by: The user terminal is also used to receive a verification request and perform identity authentication, and after the identity authentication is successful, generate an identification pattern; The server also includes a verification module, which is used to extract the identification pattern from the video data. If the identification pattern is successfully extracted, the image containing the identification pattern is saved. The push module is also used to push the image containing the identification pattern and the video stream containing the corresponding staff to a preset user terminal.

3. The dangerous area and important place monitoring platform for smart city according to claim 2 is characterized by: The user terminal is also used to send pattern generation information to the server after generating the recognition pattern; The verification module is also used to extract the identification pattern from the video data after receiving the pattern generation information.

4. The dangerous area and important place monitoring platform for smart city according to claim 3 is characterized by: If the identification module determines that the person is not a staff member, the push module is used to obtain the judgment result of whether the door is opened abnormally. If the door is opened abnormally, the video stream containing the non-staff member is pushed to the preset user terminal; the push module is also used to send the scheduled viewing information to the preset user terminal; The preset user terminal is used to determine whether the video stream containing non-staff members has been viewed within the first preset time after receiving the scheduled viewing information, and if not, generate a reminder message.

5. The dangerous area and important place monitoring platform for smart city according to claim 4 is characterized by: The verification module does not receive the pattern generation information within the second preset time, and the push module is further used to push the video stream containing the corresponding staff member to the preset user terminal and send the scheduled viewing information to the preset user terminal.

6. The dangerous area and important place monitoring platform for smart city according to claim 5 is characterized by: The identification pattern includes a number of digits or letters.

7. The dangerous area and important place monitoring platform for smart city according to claim 6 is characterized by: The first preset time is 30-120 seconds.

Images