Intelligent panoramic security and protection monitoring system based on deep learning

Abstract

The invention discloses a deep learning-based intelligent panoramic security and protection monitoring system, which comprises a visual comprehensive management platform, a camera and an auxiliary sensing module, and is characterized in that the camera and the auxiliary sensing module are installed in a maintenance library, and the visual comprehensive management platform is electrically connected with the camera and the auxiliary sensing module; video picture information shot by the camera and information detected by the auxiliary sensing module are transmitted to the visual comprehensive management platform for processing. The visual integrated management platform comprises a full-view-field video monitoring subsystem, a virtual access control subsystem, an operation behavior analysis subsystem, an equipment state monitoring subsystem, an intelligent fire control subsystem, an alarm joint defense subsystem and a big data management subsystem. And the full-view-field video monitoring subsystem is used for acquiring full-domain dead-corner-free monitoring video data in the maintenance library, performing panoramic multi-video image fusion, and integrating one-point instant view and panoramic splicing playback and downloading.

Description

Technical Field

[0001] The present invention belongs to the fields of computer video analysis technology, intelligent security monitoring technology and intelligent control technology, and specifically relates to an intelligent panoramic security monitoring system based on deep learning. Background Art

[0002] In recent years, with the development of advanced information technology, the reform and development of railway transportation equipment maintenance in my country has been strongly promoted, and the maintenance management process has been greatly optimized. However, some problems are inevitably encountered in the actual application process.

[0003] Railway yards, especially vehicle depots, face a large volume of production, many aisles, and busy shunting in and out of the garage, and the personal safety protection of maintenance workers is becoming increasingly prominent. At present, workshops and depot dispatchers are unable to monitor the status of the maintenance depot and the aisles in the shunting yard in real time. Maintenance and shunting operations in the garage are all done by manual intercom and mutual control reminders. When a vehicle is about to pass or enter the depot, there is no intuitive and effective means to remind the station operators and maintenance workers in the depot to avoid the vehicle in time. Due to the nature of their work, the workers in the depot are prone to mental paralysis. Once safety awareness is not in place, personal safety accidents during vehicle maintenance and motor vehicle accidents in the aisles will occur, which will bring great hidden dangers to transportation safety production.

[0004] Therefore, developing an integrated safety protection system for railway stations and maintenance depots to effectively improve the safety factor of transportation production has become an urgent task for current production safety, so as to find out the key causes of human errors and reasonably formulate risk control measures, which has a strong practical significance for standardizing the operating behavior of railway vehicle maintenance personnel, effectively reducing the occurrence rate of human errors, eliminating human risks to vehicle safety, and ensuring railway transportation safety. Summary of the invention

[0005] The purpose of the present invention is to solve the above-mentioned problems and to provide a kind of intelligent panoramic security monitoring system based on deep learning, which evaluates and analyzes the possible safety risks in the operation and operation of equipment in the factory, especially for personal safety in the factory, according to the production processes of different maintenance lines of railway equipment companies.

[0006] In order to solve the above technical problems, the technical solution of the present invention is: an intelligent panoramic security monitoring system based on deep learning, including a visual integrated management platform, a camera and an auxiliary sensor module, the camera and the auxiliary sensor module are installed in the maintenance warehouse, the visual integrated management platform is electrically connected to the camera and the auxiliary sensor module, and the video image information taken by the camera and the information detected by the auxiliary sensor module are transmitted to the visual integrated management platform for processing; the visual integrated management platform includes a full-field video monitoring subsystem, a virtual access control subsystem, an operation behavior analysis subsystem, an equipment status monitoring subsystem, an intelligent fire control subsystem, an alarm joint defense subsystem and a big data management subsystem, the full-field video monitoring subsystem is used to obtain the full-domain monitoring video data without blind spots in the maintenance warehouse, perform panoramic multi-video image fusion, and integrate one-click viewing, panoramic stitching playback and download; the virtual access control subsystem is used to use the video data taken by the cameras installed at each entrance and exit of the maintenance warehouse to perform face recognition on all personnel entering and leaving the maintenance warehouse, and identify their faces in the maintenance warehouse The walking trajectory in the warehouse can be tracked in real time; the operation behavior analysis subsystem is used to track the trajectory and analyze the behavior of personnel in the workstations that require on-site operation by operators, key monitoring areas and public areas of all maintenance depots; the equipment status monitoring subsystem is used to evaluate the operating status of each equipment in the maintenance depot, and once an equipment abnormality is found, the alarm information is immediately uploaded to the visual integrated management platform; the intelligent fire control subsystem is used to build a fire prevention and control system for the factory area and maintenance depot, monitor the fire safety status at all times and visually, and evaluate the fire safety risks in real time and intelligently; the alarm joint defense subsystem is used to send the alarm pictures and videos monitored by each camera at the maintenance workshop site and the operation assembly line workstation to the back-end big data management subsystem, and push them to the visual integrated management platform; the big data management subsystem integrates the full-field video monitoring subsystem, virtual access control subsystem, operation behavior analysis subsystem, equipment status monitoring subsystem, intelligent fire control subsystem and alarm joint defense subsystem into a visual integrated management platform.

[0007] Preferably, the full-field-of-view video surveillance subsystem includes the following functional modules: a full-domain no-blind-angle monitoring module, a panoramic multi-video fusion module, a one-click viewing module, and a panoramic stitching playback and download module; the full-domain no-blind-angle monitoring module is used to connect all video surveillance equipment in the assembly line operation areas, office areas, public areas, equipment storage areas, and warehouses in the entire maintenance warehouse to a unified monitoring platform, so that there are no blind spots in the monitoring of the maintenance warehouse; it also completely covers the front-end video acquisition, encoding, transmission to the back-end storage, control, decoding output, and even to the integrated management platform, supporting the unified coordination and emergency command of the monitoring center; the panoramic multi-video fusion module is used to project the real-time images taken by the camera into a three-dimensional real scene In the model panorama, adjacent images can be stitched and fused to form a larger resolution image, that is, a bird's-eye view video surveillance panorama of the entire maintenance depot; the one-click view module is used to double-click the target of interest (such as personnel, vehicles, equipment) or area on the panoramic video, and the turntable will automatically jump and zoom in to clearly present the target, meeting the user's needs for detailed viewing of the area of ​​interest or target; the panoramic stitching playback and download module is used to store the panoramic fused video, and synchronously review all video playback recordings, stitching them into a complete picture for users to selectively download and store; compared with the traditional video playback method, you can have a panoramic view, check the entire screen in real time, and understand the real-time and comprehensive trends.

[0008] Preferably, the virtual access control subsystem includes the following functional modules: an entrance and exit face recognition module and an in-warehouse cross-camera tracking module; the entrance and exit face recognition module is used to perform contactless face recognition on all personnel entering and exiting the maintenance warehouse through high-definition cameras at each entrance and exit of the maintenance warehouse; the in-warehouse cross-camera tracking module is used to combine with the entrance and exit face recognition module to realize continuous cross-camera tracking of all personnel entering the maintenance warehouse, enhance the spatiotemporal continuity of the data, record their walking trajectory, and quickly lock the identity of the personnel.

[0009] Preferably, the operation behavior analysis subsystem includes the following functional modules: an alarm module for illegal crossing of boundaries when pushing a cart into the warehouse, an alarm module for not wearing a safety helmet when entering the maintenance warehouse, an alarm module for not wearing a mask at a designated workstation, an alarm module for making and receiving calls in the warehouse during working hours, and an alarm module for not wearing goggles during welding operations.

[0010] Preferably, the equipment status monitoring and control subsystem includes the following functional modules: an equipment status real-time monitoring module, an abnormal status real-time early warning module and a fault event level processing module.

[0011] Preferably, the intelligent fire control subsystem includes a real-time monitoring module, and the real-time detection module monitors the status of each system of fire monitoring, combustible gas, toxic gas, fire water, and charging piles in real time, and pushes fault information and alarms in a timely manner.

[0012] Preferably, the alarm joint defense subsystem is mainly responsible for sending the alarm pictures and videos monitored by various cameras at each maintenance workshop site and operation line workstation to the big data management subsystem, and pushing them to the visual integrated management platform.

[0013] Preferably, the big data management subsystem is mainly responsible for integrating multiple subsystems into a comprehensive visual management platform, connecting with the enterprise's internal management and control software, and publishing its data to the intelligent management platform in real time to realize a comprehensive intelligent management platform.

[0014] Preferably, the auxiliary sensing module includes a smoke sensor, a temperature and humidity sensor, a combustible gas leak detector, a harmful gas leak detector, an over-line current sensor, a residual current sensor, a power wizard and an audible and visual alarm.

[0015] The beneficial effects of the present invention are:

[0016] 1. The intelligent panoramic security monitoring system based on deep learning provided by the present invention uses advanced AI video analysis technology, combined with the enterprise's production safety management concept and truck maintenance process characteristics, to conduct behavioral analysis and risk assessment of various operations in the workshop maintenance process. The ultimate goal is to study and establish a full-field intelligent monitoring system that integrates standardized management, operation evaluation, safety control and other functions, integrates production safety intelligent early warning and emergency response, and avoids safety accidents, personal and economic losses caused by equipment failure, operational errors and network paralysis.

[0017] 2. The present invention evaluates and analyzes the possible safety risks in the operation and operation of equipment in the factory according to the production processes of different maintenance lines of railway equipment companies, and conducts risk assessment on personal safety in the factory area, thereby avoiding the existence of potential dangers in advance, handling them in time, and avoiding further losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a principle framework diagram of an intelligent panoramic security monitoring system based on deep learning in the present invention;

[0019] Figure 2 It is a flowchart of the working steps of the panoramic multi-video fusion module of the present invention;

[0020] Figure 3 It is a flowchart of the steps of the intelligent video analysis technology based on deep learning of the present invention;

[0021] Figure 4 is a flow chart of the steps of fireworks detection of the present invention;

[0022] Figure 5 It is a flow chart of the abnormal state determination steps of the device of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments:

[0024] like Figures 1 to 5 As shown, the present invention provides an intelligent panoramic security monitoring system based on deep learning, including a visual integrated management platform, a camera and an auxiliary sensor module. The camera and the auxiliary sensor module are installed in the maintenance warehouse. The visual integrated management platform is electrically connected to the camera and the auxiliary sensor module. The video image information captured by the camera and the information detected by the auxiliary sensor module are transmitted to the visual integrated management platform for processing. The visual integrated management platform includes a full-field video monitoring subsystem, a virtual access control subsystem, an operation behavior analysis subsystem, an equipment status monitoring subsystem, an intelligent fire control subsystem, an alarm joint defense subsystem and a big data management subsystem. The full-field video monitoring subsystem is used to obtain the full-domain monitoring video data without blind spots in the maintenance warehouse, perform panoramic multi-video image fusion, and integrate one-click viewing, panoramic splicing playback and download. The virtual access control subsystem is used to use the video data captured by the cameras installed at each entrance and exit of the maintenance warehouse to perform face recognition on all personnel entering and leaving the maintenance warehouse, and to track their walking trajectories in the maintenance warehouse in real time. The operation behavior analysis subsystem is used to track and analyze the behavior of personnel in the workstations, key monitoring areas and public areas of all maintenance depots that require on-site operation by operators. The equipment status monitoring subsystem is used to evaluate the operating status of each device in the maintenance depot. Once an abnormality is found in the equipment, the alarm information is immediately uploaded to the visual integrated management platform. The intelligent fire control subsystem is used to build a fire prevention and control system for the factory area and maintenance depot, monitor the fire safety status at all times and visually, and evaluate the fire safety risks in real time and intelligently. The alarm joint defense subsystem is used to send the alarm pictures and videos monitored by each camera at the maintenance workshop site and the operation assembly line to the back-end big data management subsystem, and push them to the visual integrated management platform. The big data management subsystem integrates the full-field video monitoring subsystem, virtual access control subsystem, operation behavior analysis subsystem, equipment status monitoring subsystem, intelligent fire control subsystem and alarm joint defense subsystem into a visual integrated management platform.

[0025] The full-field video surveillance subsystem includes the following functional modules: full-domain no-blind-angle monitoring module, panoramic multi-video fusion module, one-click viewing module, panoramic stitching playback and download module. The full-domain no-blind-angle monitoring module is used to connect all video surveillance equipment in the assembly line operation areas, office areas, public areas, equipment storage areas, and warehouses in the entire maintenance warehouse to a unified monitoring platform, so that there are no blind spots in the maintenance warehouse. It also fully covers the front-end video acquisition, encoding, transmission to the back-end storage, control, decoding output, and even to the integrated management platform, supporting the unified coordination and emergency command of the monitoring center. The panoramic multi-video fusion module is used to project the real-time images captured by the camera onto the panoramic view of the three-dimensional real-scene model, and can splice and fuse adjacent images to form a larger resolution image after splicing, that is, a bird's-eye video surveillance panoramic view of the entire maintenance warehouse, which can better monitor the overall large scene.

[0026] In this example, the specific process is as follows: "Configuration of monitoring area and shooting angle parameters of multiple cameras", "Preprocessing of monitoring video data", "Collection of on-site target image data", "Data preprocessing", "AI modeling", to form a real-scene model "1". After forming "1", the following steps are performed in sequence for splicing and fusion, and the steps adopted are: "Regular acquisition of all camera images", "Extracting spliced ​​video stream data", "AI preprocessing", "Adjacent camera matching point detection", "AI video splicing preprocessing", "Image alignment", "Image synthesis", "Crop output", "Panoramic video transmission, storage and display", and the results of panoramic video transmission, storage and display are then matched with the regular acquisition of all camera images. The full-field video monitoring subsystem in the present invention obtains the full-field monitoring video data of the maintenance warehouse without blind spots through the full-field monitoring module, and performs panoramic video picture fusion and display.

[0027] The one-click-to-view module is used to double-click the target of interest (such as people, vehicles, equipment) or area on the panoramic video. The turntable will automatically jump and zoom in to clearly present the target, meeting the user's needs for viewing the detailed details of the area of ​​interest or target.

[0028] The one-click viewing module divides the fused panoramic video into regions according to the working areas. By double-clicking the corresponding area, the original video image of the area will automatically pop up and be magnified to meet the needs of viewing regional details.

[0029] The panoramic stitching playback and download module is used to store the panoramic fusion video, and synchronously review all video playback recordings, stitching them into a complete picture, which can be selectively downloaded and stored by users. Compared with the traditional video playback method, you can have a panoramic view, check the entire screen in real time, and understand the real-time and comprehensive trends.

[0030] The panoramic stitching playback and download module stores the panoramic fused video image data and the original video data of each area in real time synchronously, meeting the needs of later synchronous review and selective downloading of all relevant video playback recordings.

[0031] The virtual access control subsystem includes the following functional modules: entrance and exit face recognition module and in-store cross-camera tracking module. The entrance and exit face recognition module is used to perform contactless face recognition on all personnel entering and leaving the maintenance warehouse through high-definition cameras at each entrance and exit of the maintenance warehouse. The in-store cross-camera tracking module is used to combine the entrance and exit face recognition module to achieve cross-camera continuous tracking of all personnel entering the maintenance warehouse, enhance the spatiotemporal continuity of the data, record their walking trajectory, and quickly lock the identity of the personnel.

[0032] The entrance and exit face recognition module obtains high-definition video data of each entrance and exit of the maintenance warehouse, uses advanced face recognition technology, performs non-contact face recognition on all personnel entering and leaving the maintenance warehouse, extracts and stores the face data of all personnel entering and leaving the maintenance warehouse, and performs identity recognition, etc. In this embodiment, the face recognition technology is an existing mature technology.

[0033] The cross-camera trajectory tracking module uses pedestrian target detection and pedestrian re-identification technology to search for a pedestrian target for all high-definition video data in the maintenance warehouse, form a target movement trajectory, and quickly locate the target position.

[0034] The operation behavior analysis subsystem includes the following functional modules: an alarm module for illegal crossing of boundaries when pushing carts into the warehouse, an alarm module for not wearing a safety helmet when entering the maintenance warehouse, an alarm module for not wearing a mask at a designated workstation, an alarm module for making and receiving phone calls in the warehouse during working hours, and an alarm module for not wearing goggles during welding operations.

[0035] The operation behavior analysis system in the present invention monitors the operation behavior of personnel by acquiring all monitoring video data in the maintenance warehouse, formulates the judgment criteria of personnel operation behavior norms and the judgment criteria of illegal behavior according to the safety operation specifications, and gives early warning prompts to the operators with irregular operation behaviors or illegal behaviors, and uploads the warning related information (including the description of the illegal behavior of the personnel, pictures and video data) to the alarm joint defense system for the monitoring management personnel to review and process. The operation behavior analysis subsystem decomposes the safe operation behavior action, decomposes the operation behavior to be detected according to the safety operation specification requirement instruction book, and formulates the judgment criteria of operation norms and the judgment criteria of illegal behavior. Among them, the illegal behaviors include: illegal crossing of boundaries when the cart enters the warehouse, entering the warehouse without wearing a helmet, not wearing a mask at the designated workstation, answering and calling during the operation time in the warehouse, not wearing goggles during welding operation, etc., which are processed by the corresponding illegal crossing of boundaries when the cart enters the warehouse, the alarm module for not wearing a helmet when entering the maintenance warehouse, the alarm module for not wearing a mask at the designated workstation, the alarm module for answering and calling during the working time in the warehouse, and the alarm module for not wearing goggles during welding operation.

[0036] The equipment status monitoring and control subsystem includes the following functional modules: equipment status real-time monitoring module, abnormal status real-time warning module and fault event level processing module.

[0037] The equipment status monitoring and control system in the present invention obtains the monitoring video data of the equipment at the designated workstation in the maintenance warehouse, and uses the intelligent video analysis technology based on deep learning and various auxiliary sensor modules to identify and analyze the current status of the equipment and the surrounding environment to determine whether there is an abnormality. Once an equipment abnormality is found, the alarm information is immediately uploaded to the management platform and pushed to the relevant person in charge for timely processing.

[0038] The intelligent video analysis technology based on deep learning in the present invention includes: target device detection, device operation trajectory tracking and device operation status determination. The specific steps are: "segmentation of assembly line operation area", "setting monitoring targets and types of each area", "setting behavioral requirements and determination rules for each target", "collection of on-site target image data", "data preprocessing", "AI modeling", forming data model "1", and then successively through "regular collection of all camera image data in the library", "AI preprocessing", "AI video structured analysis", "personnel target extraction and posture analysis", "personnel target behavior analysis" to form data model "2", and then successively through "personnel target behavior violation determination", "determination of violation", if violation, "alarm information storage and upload", and "on-site sound and light alarm activation", and then determine "whether the upload is successful", if unsuccessful, re-upload, if successful, "platform alarm prompt", and finally "manual review and disposal".

[0039] Each auxiliary sensing module in the present invention includes a vibration sensor, a temperature sensor, a current sensor, a rotation speed sensor, a wireless network communication module, etc. A vibration sensor, a temperature sensor, a current sensor, a rotation speed sensor, a wireless communication processing module and a wireless data receiving module.

[0040] Determination of abnormal equipment status: The real-time warning module for abnormal status analyzes the equipment monitoring video data and combines it with the implementation data of the auxiliary sensor module to determine whether the production equipment is operating abnormally.

[0041] The specific steps are as follows: "Extract the equipment information that needs to be monitored in each area", "Obtain key parameters for determining the status of each device", "Set the threshold range of specific parameters for the equipment's operating status", "Set the fault type, risk level and fault alarm emergency response plan", "Equipment fault status detection algorithm design", "AI modeling" to form data model "1", and then successively through "regular collection of key parameters of equipment operating status", "AI preprocessing", "AI real-time analysis and fault mining", and then judge "Is there any abnormality / fault risk at present?" If not, return to "regular collection of key parameters of equipment operating status", if so, "determine the fault type and risk level". Figure 5 Equipment fault event processing: The fault event level processing module obtains the fault level of the equipment abnormal fault event according to the pre-established fault level classification rules and corresponding handling methods, and sends the fault event information to the relevant departments and persons in charge. At the same time, the relevant fault equipment is handled according to the fault handling method (such as power off, speed reduction, suspension of operation, etc.).

[0042] The intelligent fire control subsystem includes a real-time monitoring module. The real-time detection module monitors the status of each system including fire monitoring, combustible gas, toxic gas, fire water, and charging piles in real time, and pushes fault information and alarms in a timely manner.

[0043] The intelligent fire control subsystem obtains all-around surveillance video data in the maintenance warehouse, uses deep learning-based intelligent video analysis technology, and combines various auxiliary sensor modules to conduct full-time, visual fire safety status monitoring and real-time, intelligent fire safety risk assessment. The deep learning-based intelligent video analysis technology includes: smoke and fire detection. The steps of fire and smoke detection are: "divide the assembly line operation area", "set the fire risk level", "set emergency response measures for the fire risk level", "collect fire image data", "preprocess data", and "AI modeling" to form data model "1", and then "regularly collect all camera image data in the library" and "regularly collect environmental data by environmental monitoring devices". After "regularly collect all camera image data in the library", "AI preprocessing", "AI video structured analysis", and "smoke and flame target detection and recognition" are carried out in sequence; after "regularly collect environmental data by environmental monitoring devices", "fire factor data analysis and mining" and "real-time determination of environmental status" are carried out in sequence. After "real-time determination of environmental status" and "smoke and flame target detection and recognition", they are merged and unified to carry out "data fusion, fire risk level determination" in sequence. When judging whether it is a "suspected fire", if the judgment result is "yes", "extract the fire area location" to form data model "2". On the basis of "2", "alarm information storage and upload" are carried out in sequence to judge "whether the upload is successful". If successful, "platform alarm prompt" and "manual review and disposal" are carried out. Figure 4 In this embodiment, data model "1" changes internal data according to the corresponding step content and function.

[0044] The alarm joint defense subsystem is mainly responsible for sending the alarm pictures and videos monitored by various cameras at each maintenance workshop and operation line workstation to the big data management subsystem, and pushing them to the visual integrated management platform.

[0045] The alarm linkage sub-defense system sends the relevant information of the alarm events occurring at each maintenance workshop and operation line workstation, as well as the alarm pictures and videos monitored by the corresponding cameras, to the back-end management subsystem and pushes them to the integrated management platform.

[0046] The big data management subsystem is mainly responsible for integrating multiple subsystems into a comprehensive visual management platform, connecting with the enterprise's internal management and control software, and publishing its data to the intelligent management platform in real time to realize a comprehensive intelligent management platform.

[0047] The big data management platform is an intelligent integrated security protection management platform built on the server side. Its main function is to integrate multiple subsystems into a visual intelligent management platform. It also supports docking with the company's internal management and control software, and publishes its data to the platform in real time to realize a comprehensive intelligent management platform.

[0048] The auxiliary sensor module includes a smoke sensor, a temperature and humidity sensor, a combustible gas leak detector, a harmful gas leak detector, an overcurrent sensor, a residual current sensor, a power wizard and an audible and visual alarm. The various sensor devices in the auxiliary sensor module of the present invention are installed in the maintenance warehouse according to actual use needs so as to be able to detect and transmit corresponding information in real time.

[0049] The present invention is a complete integrated safety protection system for smart factories, which has the characteristics of a full-field intelligent monitoring system integrating standardized management, operation evaluation, safety control and other functions. The present invention integrates production safety intelligent early warning and emergency response to avoid safety accidents, personal and economic losses caused by equipment failure, operating errors and network paralysis.

[0050] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. An intelligent panoramic security monitoring system based on deep learning, characterized by: It includes a visual integrated management platform, a camera and an auxiliary sensor module. The camera and the auxiliary sensor module are installed in the maintenance warehouse. The visual integrated management platform is electrically connected to the camera and the auxiliary sensor module. The video image information taken by the camera and the information detected by the auxiliary sensor module are transmitted to the visual integrated management platform for processing; the visual integrated management platform includes a full-field video monitoring subsystem, a virtual access control subsystem, an operation behavior analysis subsystem, an equipment status monitoring subsystem, an intelligent fire control subsystem, an alarm joint defense subsystem and a big data management subsystem. The full-field video monitoring subsystem is used to obtain the full-area monitoring video data without blind spots in the maintenance warehouse, perform panoramic multi-video image fusion, and integrate one-click viewing, panoramic stitching playback and download; the virtual access control subsystem is used to use the video data taken by the cameras installed at each entrance and exit of the maintenance warehouse to perform face recognition on all personnel entering and leaving the maintenance warehouse, and track their walking trajectory in real time in the maintenance warehouse; the operation behavior analysis subsystem The system is used to track the trajectories and conduct behavioral analysis and identification of personnel at workstations that require on-site operation by operators, key monitoring areas, and public areas in all maintenance depots; the equipment status monitoring subsystem is used to evaluate the operating status of each device in the maintenance depot, and once an equipment abnormality is found, the alarm information is immediately uploaded to the visual integrated management platform; the intelligent fire control subsystem is used to build a fire prevention and control system for the factory area and maintenance depot, monitor the fire safety status visually at all times, and evaluate the fire safety risks in real time and intelligently; the alarm joint defense subsystem is used to send the alarm images and videos monitored by each camera at the maintenance workshop site and the operation line workstation to the back-end big data management subsystem, and push them to the visual integrated management platform; the big data management subsystem integrates the full-field video monitoring subsystem, virtual access control subsystem, operation behavior analysis subsystem, equipment status monitoring subsystem, intelligent fire control subsystem, and alarm joint defense subsystem into a visual integrated management platform.

2. The intelligent panoramic security monitoring system based on deep learning according to claim 1, characterized in that: The full-view video monitoring subsystem includes the following functional modules: full-domain no-blind-angle monitoring module, panoramic multi-video fusion module, one-click viewing module, panoramic stitching playback and download module; the full-domain no-blind-angle monitoring module is used to connect all video monitoring equipment in the assembly line operation area, office area, public area, equipment storage area, and warehouse in the entire maintenance warehouse to a unified monitoring platform, so that there are no blind spots in the maintenance warehouse monitoring; It also fully covers the entire process from front-end video acquisition, encoding, and transmission to back-end storage, control, decoding and output, and even the integrated management platform, supporting the unified coordination and emergency command of the monitoring center; The panoramic multi-video fusion module is used to project the real-time images captured by the camera onto the panoramic 3D real-scene model, and can stitch and fuse adjacent images to form a larger resolution image, that is, a bird's-eye view of the entire maintenance depot. The one-click viewing module is used to double-click the target (such as personnel, vehicles, equipment) or area on the panoramic video, and the turntable automatically jumps and zooms in, presenting the target clearly, meeting the user's needs for viewing the detailed details of the area or target of interest; The panoramic stitching playback and download module is used to store the panoramic fused video, and then synchronously review all video playback recordings and stitch them into a complete picture, which can be selectively downloaded and stored by users. Compared with the traditional video playback method, users can have a panoramic view, view the entire screen in real time, and understand the real-time and comprehensive trends.

3. The intelligent panoramic security monitoring system based on deep learning according to claim 1, characterized in that: The virtual access control subsystem includes the following functional modules: entrance and exit face recognition module and in-store cross-mirror tracking module; The entrance and exit facial recognition module is used to perform contactless facial recognition on all personnel entering and exiting the maintenance warehouse through the high-definition cameras at each entrance and exit of the maintenance warehouse; the cross-camera tracking module within the warehouse is used in combination with the entrance and exit facial recognition module to achieve continuous cross-camera tracking of all personnel entering the maintenance warehouse, enhance the temporal and spatial continuity of the data, record their walking trajectory, and quickly lock the identity of the personnel.

4. The intelligent panoramic security monitoring system based on deep learning according to claim 1, characterized in that: The operation behavior analysis subsystem includes the following functional modules: an alarm module for illegal crossing of boundaries when pushing a cart into the warehouse, an alarm module for not wearing a safety helmet when entering the maintenance warehouse, an alarm module for not wearing a mask at a designated workstation, an alarm module for making and receiving calls in the warehouse during working hours, and an alarm module for not wearing goggles during welding operations.

5. The deep learning-based intelligent panoramic security monitoring system according to claim 1, characterized in that: The equipment status monitoring and control subsystem includes the following functional modules: an equipment status real-time monitoring module, an abnormal status real-time early warning module and a fault event level processing module.

6. The intelligent panoramic security monitoring system based on deep learning according to claim 1, characterized in that: The intelligent fire control subsystem includes a real-time monitoring module. The real-time detection module monitors the status of each system of fire monitoring, combustible gas, toxic gas, fire water, and charging piles in real time, and pushes fault information and alarms in a timely manner.

7. The intelligent panoramic security monitoring system based on deep learning according to claim 1, characterized in that: The alarm joint defense subsystem is mainly responsible for sending the alarm pictures and videos monitored by various cameras at each maintenance workshop and operation line workstation to the big data management subsystem, and pushing them to the visual integrated management platform.

8. The deep learning-based intelligent panoramic security monitoring system according to claim 1, characterized in that: The big data management subsystem is mainly responsible for integrating multiple subsystems into a comprehensive visual management platform, connecting with the enterprise's internal management and control software, and publishing its data to the intelligent management platform in real time to realize a comprehensive intelligent management platform.

9. The intelligent panoramic security monitoring system based on deep learning according to claim 1, characterized in that: The auxiliary sensing module includes a smoke sensor, a temperature and humidity sensor, a combustible gas leak detector, a harmful gas leak detector, an overcurrent sensor, a residual current sensor, an electricity wizard and an audible and visual alarm.

Images