Limited space operation intelligent monitoring system based on multi-sensor fusion

Abstract

The invention discloses a limited space operation intelligent monitoring system based on multi-sensor fusion, and relates to the technical field of intelligent monitoring. Compared with a previous intelligent monitoring system, the system solves the problems that the space limitation degree and the environment complexity in a limited space are relatively high, and the operation safety risk is relatively high; a traditional monitoring method carries out monitoring based on a single sensor and lacks real-time performance and accuracy. The condition in a limited space is sensed and monitored in real time by fusing data of various sensors, so that the operation safety and the normal operation of equipment are ensured; intelligent mobile equipment is also integrated, so that the position and the state of an operator in a limited space can be accurately determined in real time; behavior recognition is carried out on people, vehicles, non-motor vehicles and the like through a behavior detection method, corresponding alarm and enhanced management are carried out, and the operation safety is further improved; and remote real-time monitoring of limited space operation is realized in combination with the Internet of Things technology, and all-around safety guarantee is provided for the limited space operation.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to an intelligent monitoring system for limited space operations based on multi-sensor fusion. Background Art

[0002] Confined spaces are usually relatively closed or partially closed, such as storage tanks, basements, tunnels, etc. Their internal space is narrow and the structure is complex, which limits the movement of operators and the arrangement of equipment. For example, in underground pipeline operations, operators have limited space for movement and it is difficult to move, and there may be various obstacles inside the pipeline.

[0003] Due to poor ventilation in confined spaces, once a flammable gas leak occurs, an explosive environment is likely to form, causing a fire or explosion. For example, when performing maintenance work in a chemical raw material storage tank, if the residual flammable gas in the tank is not cleaned up, it may cause an explosion when it encounters a fire source. In a cellar that has not been opened for a long time, the oxygen content may be far below normal levels, and workers may easily suffocate due to lack of oxygen after entering. When working in underground culverts, there may be a risk of drowning if there is a rainstorm or other conditions; when working in the basements of some old buildings, there may be a risk of collapse of walls and other structures.

[0004] Traditional monitoring of confined space operations often relies on a single type of sensor. For example, only using gas sensors to detect the gas composition in a confined space can obtain information on gas concentration, but it is not possible to fully understand other conditions in the operating environment. For example, it is impossible to accurately determine the location and activity status of the operator, and when a gas leak occurs, it is impossible to determine in time whether the operator is in a dangerous area.

[0005] For monitoring of environmental parameters such as temperature and humidity, if only temperature sensors or humidity sensors are used, the impact of environmental factors on operational safety cannot be comprehensively considered.

[0006] When the gas concentration in a confined space changes suddenly, traditional monitoring equipment may take a long time to detect and issue an alarm, resulting in operators being unable to take timely evasive measures. For example, when the gas concentration in a confined space changes suddenly, traditional monitoring equipment may take a long time to detect and issue an alarm, resulting in operators being unable to take timely evasive measures. Gas sensors are easily interfered by other gases, resulting in inaccurate measurement results, thereby misjudging the safety status in the confined space. For example, in an environment with multiple volatile organic compounds, a single gas sensor may not be able to accurately distinguish the concentrations of the target toxic gas and the interfering gas.

[0007] In order to solve the above problems, the present invention proposes an intelligent monitoring system for limited space operations based on multi-sensor fusion. Summary of the invention

[0008] The purpose of the present invention is to propose an intelligent monitoring system for limited space operations based on multi-sensor fusion to solve the problems raised in the background technology:

[0009] The spatial limitations and environmental complexity in confined spaces are high, and the operational safety risks are high; traditional monitoring methods are based on a single sensor and lack real-time and accuracy.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions:

[0011] An intelligent monitoring system for limited space operations based on multi-sensor fusion, comprising:

[0012] Data acquisition module: used to collect operation data in a limited space through a number of data acquisition sensors;

[0013] Data storage module: used to store operation data through solid state hard disk;

[0014] Data transmission module: used to transmit operation data wirelessly or wirelessly based on cloud networking technology;

[0015] Data processing module: used for centralized processing and management of received data;

[0016] The data processing module comprises:

[0017] Operation training unit: used to train operators on safe operation procedures, and generate corresponding safe operation procedures for enhanced training based on the detection results of the behavior detection module;

[0018] Behavior detection unit: used to monitor the behavior of operators in real time through high-definition AI behavior monitoring dome camera based on image recognition technology, and correct unsafe behaviors;

[0019] Data monitoring unit: used to collect and analyze operation data in real time;

[0020] Abnormal alarm unit: used to automatically alarm detected abnormal operation data;

[0021] On-site management unit: used to conduct hierarchical authorization confirmation of the entry and exit behavior of operating personnel.

[0022] Preferably, the data acquisition sensor of the data acquisition module collects the operating data of the environmental camera, smart bracelet, gas detector and smart helmet through the intelligent data acquisition server; the environmental camera supports intelligent resource mode switching, and the intelligent resource mode includes face capture mode, road monitoring mode, Smart event mode and people counting mode.

[0023] Preferably, the data acquisition module also converts the collected operation data through a data AD converter.

[0024] Preferably, the behavior detection unit structures people, vehicles and non-motor vehicles based on a high-definition AI behavior monitoring ball camera, wherein people are divided into 17 key point sets according to skeleton key points, and vehicles are divided into 13 key point sets according to the center points of four wheels, four corner points of the front and rear bumpers of the vehicle, four center points of the headlights and taillights of the vehicle, and the center point of the top of the vehicle; non-motor vehicles are divided into 10 key point sets according to the center points of two wheels, the center points of two handlebars, four key connection points of the frame and the center points of two pedals;

[0025] Based on the behavior detection unit, images or videos are collected, and feature extraction is performed based on the divided key point set to form a feature matrix. ij ,y ij ,z ij ) Generate the corresponding key point stacking data G(x, y, z) in the form of Gaussian distribution, where i is the key point set and j is the key point; thereby obtaining the input feature matrix G:

[0026]

[0027] Where σ is the standard deviation of the Gaussian distribution;

[0028]

[0029] Where D(·) is the distance calculation function; it is used to calculate the distance between a point (x, y, z) and a line segment s(b ij ,d ij ) between the two ij ,d ij are the key points at both ends of the line segment;

[0030] Use 1×1 convolution to divide the input features into 6 different branches according to channel C: For the input feature matrix G, whose dimension is [B, C, H, W], the convolution operation is as follows:

[0031]

[0032] Among them, B represents the batch size, C represents the number of channels, H represents the height, and W represents the width. Through 6 different 1×1 convolution kernels The output of 6 branches is obtained, and its dimension is [B,C l ,H,W];

[0033] Apply 3×1 convolution, 3×3 convolution, dilated convolution, 3×1 maximum pooling, and no operation to each branch; for the output y of branch l l, and obtain z by performing convolution operation with the corresponding convolution kernel l , as follows:

[0034]

[0035]

[0036]

[0037]

[0038] in, and They are the outputs of 3×1 convolution, 3×3 convolution, dilated convolution, and 3×1 maximum pooling respectively; and They are the corresponding convolution kernels respectively;

[0039] Add an attention mechanism after the convolution operation of each branch to enhance the expression of important features:

[0040] Perform vector extraction on the output feature X after the convolution operation, extract the query vector matrix Q, key vector matrix K and value vector matrix V respectively, and calculate the attention weight A l :

[0041]

[0042] Where T is the transpose of the vector; λ is the scalar factor;

[0043] Get enhanced feature E l :

[0044] E l =A l × l

[0045] Multi-scale fusion of features processed by different branches:

[0046] F=Fusion(E l )

[0047] Among them, F is the multi-scale fusion output result; Fusion(·) is the fusion operator;

[0048] Generate training data based on the GAN network for model training;

[0049] During the training process, the cross entropy loss function, contrast loss function, center loss function and BCE logit loss function are combined, and the four loss functions are weightedly fused to obtain the overall loss function. The weight of each loss function is obtained based on genetic algorithm optimization. Quantization and pruning techniques are used to compress the model. Finally, the Softmax classifier is used to classify the extracted features into actions.

[0050] Preferably, the data monitoring unit compares the collected operation data with the normal operation data range, classifies the abnormality level according to the abnormality of the operation data, and triggers the abnormality alarm unit to issue a corresponding abnormality alarm and take corresponding measures.

[0051] Preferably, the on-site management unit pre-assigns access permissions to operating personnel.

[0052] Preferably, the system also installs the equipment in a protective box. The front side of the protective box is designed as a three-section type, and is also provided with a multifunctional front baffle table. The rear side is provided with a wire outlet, double handles are installed on both sides, and the bottom is equipped with universal wheels. The protective box has an IP20 waterproof rating.

[0053] Preferably, the system also evaluates the system's effectiveness through accident incidence, operating efficiency, operating standardization and emergency handling capabilities, and optimizes and adjusts the system based on the evaluation results.

[0054] Compared with the prior art, the present invention provides an intelligent monitoring system for limited space operations based on multi-sensor fusion, which has the following beneficial effects:

[0055] The present invention senses and monitors the situation in a confined space in real time by fusing data from multiple sensors, thereby ensuring operational safety and normal equipment operation. The data collection method using multiple sensors can improve detection accuracy and reliability, and reduce operational risks. It can also accurately determine the position and status of operators in a confined space in real time by fusing intelligent mobile devices with other sensors. It can also identify the behavior of people, vehicles, and non-motor vehicles through behavior detection methods, and make corresponding alarms and strengthen management to further improve operational safety. It can transmit multi-sensor fused data to a remote monitoring center in real time through the Internet of Things technology, thereby realizing remote real-time monitoring of confined space operations and providing all-round safety protection for confined space operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a system block diagram mentioned in Embodiment 1 of the present invention;

[0057] Figure 2 This is a schematic diagram of the helmet mentioned in Example 1 of the present invention;

[0058] Figure 3 This is a rendering of the platform mentioned in Example 1 of the present invention;

[0059] Figure 4 A schematic diagram of the key points of the human skeleton and their corresponding names mentioned in Example 1 of the present invention;

[0060] Figure 5 This is a schematic diagram of the recognition accuracy of different models mentioned in Example 1 of the present invention;

[0061] Figure 6 This is a schematic diagram of the overall structure of the protection box mentioned in Example 1 of the present invention;

[0062] Figure 7 This is a schematic diagram of the alarm pop-up window mentioned in Example 1 of the present invention;

[0063] Figure 8 This is a schematic diagram of the abnormal information notification mentioned in Example 1 of the present invention.

[0064] Meaning of the symbols in the figure:

[0065] 100, data acquisition module; 200, data storage module; 300, data transmission module; 400, data processing module; 410, operation training unit; 420, behavior detection unit; 430, data monitoring unit; 440, abnormal alarm unit; 450, on-site management unit. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0067] The present invention senses and monitors the situation in a limited space in real time by fusing data from multiple sensors, ensuring operational safety and normal operation of equipment. The data collection method using multiple sensors can improve the accuracy and reliability of detection and reduce operational risks. It can also accurately determine the position and status of operators in a limited space in real time by fusing intelligent mobile devices with other sensors. It can also identify the behavior of people, vehicles, and non-motor vehicles by behavior detection methods, and make corresponding alarms and strengthen management to further improve operational safety. The data fused by multiple sensors is transmitted to a remote monitoring center in real time through the Internet of Things technology, realizing remote real-time monitoring of limited space operations and providing all-round safety protection for limited space operations. Specifically, it includes the following contents.

[0068] Embodiment 1:

[0069] See also Figure 1-6The present invention provides an intelligent monitoring system for limited space operations based on multi-sensor fusion, comprising:

[0070] Data acquisition module 100: used to collect operation data in a limited space through a number of data acquisition sensors; limited space operations refer to operations performed in closed or semi-closed environments such as containers, equipment, and pipelines, which are prone to safety accidents due to small spaces and poor ventilation. Through a variety of data acquisition sensors, the gas, vital signs of personnel, and images at the operation site in a limited space are monitored and collected in real time, and the operation behavior is further managed based on subsequent modules, which can ensure safety during the operation.

[0071] The data acquisition sensors of the data acquisition module 100 include an intelligent data acquisition server, a high-definition AI behavior monitoring ball camera, an environmental camera, a smart bracelet, a gas detector, and a smart helmet. The collected operation data is also converted through a data AD converter. The functions and specific parameters of each sensor selected by the system are as follows:

[0072] The intelligent data acquisition server is mainly used to collect data signals from various sensors, 2.4G / 5G dual-frequency transmission, supports IEEE802.3at protocol, the highest speed is 867Mbps, uses POE remote power supply, uses 1000Mbps electrical port uplink, and supports wireless WIFI access point LAN, point-to-multipoint MESH networking, and IP65 standard.

[0073] The highest resolution of the environmental camera can reach 2560×1440@25fps, and it can output real-time images at this resolution; it also supports backlight compensation, strong light suppression, 3D digital noise reduction, 120dB wide dynamic range, and supports ROI region of interest enhancement coding. The device is also equipped with an electric zoom lens, which is easy to operate and the zoom process is smooth.

[0074] The environmental camera supports intelligent resource mode switching, including face capture mode, road monitoring mode, Smart event mode and people counting mode.

[0075] Face capture modes: a supports detection, capture, scoring, and screening of moving faces, and outputs the best face; b supports face false alarm removal and fast face capture; c supports two modes: fast capture and best capture; d can detect up to 30 faces at the same time; e supports face deduplication.

[0076] Road monitoring modes: a. Vehicle detection: supports license plate recognition and capture, vehicle model / brand / body color / license plate color recognition; b. Mixed traffic detection: detects vehicles traveling in the forward or reverse direction as well as pedestrians and non-motor vehicles, automatically identifies vehicle license plates, and can capture images of vehicles without license plates; c. Supports switching between checkpoint and entrance and exit modes.

[0077] Smart event mode: supports cross-border detection, area intrusion detection, entry area detection, exit area detection, item left behind detection, item taking detection, loitering detection, crowd gathering detection, rapid motion detection, and parking detection.

[0078] People counting mode: a) People counting: supports real-time alarm, alarm for number of people change and alarm for congestion level change, and supports alarm for abnormal number of people and abnormal stay time, b) Behavior analysis: supports off-post detection and off-post detection alarm, c) Regional attention: supports regional number of people detection, stay time detection, real-time data upload, and supports regional number of people analysis and queue status analysis display, d) Heat map: supports device reporting and platform query to obtain information, and supports reporting of pseudo-color background large pictures.

[0079] The specifications of the smart bracelet can be found in Table 1:

[0080] Table 1 Specifications of smart bracelet

[0081]

[0082]

[0083] The smart bracelet based on the above-mentioned specifications can detect the wearer's health, heart rate, blood pressure, body temperature, step counting, sleep detection, accelerometer belt, etc.

[0084] Gas detectors include handheld high-function gas detectors and portable gas detectors. The parameter specifications of handheld high-function gas detectors can be found in Table 2:

[0085] Table 2 Specifications of handheld high-function gas detectors

[0086]

[0087]

[0088] The parameter specifications of portable gas detectors can be found in Table 3:

[0089] Table 3 Specifications of portable gas detectors

[0090]

[0091]

[0092] The specifications of the smart helmet can be found in Table 4:

[0093] Table 4 Specifications of smart helmets

[0094]

[0095]

[0096]

[0097] The various data acquisition sensors of the data acquisition module 100 also convert the collected operation data through a data AD converter. The technical parameters of the data AD converter can be referred to Table 5:

[0098] Table 5 Technical parameters of data AD converter

[0099]

[0100]

[0101] The data storage module 200 can store alarm pictures and alarm videos through a 4T solid-state hard drive. It can be configured with 2 10M / 100M / 1000Mbps network ports, 1 USB2.0 interface, 1 USB3.0 interface and an alarm IO interface, including 4 alarm inputs and 1 alarm output.

[0102] The data transmission module 300 supports wireless, 4G / 5G dual-band WIFI, wired and other methods, and also supports the expansion of 5G. It replaces the traditional networking solution through cloud networking technology. It is not restricted by the network and can quickly assemble remote virtual LANs to achieve interconnection between computers, networks, and servers at both ends of the network, and share important information such as documents, pictures, audio, video, and databases.

[0103] The data processing module 400 processes and manages the received data in a centralized manner through the central control system, and displays the data on the display screen of the industrial host. Figure 3

[0104] The data processing module 400 includes:

[0105] Operation training unit 410: used to train operators on safe operation procedures, and generate corresponding safety operation procedures for enhanced training according to the detection results of the behavior detection module;

[0106] Design training content: The training content of safe operating procedures should cover all aspects of the operation, including the correct operation of equipment, identification of hazardous areas, use of personal protective equipment, etc. For example, in chemical production operations, the training content may include the operation procedures of specific chemical reactors, storage and handling specifications for hazardous chemicals, etc.

[0107] Training materials can include text instructions, picture examples, video tutorials, and other forms to meet the needs of operators with different learning styles. For example, a series of short videos on the operation steps of the equipment can be produced to clearly show the details and precautions of each operation.

[0108] Enhanced training based on behavior detection: When the behavior detection unit 420 finds that the operator has engaged in behavior that does not comply with safety operating procedures, the operation training unit 410 should be able to automatically generate a targeted enhanced training plan. For example, if the behavior detection unit 420 finds that the operator does not wear protective glasses as required when operating a certain device, the operation training unit 410 will generate an enhanced training course on the importance of protective glasses and the correct way to wear them, and arrange for the operator to study at an appropriate time. Enhanced training can be implemented using an online learning platform, where operators can access training content at any time through computers or mobile devices, and take online tests after completing the study to verify the learning effect.

[0109] Behavior detection unit 420: used to monitor the behavior of operators in real time through a high-definition AI behavior monitoring ball camera based on image recognition technology, and correct unsafe behaviors;

[0110] The parameters and specifications of the HD AI behavior monitoring ball camera can be found in Table 6:

[0111] Table 6 Specifications of HD AI behavior monitoring dome camera

[0112]

[0113]

[0114] Based on the above parameters and specifications, the high-definition AI behavior monitoring ball camera structures people, cars and non-motor vehicles, among which, refer to Figure 4 , people are divided into 17 key point sets according to the key points of the skeleton, and cars are divided into 13 key point sets according to the center points of the four wheels, the four corner points of the front and rear bumpers of the vehicle, the four center points of the headlights and taillights of the vehicle, and the center point of the top of the vehicle; non-motor vehicles are divided into 10 key point sets according to the center points of the two wheels, the center points of the two handlebars, the four key connection points of the frame, and the center points of the two pedals;

[0115] Based on the behavior detection unit 420, images or videos are collected, and feature extraction is performed based on the divided key point sets to form a feature matrix. ij ,y ij ,z ij ) Generate the corresponding key point stacking data G(x, y, z) in the form of Gaussian distribution, where i is the key point set and j is the key point; thereby obtaining the input feature matrix G:

[0116]

[0117] Where σ is the standard deviation of the Gaussian distribution;

[0118]

[0119] Where D(·) is the distance calculation function; it is used to calculate the distance between a point (x, y, z) and a line segment s(b ij ,d ij ) between the two ij ,d ij are the key points at both ends of the line segment;

[0120] Use 1×1 convolution to divide the input features into 6 different branches according to channel C: For the input feature matrix G, whose dimension is [B, C, H, W], the convolution operation is as follows:

[0121]

[0122] Among them, B represents the batch size, C represents the number of channels, H represents the height, and W represents the width. Through 6 different 1×1 convolution kernels The output of 6 branches is obtained, and its dimension is [B,C l ,H,W];

[0123] Apply 3×1 convolution, 3×3 convolution, dilated convolution, 3×1 maximum pooling, and no operation to each branch; for the output y of branch l l , and obtain z by performing convolution operation with the corresponding convolution kernel l , as follows:

[0124]

[0125]

[0126]

[0127]

[0128] in, and They are the outputs of 3×1 convolution, 3×3 convolution, dilated convolution, and 3×1 maximum pooling respectively; and They are the corresponding convolution kernels respectively;

[0129] Add an attention mechanism after the convolution operation of each branch to enhance the expression of important features:

[0130] Perform vector extraction on the output feature X after the convolution operation, extract the query vector matrix Q, key vector matrix K and value vector matrix V respectively, and calculate the attention weight A l :

[0131]

[0132] Where T is the transpose of the vector; λ is the scalar factor;

[0133] Get enhanced feature E l :

[0134] E l =A l × l

[0135] Multi-scale fusion of features processed by different branches:

[0136] F=Fusion(E l )

[0137] Among them, F is the multi-scale fusion output result; Fusion(·) is the fusion operator;

[0138] Generate training data based on the GAN network for model training;

[0139] During the training process, the cross entropy loss function, contrast loss function, center loss function and BCE logit loss function are combined, and the four loss functions are weighted and fused to obtain the overall loss function. The weights of each loss function are obtained based on the genetic algorithm optimization. The model is compressed by applying quantization and pruning techniques. Finally, the Softmax classifier is used to classify the extracted features. When unsafe behavior is detected, the system should immediately issue an alarm and notify the operator through sound, light or display prompts. For example, in a factory workshop, when it is detected that a worker stays in a dangerous area for too long, the system will sound a loud alarm and display a warning message on a nearby display.

[0140] At the same time, the system can send records of unsafe behaviors to relevant managers so that they can take timely measures to correct and guide them. For example, managers can receive a report containing the time, location and specific circumstances of the unsafe behavior, and then go to the site to provide face-to-face guidance and education to the operators.

[0141] Take the CNN network as model 1, the CNN network integrating multi-convolution model as model 2, the CNN network integrating multi-convolution model and multi-scale fusion as model 3, and the above model 4 as model 5. Based on the above, various behaviors are monitored and identified. The recognition accuracy can be referred to Figure 5,It can be seen from the figure that the recognition accuracy of model 4 is higher than that of models 1, 2, and 3, and has higher application value.

[0142] Data monitoring unit 430: used to collect and analyze operation data in real time;

[0143] Collect various operation data in real time, including equipment operation parameters such as temperature, pressure, speed, etc., environmental data such as air quality, humidity, noise, etc., and operator operation data such as the time and frequency of equipment operation. For example, in a machining workshop, sensors are used to collect data such as machine tool operation temperature and cutting force in real time.

[0144] Data acquisition equipment should have high reliability and accuracy, and be able to carry out stable data transmission with the data monitoring unit. Technologies such as wireless sensor networks can be used to facilitate deployment and data transmission in complex work sites.

[0145] The collected operation data is compared with the normal operation data range, and the abnormal level is divided according to the abnormal situation of the operation data, and the abnormal alarm unit 440 is triggered to make corresponding abnormal alarms and take countermeasures. For example, the hardware device or software system may fail or be unstable. The countermeasures taken are: conduct sufficient testing, select equipment and technical solutions with high stability; regularly maintain and upgrade to ensure stable operation of the system.

[0146] Operators may use the system improperly due to lack of familiarity. The countermeasures are: provide operators with adequate training to ensure that they are proficient in system operation methods; set up authority control to prevent misoperation.

[0147] Data transmission may be attacked, resulting in data leakage. The countermeasures taken are: strengthen data encryption and access permission control to ensure data security; conduct security checks and vulnerability repairs regularly to prevent security risks.

[0148] Abnormal alarm unit 440: used for automatically alarming detected abnormal operation data;

[0149] When the data monitoring unit 430 detects abnormal operation data, the abnormal alarm unit 44 should be able to quickly trigger an alarm. The alarm method can be diversified, including sending text messages or emails to relevant personnel, displaying alarm information in the monitoring center, triggering on-site sound and light alarms, etc. For example, refer to Figure 7-8 When the data detected by the gas detector is abnormal, the system will pop up an alarm window, and the abnormal information will be displayed synchronously in the abnormal information column.

[0150] The alarm level can be classified according to the severity of the abnormal situation, and different response measures can be taken for different levels of alarms. For example, for a minor abnormal situation, it may only be necessary to notify the on-site operators to check and deal with it; while for a serious abnormal situation, it may be necessary to immediately activate the emergency plan and evacuate the operators.

[0151] The abnormal alarm unit 440 should work closely with the behavior detection unit 420 and the data monitoring unit 430. When an abnormal alarm occurs, the behavior detection unit 420 can further strengthen the monitoring of the operator's behavior to prevent the operator from engaging in unsafe behavior due to abnormal conditions. For example, when equipment failure causes changes in the on-site environment, the behavior detection unit 420 will pay closer attention to whether the operator has taken the correct response measures.

[0152] The data monitoring unit 430 can continue to collect and analyze relevant data after the alarm to provide more information for troubleshooting and problem solving. For example, after the alarm, the changing trend of the equipment parameters can be continuously monitored to help technicians determine the cause of the fault.

[0153] On-site management unit 450: used to perform hierarchical authorization confirmation on the entry and exit behavior of operators.

[0154] Authorize operators to enter and exit different areas or perform different operations in different levels. For example, in places with extremely high safety requirements such as nuclear power plants, the operating areas are divided into different security levels, and only operators who have received specific training and authorization can enter the core area to operate. For general maintenance personnel, they may only be authorized to work in external auxiliary areas.

[0155] Authorization can be achieved through smart cards, biometric technologies such as fingerprint recognition, face recognition, etc. Before entering the corresponding area or performing operations, operators need to pass the corresponding identity authentication and authorization confirmation.

[0156] The site management unit 450 should record the entry and exit behavior and operation authorization of the operator to form a detailed log record. These records can be used for post-event security audit and tracing. For example, if a safety accident occurs, the cause and responsibility of the accident can be analyzed by viewing the entry and exit records and operation authorization of the operator.

[0157] A strict management process should be established for changes and updates to authorizations. For example, when an operator’s position or responsibilities change, their authorization level and scope of authority need to be updated in a timely manner to ensure that they can only perform operations that meet their qualifications and safety requirements.

[0158] The system also generates corresponding logs, and relevant personnel can search for relevant records based on personnel, equipment, status, time, etc.

[0159] The system equipment is also installed in a protective box, please refer to Figure 6 The front side of the protection box is designed in three sections, with a multifunctional front baffle table, a cable outlet on the rear side, universal wheels on the bottom, and double handles on both sides, making it easier to move and carry. The protection box has an IP20 waterproof rating to ensure the safety of the internal equipment. You can easily install the display equipment, storage devices, or wireless acquisition servers in the protection box.

[0160] The system also conducts an annual effectiveness evaluation of the system through accident rate, operating efficiency, operating standardization and emergency response capabilities, and optimizes and adjusts the system based on the evaluation results.

[0161] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An intelligent monitoring system for limited space operations based on multi-sensor fusion, characterized in that: include: A data acquisition module (100): used to collect operation data in a limited space through a plurality of data acquisition sensors; Data storage module (200): used for storing operation data via a solid state hard disk; Data transmission module (300): used for transmitting operation data wirelessly or wirelessly based on cloud networking technology; Data processing module (400): used for centrally processing and managing received data; The data processing module (400) comprises: Operation training unit (410): used to train operators on safe operation procedures, and generate corresponding safety operation procedures for enhanced training according to the detection results of the behavior detection module; Behavior detection unit (420): used to monitor the behavior of operators in real time based on image recognition technology through a high-definition AI behavior monitoring ball camera, and correct unsafe behaviors; Data monitoring unit (430): used to collect and analyze operation data in real time; Abnormal alarm unit (440): used for automatically alarming detected abnormal operation data; On-site management unit (450): used to perform hierarchical authorization confirmation on the entry and exit behavior of operators.

2. According to claim 1, the intelligent monitoring system for limited space operations based on multi-sensor fusion is characterized in that: The data acquisition sensor of the data acquisition module (100) collects operating data of an environmental camera, a smart bracelet, a gas detector, and a smart helmet through an intelligent data acquisition server; the environmental camera supports intelligent resource mode switching, and the intelligent resource mode includes a face capture mode, a road monitoring mode, a smart event mode, and a people counting mode.

3. According to claim 2, the intelligent monitoring system for limited space operations based on multi-sensor fusion is characterized in that: The data acquisition module (100) also converts the collected operation data through a data AD converter.

4. According to claim 1, the intelligent monitoring system for limited space operations based on multi-sensor fusion is characterized in that: The behavior detection unit (420) structures people, vehicles and non-motor vehicles based on a high-definition AI behavior monitoring ball camera, wherein people are divided into 17 key point sets according to skeleton key points, and vehicles are divided into 13 key point sets according to the center points of four wheels, four corner points of the front and rear bumpers of the vehicle, four center points of the front headlights and rear taillights of the vehicle, and the center point of the top of the vehicle; non-motor vehicles are divided into 10 key point sets according to the center points of two wheels, the center points of two handlebars, four key connection points of the frame and the center points of two pedals; Based on the behavior detection unit (420), images or videos are collected, and features are extracted based on the divided key point sets to form a feature matrix. ij ,y ij ,z ij ) Generate the corresponding key point stacking data G(x, y, z) in the form of Gaussian distribution, where i is the key point set and j is the key point; thereby obtaining the input feature matrix G: Where σ is the standard deviation of the Gaussian distribution; Where D(·) is the distance calculation function; it is used to calculate the distance between a point (x, y, z) and a line segment s(b ij ,d ij ) between the two ij ,d ij are the key points at both ends of the line segment; Use 1×1 convolution to divide the input features into 6 different branches according to channel C: For the input feature matrix G, whose dimension is [B, C, H, W], the convolution operation is as follows: Among them, B represents the batch size, C represents the number of channels, H represents the height, and W represents the width. Through 6 different 1×1 convolution kernels The output of 6 branches is obtained, and its dimension is [B,C l ,H,W]; Apply 3×1 convolution, 3×3 convolution, dilated convolution, 3×1 maximum pooling, and no operation to each branch; for the output y of branch l l , and obtain z by performing convolution operation with the corresponding convolution kernel l , as follows: in, and They are the outputs of 3×1 convolution, 3×3 convolution, dilated convolution, and 3×1 maximum pooling respectively; and They are the corresponding convolution kernels respectively; Add an attention mechanism after the convolution operation of each branch to enhance the expression of important features: Perform vector extraction on the output feature X after the convolution operation, extract the query vector matrix Q, key vector matrix K and value vector matrix V respectively, and calculate the attention weight A l : Where T is the transpose of the vector; λ is the scalar factor; Get enhanced feature E l : AND l =A l ×z l Multi-scale fusion of features processed by different branches: F=Fusion(E l ) Among them, F is the multi-scale fusion output result; Fusion(·) is the fusion operator; Generate training data based on the GAN network for model training; During the training process, the cross entropy loss function, contrast loss function, center loss function and BCE logit loss function are combined, and the four loss functions are weightedly fused to obtain the overall loss function. The weight of each loss function is obtained based on genetic algorithm optimization. Quantization and pruning techniques are used to compress the model. Finally, the Softmax classifier is used to classify the extracted features into actions.

5. The intelligent monitoring system for limited space operations based on multi-sensor fusion according to claim 1 is characterized in that: The data monitoring unit (430) compares the collected operation data with the normal operation data range, classifies the abnormality level according to the abnormality of the operation data, and triggers the abnormality alarm unit (440) to issue a corresponding abnormality alarm and take corresponding measures.

6. The intelligent monitoring system for limited space operations based on multi-sensor fusion according to claim 1 is characterized in that: The on-site management unit (450) pre-assigns entry and exit permissions to operators.

7. The intelligent monitoring system for limited space operations based on multi-sensor fusion according to claim 1 is characterized in that: The system also installs the equipment in a protective box. The front side of the protective box is designed as a three-section type, and is also provided with a multifunctional front baffle table. The rear side is configured with a wire outlet, double handles are installed on both sides, and the bottom is equipped with universal wheels. The protective box has an IP20 waterproof rating.

8. The intelligent monitoring system for limited space operations based on multi-sensor fusion according to claim 1 is characterized in that: The system also evaluates its effectiveness through accident rate, operating efficiency, operating standardization and emergency response capabilities, and optimizes and adjusts the system based on the evaluation results.

Images