Limited space safety risk grading early warning method and system based on multi-modal fusion

Abstract

The invention provides a limited space safety risk grading early warning method and system based on multi-modal fusion. The limited space safety risk grading early warning method comprises the steps that multi-modal data acquisition device deployment setting is carried out for a limited space; performing video monitoring and gas monitoring on the limited space through a multi-modal data acquisition device based on a defense deployment setting result, acquiring multi-modal data acquisition information, and wirelessly transmitting the multi-modal data acquisition information to a data analysis center; the data analysis center is used for performing personnel state analysis and gas detection analysis on the multi-modal data acquisition information by adopting an artificial intelligence algorithm to obtain safety analysis data; and determining a safety risk level according to the safety analysis data, and performing graded early warning according to the safety risk level. According to the invention, multi-information detection analysis of the restricted space is realized, and the comprehensiveness of security risk analysis of the restricted space is improved.

Description

Technical Field

[0001] This invention relates to the field of detection and early warning technology, and in particular to a method and system for graded early warning of safety risks in confined spaces based on multimodal fusion. Background Technology

[0002] Confined spaces are enclosed or semi-enclosed facilities and places with restricted access, poor ventilation, and the potential presence of flammable, explosive, toxic, or hazardous substances or oxygen deficiency, posing a threat to the health and safety of personnel entering them. Confined spaces are widespread in all sectors of society. Working in confined spaces presents various safety risks, such as gas poisoning, asphyxiation, and explosions, seriously threatening the lives of workers.

[0003] With the development of construction technology and the improvement of construction systems, construction units are paying increasing attention to the safety management of operations. Traditional confined space safety risk early warning methods mainly rely on single gas detection or personnel experience judgment, which has problems such as incomplete detection, untimely response, and low accuracy. Therefore, there is an urgent need for a confined space safety risk classification and early warning method and system based on multimodal fusion to solve the above-mentioned technical problems. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for classifying and warning safety risks in confined spaces based on multimodal fusion, which enables multi-information detection and analysis of confined spaces, improves the comprehensiveness of confined space safety risk analysis, and solves the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for graded early warning of safety risks in confined spaces based on multimodal fusion, comprising:

[0006] Deployment and setup of multimodal data acquisition devices for confined spaces;

[0007] Based on the deployment results, video monitoring and gas monitoring are carried out in the confined space using multimodal data acquisition devices to obtain multimodal data acquisition information, and the multimodal data acquisition information is wirelessly transmitted to the data analysis center.

[0008] At the data analysis center, artificial intelligence algorithms are used to analyze personnel status and gas detection based on multimodal data collection information to obtain safety analysis data;

[0009] The security risk level is determined based on the security analysis data, and a graded warning is issued according to the security risk level.

[0010] Furthermore, the multimodal data acquisition device includes:

[0011] The first acquisition unit uses various gas acquisition devices to sense gases in the confined space and obtains gas sensing acquisition data.

[0012] The second acquisition unit acquires images of the confined space using an image acquisition device to obtain video monitoring data.

[0013] The communication transmission unit includes an internal transmission unit and an external transmission unit. The internal transmission unit is used to transmit and converge gas sensing data and video monitoring data in a confined space to determine multimodal data acquisition information in real time. The external transmission unit is used to wirelessly transmit the multimodal data acquisition information to the data analysis center.

[0014] The battery power supply unit is used to supply power and detect the power level of the first and second acquisition units.

[0015] Furthermore, before wirelessly transmitting the multimodal data acquisition information to the data analysis center, wireless gateway simulation tests and optimizations are conducted to determine the target wireless gateway, and then the multimodal data acquisition information is wirelessly transmitted to the data analysis center based on the target wireless gateway.

[0016] Furthermore, artificial intelligence algorithms are employed to perform personnel status analysis and gas detection analysis on multimodal data acquisition information, including:

[0017] A gas analysis algorithm is used to identify and filter target data information from multimodal data acquisition, and gas detection and safety analysis are performed based on the target data information to obtain the first safety analysis data.

[0018] Video analytics algorithms are used to identify effective data information from multimodal data collection, and personnel status identification and security analysis are performed based on the effective data information to obtain second security analysis data.

[0019] Furthermore, gas analysis algorithms are employed to identify and filter target data information from multimodal data acquisition, and gas detection and safety analysis are performed based on the target data information, including:

[0020] Analyze multimodal data acquisition information, obtain gas sensing data from the multimodal data acquisition information, and obtain target data information;

[0021] The oxygen content is determined based on the target data information, and a safety analysis is performed based on the oxygen content to obtain oxygen safety analysis data;

[0022] Flammable and explosive gases and harmful gases are screened and extracted from the target data information to obtain the first screening information and the second screening information.

[0023] Based on the first screening information and the combustion and explosion phenomena, the first gas analysis is performed to obtain combustion and explosion safety analysis data;

[0024] Based on the second screening information and the hazardous phenomena, a second gas analysis is performed to obtain hazardous safety analysis data.

[0025] Furthermore, video analytics algorithms are employed to identify effective data information from multimodal data collection, and personnel status identification and security analysis are performed based on this effective data information, including:

[0026] Analyze the multimodal data acquisition information to obtain the video monitoring data from the multimodal data acquisition information and obtain effective data information;

[0027] Video analytics algorithms are used to perform image recognition and analysis on valid data information to locate people in confined spaces and obtain their status information.

[0028] By combining the safety standards for personnel working in confined spaces with personnel status information, a second safety analysis data is obtained.

[0029] Furthermore, video analysis algorithms are used to perform image recognition analysis on valid data information, including:

[0030] Analyze the video angles of valid data to determine the video acquisition perspective for video monitoring data collection.

[0031] Based on the video surveillance data, the video acquisition perspective is used to perform image recognition scanning analysis on the effective data information in sequence to determine whether there are people in the confined space and obtain preliminary recognition results.

[0032] Further video analysis and processing are carried out based on the preliminary identification results. When the preliminary identification results indicate that a person is present in a confined space, the target range of the person present in the confined space is locked and the image is processed in the video monitoring data to obtain the target image of the person in the video monitoring.

[0033] Perform personnel status recognition and analysis on target images of personnel video monitoring to obtain personnel status information.

[0034] Furthermore, when determining the security risk level based on security analysis data, the security analysis data is summarized and organized, and the security risk level is obtained based on the summary and organization results and the level determination rules.

[0035] Furthermore, different security risk levels correspond to different early warning modes. After issuing early warnings according to the security risk level, related data can be viewed based on the security risk level.

[0036] A confined space safety risk classification and early warning system based on multimodal fusion includes:

[0037] The deployment module is used to deploy multimodal data acquisition devices in confined spaces.

[0038] The multimodal data acquisition module is used to perform video monitoring and gas monitoring in confined spaces based on the deployment settings through a multimodal data acquisition device, acquire multimodal data acquisition information, and wirelessly transmit the multimodal data acquisition information to the data analysis center;

[0039] The data analysis center is used to perform personnel status analysis and gas detection analysis on multimodal data collection information using artificial intelligence algorithms to obtain safety analysis data;

[0040] The graded early warning module is used to determine the security risk level based on security analysis data and to issue graded early warnings according to the security risk level.

[0041] This invention achieves multi-information detection and analysis of confined spaces through multimodal data acquisition, improving the comprehensiveness of confined space safety risk analysis. Furthermore, it incorporates artificial intelligence algorithms to determine safety analysis data, eliminating the need for subjective analysis and judgment based on human experience, thus ensuring the accuracy and efficiency of safety analysis data. This allows for timely determination of safety risk levels and tiered early warning, enhancing the responsiveness of safety analysis and enabling timely implementation of measures based on safety risk levels to ensure the safety of personnel working in confined spaces.

[0042] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the application.

[0043] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0044] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0045] Figure 1 This is a flowchart illustrating the confined space safety risk classification and early warning method described in this invention.

[0046] Figure 2 This is a schematic diagram of step three in the confined space safety risk classification and early warning method described in this invention;

[0047] Figure 3 This is a schematic diagram of the confined space safety risk classification and early warning system described in this invention. Detailed Implementation

[0048] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0049] like Figure 1 As shown, this embodiment of the invention provides a method for graded early warning of safety risks in confined spaces based on multimodal fusion, including:

[0050] Step 1: Deploy and configure multimodal data acquisition devices for confined spaces.

[0051] The multimodal data acquisition device is a system that combines gas and image acquisition. When deploying this device in a confined space, an environmental assessment is conducted to determine the type of confined space, its structure, and potential hazardous gases. Based on the spatial structure, image acquisition analysis is performed to determine the placement of the image acquisition devices. Simultaneously, gas acquisition devices are selected based on the potential hazardous gases, and their installation methods are determined through gas acquisition analysis based on their properties. The corresponding gas acquisition devices are then deployed according to their installation method: top-mounted, ground-mounted, or side-wall adsorption deployment. Here, the properties of potential hazardous gases include gas density and the location of the release source. When determining the placement of the gas acquisition devices, these properties are compared with air density, and the top-mounted, ground-mounted, or side-wall adsorption deployment is based on the comparison analysis results.

[0052] Step 2: Based on the deployment results, use a multimodal data acquisition device to conduct video monitoring and gas monitoring in the confined space, acquire multimodal data acquisition information, and wirelessly transmit the multimodal data acquisition information to the data analysis center.

[0053] Specifically, when conducting video surveillance and gas monitoring in a confined space based on the deployment settings, the video and gas acquisition devices are used to monitor the space according to their deployment configuration, thereby obtaining multimodal data acquisition information. This multimodal data acquisition information includes gas sensing data and video monitoring data. When wirelessly transmitting the multimodal data acquisition information to the data analysis center, real-time wireless transmission is performed on the obtained multimodal data acquisition information.

[0054] Step 3: At the data analysis center, artificial intelligence algorithms are used to perform personnel status analysis and gas detection analysis on the multimodal data collection information to obtain safety analysis data.

[0055] The data analysis center is located in a different area from the confined space. Artificial intelligence algorithms include gas analysis algorithms and video analysis algorithms. Safety analysis data includes gas safety analysis data and personnel status analysis data. Gas safety analysis data is obtained by using gas analysis algorithms to perform gas detection and analysis based on multimodal data acquisition information. Personnel status analysis data is obtained by using video analysis algorithms to perform personnel status analysis based on multimodal data acquisition information.

[0056] Step 4: Determine the security risk level based on the security analysis data, and issue graded warnings according to the security risk level.

[0057] The safety risk level is determined based on the consequences of the safety hazard, and includes: Level 1 risk, Level 2 risk, and Level 3 risk.

[0058] The aforementioned multimodal data acquisition system enables comprehensive detection and analysis of confined spaces, enhancing the overall comprehensiveness of confined space safety risk analysis. Furthermore, it incorporates artificial intelligence algorithms for safety analysis data determination, eliminating the need for subjective judgment based on human experience, thus ensuring the accuracy and efficiency of safety analysis data. This allows for timely determination of safety risk levels and tiered early warnings, improving the responsiveness of safety analysis and enabling timely implementation of measures based on safety risk levels to ensure the safety of personnel working in confined spaces. The deployment of multimodal data acquisition devices in confined spaces ensures comprehensive and accurate data acquisition, enabling better safety analysis based on this data and providing a basis for determining safety risk levels and issuing tiered early warnings. Wireless transmission of multimodal data to the data analysis center allows for timely transfer of acquired multimodal data during video monitoring and gas monitoring of confined spaces, improving response timeliness and reducing latency. By employing artificial intelligence algorithms to analyze personnel status and gas detection data from multimodal data collection, manpower is saved. This eliminates the need for human experience in analysis and judgment, reducing errors in safety analysis, ensuring the accuracy of safety analysis data, and efficiently obtaining data while minimizing time consumption. By determining safety risk levels based on the analysis data and implementing tiered early warning systems, different signals can be conveyed through different warning methods, increasing the vigilance of relevant personnel and enabling them to take appropriate measures based on the safety analysis, thereby better ensuring the safety of personnel working in confined spaces.

[0059] In one embodiment of the present invention, the multimodal data acquisition device includes: a first acquisition unit, a second acquisition unit, a communication transmission unit, and a battery power supply unit; both the communication transmission unit and the battery power supply unit are connected to the first acquisition unit and the second acquisition unit.

[0060] The first acquisition unit uses various gas acquisition devices to sense gases in a confined space and obtain gas sensing data.

[0061] The gas collection device includes various gas sensors that can detect and collect different types of gases, including oxygen, carbon monoxide, hydrogen sulfide, vinyl chloride, benzene, and other gases.

[0062] The second acquisition unit acquires images of the confined space using an image acquisition device, thereby obtaining video monitoring data.

[0063] Among them, the image acquisition device can perform all-round monitoring and image acquisition in confined spaces.

[0064] The communication transmission unit includes an internal transmission unit and an external transmission unit. The internal transmission unit is used to transmit and converge gas sensing data and video monitoring data in a confined space to determine multimodal data acquisition information in real time. The external transmission unit is used to wirelessly transmit the multimodal data acquisition information to the data analysis center.

[0065] The internal transmission unit enables high-speed wireless communication within a confined space. The external transmission unit communicates with 4G or 5G base stations within the data analysis center. When transmitting and aggregating gas sensing and video monitoring data within the confined space, the internal transmission unit timestamps the real-time gas sensing data from the first acquisition unit and the real-time video monitoring data from the second acquisition unit for external transmission preprocessing. The external transmission unit then wirelessly transmits this preprocessed multimodal data to the data analysis center for security analysis.

[0066] The battery power supply unit is used to supply power and detect the power level of the first and second acquisition units.

[0067] The battery supply unit provides power to both the first and second acquisition units, ensuring they can perform gas sensing and image acquisition in confined spaces. Additionally, the battery power unit detects battery level and issues a voice alert when the remaining battery is low.

[0068] The above-mentioned multimodal data acquisition system, consisting of a first acquisition unit, a second acquisition unit, a communication transmission unit, and a battery power supply unit, enables comprehensive information collection in confined spaces, providing assurance for confined space safety analysis. The first acquisition unit uses multiple gas acquisition devices to sense gases in the confined space, determining the amount of potentially hazardous gases present. The second acquisition unit uses an image acquisition device to capture images of the confined space, clarifying the status of personnel within. A communication transmission unit promptly transfers the gas sensing and video monitoring data obtained from the first and second acquisition units for safety analysis. This not only avoids the need for continuous gas sensing and image acquisition by the first and second acquisition units but also allows for timely safety analysis based on the gas sensing and video monitoring data. Simultaneously, an internal transmission unit ensures communication within the confined space, enabling data transmission and providing data support for the external transmission unit. The external transmission unit also establishes a communication link between the confined space and an external data analysis center, allowing for timely transfer of multi-dimensional data acquisition information to the data analysis center for safety analysis, reducing response time and enabling timely tiered early warning for the confined space, thus ensuring its safety. A battery power unit ensures the gas sensing and image acquisition of the first and second acquisition units, preventing insufficient power from affecting the acquisition of multi-modal data.

[0069] In one embodiment of the present invention, before wirelessly transmitting multimodal data acquisition information to the data analysis center, wireless gateway simulation tests and optimizations are performed for wireless transmission to determine the target wireless gateway, and then the multimodal data acquisition information is wirelessly transmitted to the data analysis center based on the target wireless gateway.

[0070] The wireless gateway serves as the carrier for wireless communication. During the simulation testing and optimization of the wireless gateway for wireless transmission, a preliminary analysis is conducted on the confined space and data analysis center to determine if they are within the communication coverage area. If they are, simulated data is used to simulate wireless communication through the wireless gateway. The performance parameters of the simulated data during wireless transmission are analyzed, and the wireless gateway is optimized based on these parameters until they meet expectations. The current configuration state of the wireless gateway is then determined, resulting in the target network gateway. Subsequently, when wirelessly transmitting multimodal data acquisition information to the data analysis center, wireless transmission of the multimodal data acquisition information is performed based on the target wireless gateway. Here, performance parameters include transmission latency, data security, and anti-interference capabilities.

[0071] Before wirelessly transmitting multimodal data acquisition information to the data analysis center, the aforementioned wireless gateway simulation tests and optimizations are conducted to ensure successful wireless transmission. This guarantees that the multimodal data acquisition information can be transmitted to the data analysis center. Furthermore, preliminary analysis of the confined space and the data analysis center confirms that they are within the communication coverage area, preventing the wireless gateway from failing to transmit the multimodal data acquisition information to the data analysis center. Simultaneously, performance parameter analysis and optimization ensure the performance of wireless transmission, improve the transmission efficiency and security of multimodal data acquisition information, and thus provide a guarantee for the secure acquisition of analytical data.

[0072] In one embodiment provided by the present invention, such as Figure 2 As shown, artificial intelligence algorithms are used to perform personnel status analysis and gas detection analysis on multimodal data acquisition information, including:

[0073] S1. A gas analysis algorithm is used to identify and filter target data information from multimodal data acquisition, and gas detection and safety analysis are performed based on the target data information to obtain the first safety analysis data.

[0074] The gas analysis algorithm is pre-determined through artificial intelligence algorithm optimization training based on the potentially hazardous gases that may exist in the confined space. When using the gas analysis algorithm to identify and filter target data information from multimodal data acquisition information, only the data information related to gas analysis is selected from the multimodal data acquisition information for gas monitoring and safety analysis.

[0075] S2. A video analysis algorithm is used to identify effective data information from multimodal data collection, and personnel status identification and security analysis are performed based on the effective data information to obtain the second security analysis data.

[0076] The video analytics algorithm, while performing target data identification and filtering on multimodal data acquisition information using the gas analysis algorithm, also identifies valid data information within the multimodal data acquisition information. Based on this valid data information, it then performs personnel status identification and safety analysis. The video analytics algorithm is pre-determined through artificial intelligence algorithm optimization training based on confined space personnel operation safety regulations. Valid data information refers to the video monitoring data acquired for personnel status identification and safety analysis within the multimodal data acquisition information. When using the video analytics algorithm to identify valid data information from the multimodal data acquisition information, only the video monitoring data is parsed to determine the valid data information. Then, based on this valid data information, personnel status identification and safety analysis are performed to obtain the second safety analysis data.

[0077] The aforementioned safety analysis of multimodal data acquisition information, achieved through gas analysis and video analysis algorithms, enables efficient acquisition of safety analysis data, reducing response time and facilitating timely determination of safety risk levels for tiered early warning, thereby enhancing safety awareness. Gas analysis algorithms clearly identify safety hazards posed by gases within confined spaces, preventing combustion or explosions in limited environmental conditions. They also allow for timely detection of gas hazards to workers, preventing personal safety issues. Video analysis algorithms determine whether the workers' physical condition poses a safety hazard, preventing workers from failing to adhere to confined space safety regulations, thus further ensuring worker safety.

[0078] In one embodiment of the present invention, a gas analysis algorithm is used to identify and filter target data information from multimodal data acquisition information, and gas detection and safety analysis are performed based on the target data information, including:

[0079] The multimodal data acquisition information is analyzed to obtain the gas sensing data from the multimodal data acquisition information, and the target data information is obtained.

[0080] The target data information is the gas sensing and acquisition data. When parsing multimodal data acquisition information, the data information contained in the multimodal data acquisition information is identified, and the gas sensing and acquisition data in the multimodal data acquisition information is filtered and extracted to obtain the target data information.

[0081] The oxygen content is determined based on the target data, and a safety analysis is performed based on the oxygen content to obtain oxygen safety analysis data.

[0082] When determining oxygen content based on target data, the volume fraction of oxygen in the air is calculated to determine the oxygen content (volume concentration). When conducting safety analysis based on oxygen content, the impact of oxygen content on the human body is determined, thus obtaining oxygen safety analysis data. Here, the impact of oxygen content on the human body is determined according to the rules governing the effects of oxygen on the human body. For example: when the oxygen content (volume concentration) is 15%-19.5%, the corresponding person in the space experiences decreased physical strength, difficulty engaging in heavy physical labor, reduced motor coordination, and is prone to coronary heart disease, lung disease, etc.; when the oxygen content (volume concentration) is 12%-14%, the corresponding person in the space experiences heavier breathing, faster breathing rate, faster pulse, further reduced motor coordination, and decreased judgment ability; oxygen... When the oxygen content (volume concentration) is 10%-12%, people in the corresponding space experience increased and rapid breathing, almost lose their ability to judge, and their lips turn purple; when the oxygen content (volume concentration) is 8%-10%, people in the corresponding space experience mental instability, coma, loss of consciousness, vomiting, and a deathly pale complexion; when the oxygen content (volume concentration) is 6%-8%, people in the corresponding space can recover with treatment in 4-5 minutes, but after 6 minutes there is a 50% chance of death, and after 8 minutes there is a 100% chance of death; when the oxygen content (volume concentration) is 4%-6%, people in the corresponding space experience coma, convulsions, slowed breathing, and death within 40 seconds.

[0083] Flammable and explosive gases and harmful gases are screened and extracted from the target data information to obtain the first screening information and the second screening information.

[0084] The first screening information consists of gas sensing and acquisition data for flammable and explosive gases, while the second screening information consists of gas sensing and acquisition data for hazardous gases. When screening and extracting flammable and explosive gases and hazardous gases from the target data, the target data is matched against flammable and explosive gases, and the corresponding gas sensing and acquisition data is extracted based on the matching results to determine the flammable and explosive gas sensing and acquisition data, thus obtaining the first screening information. Simultaneously, the target data is matched against hazardous gases, and the corresponding gas sensing and acquisition data is extracted based on the matching results to determine the flammable and explosive gas sensing and acquisition data, thus obtaining the second screening information.

[0085] Based on the initial screening information and the combustion and explosion phenomena, a first gas analysis was performed to obtain combustion and explosion safety analysis data.

[0086] The first gas analysis refers to the analysis of flammable and explosive gases. Based on the first screening information and the combustion / explosion phenomenon, the first gas analysis involves analyzing the combustion / explosion phenomenon using gas sensing data of the flammable and explosive gases to determine whether the gases have reached their ignition point, thus obtaining combustion analysis data. Simultaneously, it is determined whether the gas sensing data of the flammable and explosive gases has reached their explosion limits, thus obtaining the first explosion analysis data. Furthermore, if the gas sensing data of the flammable and explosive gases has not reached their explosion limits, the difference between the gas sensing data and the lower limit of the explosion limits is obtained, thus obtaining the second explosion analysis data. Here, the combustion / explosion safety analysis data includes both combustion analysis data and explosion analysis data, and the explosion analysis data can be either the first or second explosion analysis data.

[0087] Based on the second screening information and the hazardous phenomena, a second gas analysis is performed to obtain hazardous safety analysis data.

[0088] The second gas analysis involves the analysis of hazardous gases. Based on the second screening information and hazardous phenomena, this analysis combines gas sensing data with critical data on the hazardous gas's impact on humans to determine whether the hazardous gas has reached the critical data threshold, thus obtaining the first analysis data. If the hazardous gas has not reached the critical data threshold, the difference between the gas sensing data and the corresponding critical data threshold is obtained, yielding the second analysis data. Here, the hazardous gas safety analysis data can be either the first or second analysis data. For example, for hydrogen sulfide (H2S), when the concentration exceeds 1000 mg / m³... 3 At that time, it can cause lightning-fast death within seconds; therefore, the critical data for the harmful gas to affect humans is 1000 mg / m³. 3 Harmful gas phosphine (PH3), 10 mg / m³ 3 After 6 hours of exposure, poisoning symptoms will appear in the human body. Therefore, the critical threshold for the harmful gas to affect humans is 10 mg / m³. 3 Methane (CH4), a harmful gas, can cause headaches, dizziness, fatigue, difficulty concentrating, and rapid breathing and heartbeat when its concentration in the air reaches 25%–30%. If not removed in time, it can lead to suffocation and death. Therefore, the critical threshold for this harmful gas to affect humans is 25%.

[0089] Furthermore, when conducting safety analysis based on the gas sensing data of harmful gases and the critical data on the impact of harmful gases on people, the analysis also incorporates personnel operations. This includes: determining the general distribution of harmful gases in confined spaces based on the gas sensing data to obtain harmful gas distribution information; analyzing the range of personnel activity in confined spaces based on personnel operations to obtain a first target space range; further analyzing the effective range of personnel's perception of the gas impact within the first target space range according to personnel height to obtain a second target space range. Here, the effective range of personnel's perception of the gas impact typically refers to a specific spatial area centered on the personnel's head; and based on the second target space range, locking onto the harmful gas target within the harmful gas distribution information. The system uses target distribution data to determine the effective data value of harmful gases. It then analyzes whether the effective data value reaches the critical threshold for the harmful gas to affect humans, thus obtaining the first analytical data for the harmful gas. Here, when determining the effective data value of harmful gases based on the target distribution data, the determination is based on the properties of the harmful gas. If the lower the concentration of the harmful gas, the greater the negative impact on human health, then a smaller value is obtained from the target distribution data and used as the effective data value. Conversely, if the higher the concentration of the harmful gas, the greater the negative impact on human health, then a larger value is obtained from the target distribution data and used as the effective data value.

[0090] The aforementioned implementation of safety analysis of gas sensing and acquisition data clarifies whether safety hazards exist in confined spaces, thereby reducing the possibility of dangerous situations caused by gases within these spaces and ensuring their safety. By parsing multimodal data acquisition information, gas sensing and acquisition data are identified and separated from the multimodal data. This allows gas analysis algorithms to identify and filter target data information from the multimodal data acquisition information. When performing gas detection and safety analysis based on the target data information, separate analyses are performed for oxygen, flammable and explosive gases, and hazardous gases. This avoids interference from irrelevant multimodal data acquisition information during gas detection and safety analysis, reducing the degree of confusion in gas detection and safety analysis and ensuring the accuracy of the initial safety analysis data. During gas detection and safety analysis, analysis is performed sequentially based on oxygen content, combustion and explosion phenomena, and hazardous phenomena, improving the comprehensiveness of gas detection and safety analysis and preventing personnel from facing life-threatening risks due to the gas environment within confined spaces, thus ensuring the safety of confined spaces. When conducting safety analysis based on gas sensing data of harmful gases and critical data on the impact of harmful gases on people, the analysis also incorporates personnel operations. It fully considers the uneven gas distribution in confined spaces to avoid significant errors in the initial analysis data due to uneven distribution. By combining analysis of personnel operations in confined spaces with analysis of their activity range and further analysis of their effective perception range of gas impact within the first target space based on their height, the analysis of confined space operations and personnel height is conducted when assessing whether harmful gases pose a safety hazard. This avoids the false impression of harm caused by harmful gases reaching critical data in remote areas, ensuring the accuracy of the initial analysis data.

[0091] In one embodiment of the present invention, a video analysis algorithm is used to identify effective data information from multimodal data acquisition information, and personnel status identification and security analysis are performed based on the effective data information, including:

[0092] The multimodal data acquisition information is analyzed to obtain the video monitoring data from the multimodal data acquisition information, thus obtaining effective data information.

[0093] Among them, effective data information refers to data information that can be effectively analyzed in video analysis algorithms, that is, video monitoring and collection data.

[0094] Video analytics algorithms are used to perform image recognition and analysis on valid data information to locate people in confined spaces and obtain their status information.

[0095] In the process of using video analysis algorithms to perform image recognition analysis on valid data information, the video analysis algorithm is used to identify people in the image, determine the people appearing in the confined space, and further obtain the status information of the people appearing in the confined space to obtain the personnel status information, which includes: personnel behavior, wearing of protective equipment, etc.

[0096] By combining the safety standards for personnel working in confined spaces with personnel status information, a second safety analysis data is obtained.

[0097] Specifically, when conducting personnel safety analysis based on confined space work safety regulations, the analysis includes two parts: first, whether personnel are wearing protective equipment in accordance with the regulations, yielding the first set of personnel status analysis data; and second, whether personnel behavior constitutes safe operational actions and whether accidents such as falls have occurred, yielding the second set of personnel status analysis data. The second set of safety analysis data comprises both the first and second sets of personnel status analysis data.

[0098] The aforementioned analysis of personnel status clarifies whether personnel are operating in confined spaces in accordance with confined space safety regulations, preventing life-threatening hazards caused by personnel not wearing protective equipment correctly or not performing operations correctly. It also enables the timely detection of accidents such as falls, ensuring that accidents in confined spaces are not detected in time and thus preventing life-threatening hazards. This guarantees the comprehensiveness and accuracy of the second safety analysis data.

[0099] In one embodiment of the present invention, a video analysis algorithm is used to perform image recognition analysis on valid data information, including:

[0100] Analyze the video angles based on the valid data to determine the video acquisition perspective for the video monitoring data.

[0101] Specifically, when performing video angle analysis on valid data information, the spatial position of the image acquisition device corresponding to the video monitoring data is analyzed when it acquires images in a confined space, and the viewing angle information of the image acquisition device in the confined space is determined, thereby obtaining the video acquisition viewing angle of the video monitoring data.

[0102] Based on the video surveillance data, the video acquisition perspectives are sequentially used to perform image recognition scanning analysis on the valid data information to determine whether there are people in the confined space and obtain preliminary identification results.

[0103] Specifically, when performing image recognition scanning analysis on valid data information based on the video acquisition perspectives of the video monitoring data, the image recognition scanning analysis is sorted according to the video acquisition perspectives to determine the image recognition scanning analysis sequence. Here, the image recognition scanning analysis is sorted according to the overlapping and correlation relationships in the video acquisition perspectives. Personnel identification is performed on the video monitoring data according to the image recognition scanning analysis sequence, resulting in multiple personnel identification results. These multiple personnel identification results are then combined with the acquisition perspectives of the video monitoring data for comprehensive analysis to determine the presence of personnel in the confined space, thus obtaining preliminary identification results.

[0104] Further video analysis and processing are performed based on the preliminary identification results. When the preliminary identification results indicate that a person is present in a confined space, the target range of the person in the confined space is locked and the image is processed in the video monitoring data to obtain the target image of the person in the video monitoring.

[0105] If the initial identification result indicates that no personnel are present in the confined space, then there is no need to acquire personnel status information. If the initial identification result indicates that personnel are present in the confined space, the number of personnel present in the confined space is determined by locking the target range and processing the image based on the video monitoring data. If there is only one person present in the confined space, that person is taken as the analysis target. If there are two or more people present in the confined space, each person present in the confined space is taken as an analysis target. Then, for each analysis target, video monitoring data associated with the analysis target within a preset range is acquired from the valid data information, and it is determined whether the source of the video monitoring data is unique. If it is unique, the video monitoring data associated with the analysis target within the preset range is the personnel video monitoring target image. If it is not unique, the spatial perspective of the video monitoring data associated with the analysis target within the preset range is combined to obtain the personnel video monitoring target image.

[0106] Perform personnel status recognition and analysis on target images of personnel video monitoring to obtain personnel status information.

[0107] In the process of further analyzing the personnel status of video monitoring target images, the personnel are identified in accordance with the safety regulations for personnel working in confined spaces. The status of personnel wearing protective devices is obtained, resulting in the first analysis data of personnel status. At the same time, the posture information of personnel is obtained to determine the behavior information of personnel and to determine whether personnel have fallen or other accidents, resulting in the second analysis data of personnel status.

[0108] The aforementioned video image processing of valid data information enables the acquisition of detailed information about personnel in confined spaces from video surveillance data, preventing safety hazards caused by abnormal personnel conditions and ensuring the safety of personnel in confined spaces. Multi-angle image recognition scanning of valid data information enables comprehensive identification and analysis of confined spaces, avoiding obstructions that could prevent personnel from being identified and affecting subsequent safety analysis, thus ensuring the accuracy of the safety analysis. Furthermore, image recognition scanning and analysis are performed sequentially on valid data information based on the video surveillance data acquisition perspectives, preventing omissions in video surveillance data that could affect the initial identification results. It also allows for the confirmation of the same person from different video acquisition perspectives, avoiding personnel identification errors and ensuring the accuracy of the initial identification results. Furthermore, when personnel are present in a confined space, the target range is locked and image processing is performed on the video monitoring data based on the personnel present in the confined space. By using the personnel present in the confined space as the analysis target, target range locking and image processing are performed separately. This not only improves the efficiency of identifying personnel video monitoring target images but also ensures the comprehensiveness of personnel video monitoring target images. It avoids situations where personnel video monitoring target images are too one-sided, resulting in inaccurate personnel status information during personnel status identification and analysis. This ensures the accuracy of personnel status information and provides a guarantee for determining the safety risk level and providing graded early warning.

[0109] In one embodiment of the present invention, when determining the security risk level based on security analysis data, the security analysis data is summarized and organized, and the security risk level is obtained based on the summary and organization results and the level determination rules.

[0110] When summarizing and organizing safety analysis data, the first and second safety analysis data are analyzed according to their timestamps. The first and second safety analysis data obtained from source data at the same time are matched and organized to obtain the summarized and organized data. Based on the summarized and organized results and the level determination rules, a safety risk level is determined. Safety incident analysis is then performed on the summarized and organized data to determine the probability of a safety hazard event occurring. The safety risk level is then determined based on the probability of the safety hazard event occurring and the level determination rules. Here, the level determination rules are pre-defined mapping regulations. For example, when the probability of combustion or explosion in the confined space is high and the harmful gases in the confined space pose a safety hazard to personnel, or when the probability of combustion or explosion in the confined space is high and personnel's own behavior poses a danger, or when the harmful gases in the confined space pose a safety hazard to personnel and personnel's own behavior poses a danger, the safety risk level is Level 1. When any one of the following occurs: a high probability of combustion or explosion in the confined space, a harmful gas in the confined space poses a safety hazard to personnel, or personnel's own behavior poses a danger, the safety risk level is Level 2. When there are no dangers, the safety risk level is Level 3.

[0111] The above method determines the safety risk level based on safety analysis data, thus clarifying the occurrence of safety hazard events in confined spaces. Furthermore, by summarizing and organizing the safety analysis data, the first and second safety analysis data are time-corresponding, avoiding confusion caused by time discrepancies and ensuring the orderly nature of the confined space safety risk classification and early warning method. Moreover, by analyzing safety events based on the summarized and organized data, the probability of each safety hazard event occurring is clarified, thereby enabling clearer classification and early warning.

[0112] In one embodiment of the present invention, different security risk levels correspond to different early warning modes. After the early warning is classified according to the security risk level, the associated data can be viewed based on the security risk level.

[0113] Different warning modes differ in terms of alert frequency, sound, and method. When querying related data based on safety risk levels, the system obtains data query request information and retrieves related data information for the corresponding safety risk level to obtain target related data information. Then, it responds to the data query request information based on this target related data information. Here, related data information refers to the relevant data information that leads to the safety risk level, such as corresponding multimodal data acquisition information or corresponding safety analysis data. When responding to the data query request information for target related data information, if the data query request information has specific requirements, such as viewing only the first safety analysis data, only the second safety analysis data, viewing video monitoring data, or viewing gas sensing data, the system filters the target related data information according to these specific requirements and uses the filtered target data information as the response feedback information for the data query request.

[0114] The above-mentioned approach, which uses different early warning modes corresponding to different safety risk levels, enables relevant personnel to directly understand the severity and urgency of safety hazards in confined spaces through tiered early warnings. Furthermore, by reviewing related data based on safety risk levels, relevant personnel can clearly identify the relevant information that leads to that level of safety risk, thereby providing reference materials for reducing safety hazards and enabling rapid measures to be taken to ensure the safety of confined spaces.

[0115] like Figure 3 As shown in the figure, this embodiment of the invention provides a confined space safety risk classification and early warning system based on multimodal fusion, including: a deployment setting module, a multimodal data acquisition module, a data analysis center, and a classification and early warning module, wherein the deployment setting module, the multimodal data acquisition module, the data analysis center, and the classification and early warning module are sequentially connected.

[0116] The deployment module is used to deploy multimodal data acquisition devices in confined spaces.

[0117] The multimodal data acquisition module is used to perform video monitoring and gas monitoring in confined spaces based on the deployment settings through a multimodal data acquisition device, acquire multimodal data acquisition information, and wirelessly transmit the multimodal data acquisition information to the data analysis center;

[0118] The data analysis center is used to perform personnel status analysis and gas detection analysis on multimodal data collection information using artificial intelligence algorithms to obtain safety analysis data;

[0119] The graded early warning module is used to determine the security risk level based on security analysis data and to issue graded early warnings according to the security risk level.

[0120] The aforementioned confined space safety risk classification and early warning system based on multimodal fusion corresponds to a confined space safety risk classification and early warning method based on multimodal fusion. Through a deployment module, the multimodal data acquisition device is adapted to the confined space, ensuring that the collected multimodal data information can more comprehensively reflect the situation within the confined space. This improves the comprehensiveness of the multimodal data acquisition module, thereby enhancing the comprehensiveness of confined space safety risk analysis. Furthermore, the data analysis center incorporates artificial intelligence algorithms to determine safety analysis data, eliminating the need for subjective analysis and judgment based on human experience. This ensures the accuracy and efficiency of safety analysis data, enabling the classification and early warning module to promptly determine the safety risk level and issue classified warnings. This improves the responsiveness of safety analysis, allowing for timely measures to be taken based on the safety risk level, ensuring the safety of personnel working in the confined space.

[0121] Those skilled in the art should understand that the terms "first" and "second" in this invention merely refer to different application stages.

[0122] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0123] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for graded early warning of safety risks in confined spaces based on multimodal fusion, characterized in that, include: Deployment and setup of multimodal data acquisition devices for confined spaces; Based on the deployment results, video monitoring and gas monitoring are carried out in the confined space using multimodal data acquisition devices to obtain multimodal data acquisition information, and the multimodal data acquisition information is wirelessly transmitted to the data analysis center. At the data analysis center, artificial intelligence algorithms are used to analyze personnel status and gas detection based on multimodal data collection information to obtain safety analysis data; The security risk level is determined based on the security analysis data, and a graded warning is issued according to the security risk level.

2. The confined space safety risk classification and early warning method according to claim 1, characterized in that, The multimodal data acquisition device includes: The first acquisition unit uses various gas acquisition devices to sense gases in the confined space and obtains gas sensing acquisition data. The second acquisition unit acquires images of the confined space using an image acquisition device to obtain video monitoring data. The communication transmission unit includes an internal transmission unit and an external transmission unit. The internal transmission unit is used to transmit and converge gas sensing data and video monitoring data in a confined space to determine multimodal data acquisition information in real time. The external transmission unit is used to wirelessly transmit the multimodal data acquisition information to the data analysis center. The battery power supply unit is used to supply power and detect the power level of the first and second acquisition units.

3. The confined space safety risk classification and early warning method according to claim 1, characterized in that, Before wirelessly transmitting multimodal data acquisition information to the data analysis center, wireless gateway simulation tests and optimizations are performed to determine the target wireless gateway, and then the multimodal data acquisition information is wirelessly transmitted to the data analysis center based on the target wireless gateway.

4. The confined space safety risk classification and early warning method according to claim 1, characterized in that, Artificial intelligence algorithms are used to perform personnel status analysis and gas detection analysis on multimodal data acquisition information, including: A gas analysis algorithm is used to identify and filter target data information from multimodal data acquisition, and gas detection and safety analysis are performed based on the target data information to obtain the first safety analysis data. Video analytics algorithms are used to identify effective data information from multimodal data collection, and personnel status identification and security analysis are performed based on the effective data information to obtain second security analysis data.

5. The confined space safety risk classification and early warning method according to claim 4, characterized in that, Gas analysis algorithms are used to identify and filter target data information from multimodal data acquisition, and gas detection and safety analysis are performed based on the target data information, including: Analyze multimodal data acquisition information, obtain gas sensing data from the multimodal data acquisition information, and obtain target data information; The oxygen content is determined based on the target data information, and a safety analysis is performed based on the oxygen content to obtain oxygen safety analysis data; Flammable and explosive gases and harmful gases are screened and extracted from the target data information to obtain the first screening information and the second screening information. Based on the first screening information and the combustion and explosion phenomena, the first gas analysis is performed to obtain combustion and explosion safety analysis data; Based on the second screening information and the hazardous phenomena, a second gas analysis is performed to obtain hazardous safety analysis data.

6. The confined space safety risk classification and early warning method according to claim 4, characterized in that, Video analytics algorithms are used to identify effective data information from multimodal data collection, and based on this effective data information, personnel status identification and security analysis are performed, including: Analyze the multimodal data acquisition information to obtain the video monitoring data from the multimodal data acquisition information and obtain effective data information; Video analytics algorithms are used to perform image recognition and analysis on valid data information to locate people in confined spaces and obtain their status information. By combining the safety standards for personnel working in confined spaces with personnel status information, a second safety analysis data is obtained.

7. The confined space safety risk classification and early warning method according to claim 6, characterized in that, Video analysis algorithms are used to perform image recognition and analysis on valid data information, including: Analyze the video angles of valid data to determine the video acquisition perspective for video monitoring data collection. Based on the video surveillance data, the video acquisition perspective is used to perform image recognition scanning analysis on the effective data information in sequence to determine whether there are people in the confined space and obtain preliminary recognition results. Further video analysis and processing are carried out based on the preliminary identification results. When the preliminary identification results indicate that a person is present in a confined space, the target range of the person present in the confined space is locked and the image is processed in the video monitoring data to obtain the target image of the person in the video monitoring. Perform personnel status recognition and analysis on target images of personnel video monitoring to obtain personnel status information.

8. The confined space safety risk classification and early warning method according to claim 1, characterized in that, When determining the security risk level based on security analysis data, the security analysis data is summarized and organized, and the security risk level is obtained based on the summary and organization results and the level determination rules.

9. The confined space safety risk classification and early warning method according to claim 8, characterized in that, Different security risk levels correspond to different early warning modes. After issuing early warnings according to the security risk level, related data can be viewed based on the security risk level.

10. A confined space safety risk classification and early warning system based on multimodal fusion, characterized in that, include: The deployment module is used to deploy multimodal data acquisition devices in confined spaces. The multimodal data acquisition module is used to perform video monitoring and gas monitoring in confined spaces based on the deployment settings through a multimodal data acquisition device, acquire multimodal data acquisition information, and wirelessly transmit the multimodal data acquisition information to the data analysis center; The data analysis center is used to perform personnel status analysis and gas detection analysis on multimodal data collection information using artificial intelligence algorithms to obtain safety analysis data; The graded early warning module is used to determine the security risk level based on security analysis data and to issue graded early warnings according to the security risk level.

Images