Chemical leakage early warning system based on multi-modal fusion algorithm

Abstract

The invention provides a chemical leakage early warning system based on a multi-modal fusion algorithm. The chemical leakage early warning system comprises an image monitoring module, an air pressure monitoring module, a test paper monitoring module, a temperature monitoring module, a vehicle driving monitoring module and a leakage risk estimation module. And the leakage risk estimation module evaluates the leakage risk level based on the return data of the image monitoring module, the air pressure monitoring module, the test paper monitoring module, the temperature monitoring module and the vehicle driving monitoring module. The leakage risk is evaluated based on multi-dimensional fusion of the system, and chemical leakage can be found accurately and quickly.

Description

Technical Field

[0001] The present invention relates to a safety assurance device for freight or loading special goods or objects, and in particular to a chemical product leakage early warning system based on a multimodal fusion algorithm. Background Art

[0002] Chemical leaks are a serious and potentially devastating accident during storage and transportation. Their hazards are multifaceted, long-term, and widespread, affecting human health, the ecological environment, and social security. For example, leaked chemicals (such as toxic gases, corrosive liquids, and flammable and explosive substances) can enter the human body through inhalation, skin contact, and ingestion, causing immediate or long-term damage. Some toxic gases can cause respiratory burns and asphyxiation quickly, and at high concentrations, can be fatal within minutes. Corrosive substances (such as sulfuric acid and caustic soda) can cause chemical burns upon contact with skin or mucous membranes, and can cause blindness if they enter the eyes. Some chemicals (such as benzene, formaldehyde, and heavy metals) are cumulatively toxic. Long-term, low-dose exposure can cause cancer (such as benzene-induced leukemia), liver and kidney failure, and neurological damage (such as lead poisoning), with symptoms potentially taking years to manifest. Furthermore, leaked chemicals (such as organic solvents, heavy metals, and pesticides) entering rivers, lakes, or groundwater can degrade water quality and kill fish and aquatic plants. Corrosive or toxic substances seeping into the soil can kill soil microorganisms and damage soil structure, leading to desertification and crop failure. Volatile chemicals (such as pesticides) can leach into the atmosphere and form "acid rain" through precipitation, further damaging vegetation and forests. Plants and animals in the leaking area may die or migrate due to poisoning and habitat destruction, leading to the collapse of local ecosystems. Furthermore, many chemicals (such as gasoline, ethanol, and propane) are flammable and explosive. Leaks, if exposed to ignition sources (open flames, static electricity, and high temperatures), can cause fires and explosions, expanding the scope of harm. The shockwaves and high temperatures generated by explosions can destroy surrounding structures and cause casualties. Fires can also release more toxic fumes (such as carbon monoxide and phosgene), creating a "toxic-fire compound disaster." Furthermore, leaks can lead to the emergency evacuation of nearby residents, school closures, and business shutdowns, triggering panic buying of supplies and the spread of rumors, complicating emergency management.

[0003] Therefore, in general, if the chemical leakage cannot be stopped immediately, the human and economic losses it causes will be immeasurable.

[0004] Therefore, how to effectively assess the risk of chemical leakage inside trucks and how to quickly terminate and stop the leakage are problems that have always been faced and need to be solved in this field. Summary of the Invention

[0005] The present invention aims to provide a comprehensive and effective method for preventing chemical leakage during chemical truck transportation, specifically as follows: The present invention provides a chemical product leakage early warning system based on a multimodal fusion algorithm, which includes an image monitoring module, an air pressure monitoring module, a test paper monitoring module, a temperature monitoring module, a vehicle driving monitoring module, and a leakage risk estimation module; The image monitoring module, through monitoring equipment installed inside the vehicle, monitors in real time whether chemical bottles are tipping over, leaking or leaking. The above-mentioned air pressure monitoring module monitors in real time whether there are any pressure changes in the internal environment of the truck carrying chemical bottles through the pressure sensor installed inside the vehicle; The test strip monitoring module uses chemical test strips installed inside the vehicle to monitor in real time whether chemicals are volatilized due to leakage and the volatilization level; The temperature monitoring module monitors the temperature abnormality of the vehicle's internal environment in real time through temperature sensors installed inside and outside the vehicle; The vehicle driving monitoring module monitors the vehicle's bumping frequency and amplitude in real time through a displacement sensor installed on the vehicle; The above-mentioned leakage risk assessment module assesses the leakage risk level based on the feedback data from the image monitoring module, air pressure monitoring module, test paper monitoring module, temperature monitoring module, and vehicle driving monitoring module.

[0006] Furthermore, the present invention provides a chemical leakage early warning system based on a multimodal fusion algorithm. The above-mentioned monitoring instruments are multiple and are installed on the top surface of the vehicle, and are arranged in multiple matrices based on the size of the vehicle body; The above-mentioned image monitoring module stitches the images sent back by multiple monitoring devices, eliminates the image deformation caused by the monitoring devices themselves, and forms a bird's-eye view image with a nearly flat visual effect.

[0007] Furthermore, the present invention provides a chemical leakage early warning system based on a multimodal fusion algorithm. The chemical product can has an identifier on its upper surface; The image monitoring module identifies changes in the number and shape of identifiers in the image data to determine whether the chemical cans are tilted or overturned; The above identifier is a graphic set horizontally on the bottle cap of the chemical product can; The specific identification method is: SY1. Retrieve the image of each frame or the specified interval frame and extract all identifiable identifiers; SY2. Determine whether the number of identifiers is consistent with the loading quantity. When “consistent”, proceed to SY3; When "inconsistent", it is determined that the chemical bottle has been dumped, and the dumping result is returned to the front end of the equipment; SY3. Determine whether the identifier is deformed. When “no deformation occurs”, return to SY1; When “deformation occurs”, perform SY4; SY4. Determine the tilt level of the chemical bottle based on the graphic deformation model and return the tilt result to the front end of the device.

[0008] Furthermore, the present invention provides a chemical leakage early warning system based on a multimodal fusion algorithm. The above-mentioned image monitoring module also identifies leakage traces in the image data to determine whether leakage has occurred in the chemical cans; The specific identification method is: SX1. Retrieve images of each frame or a specified interval of frames and determine whether there are any suspicious traces based on the trace model; When the result is "no", continue to judge the next image; When the answer is "yes", proceed to SX2; SX2. Compare the suspicious trace with the trace in the previous image to determine whether it is a new trace; When the result is "no", continue to judge the next image; If the result is "yes", SX3 is carried out and a suspicious warning is sent to the front end of the device; SX3. For the suspicious traces, starting with the first image in which the suspicious traces are found, and then going through the N images thereafter, it is determined whether the suspicious traces show signs of expansion; If the result is "No", mark it as suspicious and continue monitoring; When the result is "yes", a leakage alarm is returned to the front end of the device.

[0009] Furthermore, the present invention provides a chemical leakage early warning system based on a multimodal fusion algorithm. The above-mentioned pressure sensors are multiple and installed on the top and side of the vehicle; The above-mentioned air pressure monitoring module monitors the return data of the pressure sensor to determine whether the chemical product has gasified or volatilized beyond a reasonable level; The specific identification method is: based on the chemical product air pressure model, determine whether the current air pressure exceeds the safety value.

[0010] Furthermore, the present invention provides a chemical leakage early warning system based on a multimodal fusion algorithm. The construction method of the above chemical product pressure model is as follows: SS1. Collect pressure data of different chemicals under different V1 / V2 and temperature conditions; SS2. Use computer learning to complete the actual data and form a chemical product pressure model.

[0011] Furthermore, the present invention provides a chemical leakage early warning system based on a multimodal fusion algorithm. A plurality of test paper installation boxes are provided inside the above-mentioned vehicle, and chemical test papers are detachably arranged in the test paper installation boxes.

[0012] Furthermore, the present invention provides a chemical leakage early warning system based on a multimodal fusion algorithm. The specific method for the temperature monitoring module to monitor the temperature abnormality of the vehicle environment in real time is as follows: ST1. Based on the vehicle temperature model, determine whether the interior temperature is within a reasonable range based on the current exterior temperature. When the answer is "yes", continue to monitor; When the answer is "no", proceed to ST2; ST2. Determine whether the current internal temperature is above or below the reasonable range; If the result is "higher than", proceed to ST3.1; If the result is "lower", proceed to ST3.2; ST3.1. Determine whether the chemicals being transported by the vehicle will increase the ambient temperature when exposed to air. When the result is "yes", the possible leaked result is returned to the device front end; If the result is "no", the abnormal temperature inside the vehicle will be reported to the front end of the device; ST3.2. Determine whether the chemicals being transported by the vehicle will lower the ambient temperature when exposed to air. When the result is "yes", the possible leaked result is returned to the device front end; When the result is "no", the abnormal temperature inside the vehicle is reported to the front end of the device.

[0013] Furthermore, the present invention provides a chemical leakage early warning system based on a multimodal fusion algorithm. The above-mentioned vehicle driving monitoring module also judges the road condition level of the road ahead based on AI road condition data.

[0014] Furthermore, the present invention provides a chemical leakage early warning system based on a multimodal fusion algorithm. The leakage risk assessment module scores the data sent back by the image monitoring module, air pressure monitoring module, test paper monitoring module, temperature monitoring module, and vehicle driving monitoring module; The leakage risk factor is calculated according to the following formula: R=k1×MI+k2×AP+k3×TP+k4×TM+k5×BP; Among them, k1, k2, k3, k4, and k5 are weight coefficients; MI is the score based on the data returned by the image monitoring module; AP is the score based on the return data from the air pressure monitoring module; TP is the score based on the return data of the test strip monitoring module; TM is the score based on the data returned by the temperature monitoring module; BP is the score based on the data sent back by the vehicle driving monitoring module; R is used to rate the risk level of the leak. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 , This embodiment provides a schematic diagram of the operation flow of a chemical leakage warning system based on a multimodal fusion algorithm. DETAILED DESCRIPTION

[0016] The present invention is susceptible to various modifications and embodiments, and thus specific embodiments are illustrated and described in the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments, but rather should be understood to encompass all modifications, equivalents, and even substitutes that fall within the spirit and technical scope of the present invention.

[0017] like Figure 1 As shown, this embodiment provides a chemical leakage warning system based on a multimodal fusion algorithm, which includes an image monitoring module, an air pressure monitoring module, a test paper monitoring module, a temperature monitoring module, a vehicle driving monitoring module, and a leakage risk estimation module; The image monitoring module uses monitoring equipment installed inside the vehicle to monitor in real time whether chemical bottles are tipping over, leaking or leaking. Specifically, there are multiple control instruments installed on the top surface of the vehicle, and multiple matrices are set up based on the size of the vehicle body. The purpose of setting up multiple monitoring instruments is to achieve a true restoration of the bird's-eye view of the interior environment of the truck, rather than the distorted visual result of near-large and far-small formed by shooting from a single angle.

[0018] This image monitoring module uses libraries such as OpenCV and JavaCV to first use relevant functions to collect video frames of images sent back by monitoring instruments. It then performs geometric correction, feature extraction and matching on the video frames to determine the overlapping areas between images. Finally, it stitches the matched images into a panoramic picture and displays it. This eliminates image deformation caused by the monitoring instrument itself and forms a bird's-eye view image with a near-flat visual effect.

[0019] The upper surface of the chemical product can is provided with an identifier, which is a graphic horizontally arranged on the bottle cap of the chemical product can, such as a straight line; The image monitoring module determines whether a chemical bottle is tilted or overturned by identifying the number and shape changes of identifiers in the image data. That is, when the bottle is overturned, the identifier of the bottle will not be captured by the camera on the top surface, so it is judged to be overturned. When the bottle is tilted, the identifier of the bottle will be deformed due to the tilt, for example, the straight shape will become shorter.

[0020] The specific identification method is: SY1. Retrieve the image of each frame or the specified interval frame and extract all identifiable identifiers; SY2. Determine whether the number of identifiers is consistent with the loading quantity. When “consistent”, proceed to SY3; When "inconsistent", it is determined that a chemical bottle has been dumped, and after the dumping result is returned to the front end of the equipment, SY3 is performed; SY3. Determine whether the identifier is deformed. When “no deformation occurs”, return to SY1; When “deformation occurs”, perform SY4; SY4. Determine the tilt level of the chemical bottle based on the graphic deformation model and return the tilt result to the front end of the device.

[0021] The image deformation module is constructed by capturing top-down views of the image at different tilt angles, measuring the image reduction ratio at each tilt angle, and plotting a linear graph. Alternatively, a linear graph of L / line and ɑ can be directly drawn using the formula L = line × cosɑ, where line is the length of a line and ɑ is the angle. This linear graph can then be used to determine the current tilt angle when measuring the captured video.

[0022] In addition, this image monitoring module can also identify leakage traces in image data to determine whether chemical leakage occurs due to severe bumps, bottle collision / aging or quality problems of the chemical itself; The specific identification method is: SX1. Retrieve images of each frame or a specified interval of frames and determine whether there are any suspicious traces based on the trace model; When the result is "no", continue to judge the next image; When the answer is "yes", proceed to SX2; The method of constructing and using the trace model is to collect images of various bottle leakages and determine whether there is leakage in the current image through machine recognition; SX2. Compare the suspicious marks with those in the previous image to determine whether they are new. This step aims to exclude marks on the original vehicle that cannot be cleaned and avoid misidentification. When the result is "no", continue to judge the next image; If the result is "yes", SX3 is carried out and a suspicious warning is sent to the front end of the device; SX3. For the suspicious trace, starting with the first image where the suspicious trace is found, and then going through the N images thereafter, it is determined whether the suspicious trace has any signs of expansion, that is, after locking the suspicious trace, the trace is extracted based on traditional image processing methods and its area is calculated; If the result is "No", mark it as suspicious and continue monitoring; When the result is "yes", a leakage alarm is returned to the front end of the device.

[0023] The air pressure monitoring module uses a pressure sensor installed inside the vehicle to monitor in real time whether there are any pressure changes in the internal environment of the truck carrying chemical bottles; The pressure sensors are multiple and are installed on the top and side of the vehicle; The air pressure monitoring module monitors the data returned by the pressure sensor to determine whether the chemical is vaporizing or volatilizing beyond a reasonable level. It is important to note that the returned data is cleaned to remove irrelevant instrument changes or those caused by vehicle vibrations. Specifically, this cleaning method eliminates sudden, disordered data during stable conditions, such as sudden increases or decreases. This is because the increase or decrease in internal pressure is not necessarily a sudden change; it follows a linear process.

[0024] The specific identification method for valid data is: based on the chemical product air pressure model, determine whether the current air pressure exceeds the safety value.

[0025] Here, the so-called chemical product pressure model is constructed as follows: the pressure conditions of different chemicals, different seepage volumes (i.e., V1 / V2, V1 is the seepage volume, V2 is the warehouse volume) under different warehouse temperature conditions are collected, and the actual data is supplemented by computer learning to form a chemical product pressure model and clarify the safety limit pressure of the seepage volume.

[0026] And because the pressure sensors are set at different positions on the vehicle body, the main leakage position can be determined by the first sensor where the pressure changes. This solution is particularly suitable for the compartmentalized storage state.

[0027] The test strip monitoring module uses chemical test strips installed inside the vehicle to monitor in real time whether there is any chemical volatilization due to leakage and the volatility level. The chemical test strips are detachably set in the test strip installation box. The chemical test strips can be pH test strips, starch-potassium iodide test strips, acid potassium permanganate test strips / fuchsin test strips, ion-specific test strips, and other commercially available general-purpose chemical test strips for detecting chemicals.

[0028] The temperature monitoring module monitors the temperature abnormalities in the vehicle environment in real time through temperature sensors installed inside and outside the vehicle; The specific method for the temperature monitoring module to monitor the temperature abnormality of the vehicle environment in real time is as follows: ST1. Determine whether the interior temperature is within a reasonable range based on the current exterior temperature, using the vehicle temperature model. The vehicle temperature model collects the interior and exterior temperatures of the truck at different exterior temperatures and generates a model. When the answer is "yes", continue to monitor; When the answer is "no", proceed to ST2; ST2. Determine whether the current internal temperature is above or below the reasonable range; If the result is "higher than", proceed to ST3.1; If the result is "lower", proceed to ST3.2; ST3.1. Determine whether the chemicals being transported by the vehicle will increase the ambient temperature when exposed to air. When the result is "yes", the possible leaked result is returned to the device front end; If the result is "no", the vehicle's internal temperature abnormalities will be reported to the front end of the equipment, which means that these abnormalities may be caused by failure of the vehicle's sealing or insulation layer; ST3.2. Determine whether the chemicals being transported by the vehicle will lower the ambient temperature when exposed to air. When the result is "yes", the possible leaked result is returned to the device front end; When the result is "no", the abnormal temperature inside the vehicle will be reported to the front end of the equipment, which means that these abnormalities may be caused by failure of the vehicle's sealing or insulation layer.

[0029] The vehicle driving monitoring module monitors the frequency and amplitude of vehicle bumps in real time through displacement sensors installed on the vehicle. In addition, the module also determines the road condition level of the road ahead based on AI road condition data, that is, whether there is road maintenance that will cause a lot of bumps on the road ahead, whether an accident has occurred and requires frequent lane changes or detours through turns, etc.

[0030] The leakage risk assessment module assesses the leakage risk level based on the feedback data from the image monitoring module, air pressure monitoring module, test paper monitoring module, temperature monitoring module, and vehicle driving monitoring module; The specific method is as follows: the leakage risk assessment module scores the data based on the feedback data from the image monitoring module, air pressure monitoring module, test paper monitoring module, temperature monitoring module, and vehicle driving monitoring module; The leakage risk factor is calculated according to the following formula: R=k1×MI+k2×AP+k3×TP+k4×TM+k5×BP; Among them, k1, k2, k3, k4, and k5 are weight coefficients; MI is the score based on the data returned by the image monitoring module; AP is the score based on the return data from the air pressure monitoring module; TP is the score based on the return data of the test strip monitoring module; TM is the score based on the data returned by the temperature monitoring module; BP is the score based on the data sent back by the vehicle driving monitoring module; R is used to rate the risk level of the leak.

[0031] While the above description focuses on the embodiments, this is merely illustrative and does not limit the present invention. Persons skilled in the art will readily appreciate that various modifications and applications not illustrated above are possible without departing from the essential characteristics of the embodiments. For example, the various components specifically illustrated in the embodiments may be implemented with modifications. Furthermore, any differences associated with such modifications and applications should be construed as being within the scope of the present invention as defined in the appended claims.

Claims

1. A chemical leakage early warning system based on a multimodal fusion algorithm, characterized by: It includes image monitoring module, air pressure monitoring module, test paper monitoring module, temperature monitoring module, vehicle driving monitoring module, and leakage risk estimation module; The image monitoring module monitors in real time whether the chemical bottles are tipping over, leaking or leaking through monitoring equipment installed inside the vehicle; The air pressure monitoring module monitors in real time whether there are any pressure changes in the internal environment of the truck carrying the chemical bottles through the pressure sensor installed inside the vehicle; The test paper monitoring module uses chemical test paper installed inside the vehicle to monitor in real time whether there is any chemical volatilization due to leakage and the volatility level; The temperature monitoring module monitors the temperature abnormality of the vehicle environment in real time through temperature sensors installed inside and outside the vehicle; The vehicle driving monitoring module monitors the vehicle's bumping frequency and amplitude in real time through a displacement sensor installed on the vehicle; The leakage risk estimation module assesses the leakage risk level based on the return data of the image monitoring module, the air pressure monitoring module, the test paper monitoring module, the temperature monitoring module, and the vehicle driving monitoring module.

2. The chemical leakage early warning system based on a multimodal fusion algorithm according to claim 1, characterized in that: The monitoring instruments are multiple and are installed on the top surface of the vehicle, and are arranged in a matrix based on the size of the vehicle body; The image monitoring module stitches the images sent back by multiple monitoring devices, eliminates image deformation caused by the monitoring devices themselves, and forms a bird's-eye view image with a nearly flat visual effect.

3. The chemical leakage early warning system based on a multimodal fusion algorithm according to claim 2, characterized in that: The upper surface of the chemical product can is provided with an identifier; The image monitoring module determines whether the chemical product bottle is tilted or overturned by identifying the number and shape changes of the identifiers in the image data; The identifier is a graphic arranged horizontally on the bottle cap of the chemical product can; The specific identification method is: SY1. Retrieve the image of each frame or the specified interval frame and extract all identifiable identifiers; SY2. Determine whether the number of identifiers is consistent with the loading quantity. When "consistent", proceed to SY3; When "inconsistent", it is determined that a chemical bottle has been dumped, and after the dumping result is returned to the front end of the equipment, SY3 is performed; SY3. Determine whether the identifier is deformed. When "no deformation occurs", return to SY1; When "deformation occurs", perform SY4; SY4. Determine the tilt level of the chemical bottle based on the graphic deformation model and return the tilt result to the front end of the device.

4. The chemical leakage early warning system based on a multimodal fusion algorithm according to claim 2, characterized in that: The image monitoring module also identifies leakage traces in the image data to determine whether leakage has occurred in the chemical cans; The specific identification method is: SX1. Retrieve images of each frame or a specified interval of frames and determine whether there are any suspicious traces based on the trace model; When the result is "no", continue to judge the next image; When the result is "yes", proceed to SX2; SX2. Compare the suspicious trace with the trace in the previous image to determine whether it is a new trace; When the result is "no", continue to judge the next image; If the result is "yes", SX3 is carried out and a suspicious warning is sent to the front end of the device; SX3. For the suspicious traces, starting with the first image in which the suspicious traces are found, and then going through the N images thereafter, it is determined whether the suspicious traces show signs of expansion; If the result is "No", mark it as suspicious and continue monitoring; When the result is "yes", a leakage alarm is returned to the device front end.

5. The chemical leakage early warning system based on a multimodal fusion algorithm according to claim 1, characterized in that: There are multiple pressure sensors installed on the top and side of the vehicle; The air pressure monitoring module monitors the data sent back by the pressure sensor to determine whether the chemical product has been gasified or volatilized beyond a reasonable level; The specific identification method is: based on the chemical product air pressure model, determine whether the current air pressure exceeds the safety value.

6. The chemical leakage early warning system based on a multimodal fusion algorithm according to claim 5, characterized in that: The method for constructing the chemical product pressure model is as follows: SS1. Collect pressure data of different chemicals under different V1 / V2 and temperature conditions; SS2. Use computer learning to complete the actual data and form a chemical product pressure model.

7. The chemical leakage early warning system based on a multimodal fusion algorithm according to claim 1, characterized in that: A plurality of test paper installation boxes are provided inside the vehicle, and chemical test papers are detachably arranged in the test paper installation boxes.

8. The chemical leakage early warning system based on a multimodal fusion algorithm according to claim 1, characterized in that: The specific method for the temperature monitoring module to monitor the temperature abnormality of the vehicle environment in real time is: ST1. Based on the vehicle temperature model, determine whether the interior temperature is within a reasonable range based on the current exterior temperature. When the result is "yes", continue monitoring; When the result is "no", proceed to ST2; ST2. Determine whether the current internal temperature is above or below the reasonable range; If the result is "higher than", proceed to ST3.1; If the result is "lower", proceed to ST3.2; ST3.

1. Determine whether the chemicals being transported by the vehicle will increase the ambient temperature when exposed to air. When the result is "yes", the possible leaked result is returned to the device front end; If the result is "No", the abnormal temperature inside the vehicle will be reported to the front end of the device; ST3.

2. Determine whether the chemicals being transported by the vehicle will lower the ambient temperature when exposed to air. When the result is "yes", the possible leaked result is returned to the device front end; When the result is "No", the abnormal temperature inside the vehicle will be reported to the front end of the device.

9. The chemical leakage early warning system based on a multimodal fusion algorithm according to claim 1, characterized in that: The vehicle driving monitoring module also determines the road condition level of the road ahead based on AI road condition data.

10. The chemical leakage early warning system based on a multimodal fusion algorithm according to claim 1, characterized in that: The leakage risk estimation module scores the data based on the feedback data from the image monitoring module, air pressure monitoring module, test paper monitoring module, temperature monitoring module, and vehicle driving monitoring module; The leakage risk factor is calculated according to the following formula: R=k1×MI+k2×AP+k3×TP+k4×TM+k5×BP; Among them, k1, k2, k3, k4, and k5 are weight coefficients; MI is the score based on the data returned by the image monitoring module; AP is the score based on the return data from the air pressure monitoring module; TP is the score based on the return data of the test strip monitoring module; TM is the score based on the data returned by the temperature monitoring module; BP is the score based on the data sent back by the vehicle driving monitoring module; R is used to rate the risk level of the leak.

Images