Gas leakage accurate early warning method and system based on digital twinning

Abstract

The invention relates to the technical field of digital twinning, and discloses a gas leakage accurate early warning method and system based on digital twinning. The method comprises the following steps: generating a digital twinborn model of a target area based on three-dimensional point cloud data of the target area; a first spatial position of each target object in the target area is acquired, each target object comprises a plurality of target parts, and a second spatial position of each target part is identified; judging whether gas leakage occurs in a surrounding preset range, if so, acquiring shape information of the gas, identifying a gas leakage source based on the shape information, determining a third spatial position of the gas leakage source, and mapping the first spatial position, the second spatial position and the third spatial position into a digital twin model; and establishing a gas diffusion model, simulating a diffusion path of gas leakage based on the environmental data of the target area, and superposing the diffusion path to the digital twinborn model to form a visual diffusion path, so that the accuracy of gas leakage source position identification can be improved.

Description

Technical Field

[0001] The present application relates to the field of digital twin technology, and in particular to a gas leakage accurate early warning method and system based on digital twin. Background Art

[0002] In the field of industrial safety and production, rapid location and accurate early warning of gas leaks are core requirements for preventing major accidents. However, existing technology systems have significant flaws and shortcomings in practical applications, making it difficult to meet the complex industrial requirements of "rapid detection, precise location, and real-time response" for leak sources.

[0003] Existing Chinese patent applications, such as those published in CN117197035A, disclose an industrial gas leak detection method based on infrared thermal imaging and deep learning models. The method includes the following steps: Step 1: Using an infrared thermal imager to photograph and image industrial gas leaking equipment; Step 2: Annotating the gas regions in the captured infrared gas images with detection boxes and creating an infrared image dataset for industrial equipment gas leaks; Step 3: Constructing a lightweight deep learning model for industrial equipment gas leak detection; Step 4: Training the lightweight deep learning model for industrial equipment gas leak detection; Step 5: Obtaining the trained model parameter configuration file and deploying it to the device for real-time, multi-scenario industrial equipment gas leak detection. However, the above-mentioned existing technologies rely on infrared thermal imaging to detect gas regions, while external imaging technology itself can only provide surface leakage area information and cannot accurately locate the specific leak source.

[0004] Another example is a Chinese patent application with publication number CN111639430A, which discloses a digital twin-driven natural gas pipeline leak identification system. The system is a natural gas pipeline system and includes a digital twin model construction module for constructing a digital twin model of the natural gas pipeline; a digital twin model online update module for adaptively updating the data in the physical information space, the structure and parameters of the digital twin model online; a pipeline leak identification module for acquiring sample data and performing preprocessing to determine whether the residual between the predicted value of the digital twin model and the actual value of the pipeline output in real-time data exceeds a residual threshold, based on which to determine whether the real-time data indicates a leak condition; and a visualization module for displaying the pipeline physical information space data and the pipeline digital twin output data in real time. When a pipeline leak is identified, an alarm is issued and an emergency rescue plan is configured accordingly. However, the digital twin models in the above-mentioned prior art can simulate the overall structure and data flow of the pipeline. However, in actual applications, especially for small cracks and local leaks, the digital twins may not provide sufficiently detailed local leak information, resulting in the inability to timely discover the leak source, and the location accuracy of the leak source is low.

[0005] Therefore, this application proposes a gas leakage accurate early warning method and system based on digital twin. Summary of the Invention

[0006] In order to solve the above technical problems, the present application provides a gas leakage accurate early warning method and system based on digital twin, which is used to improve the accuracy of gas leakage source positioning.

[0007] In a first aspect, the present application provides a gas leakage accurate early warning method based on digital twins, the method comprising: Step S1: Acquire reference data of a target area, the reference data including three-dimensional point cloud data, regional images of multiple target objects in the target area, and environmental data, and generate a digital twin model of the target area based on the three-dimensional point cloud data; Step S2: obtaining a first spatial position of each target object in the target area, where each target object includes a plurality of target components, identifying a second spatial position of each target component based on the reference data, and mapping the first spatial position and the second spatial position to the digital twin model; Step S3: Determine whether gas leakage occurs within a preset range around the first spatial position and the second spatial position. If so, obtain shape information of the gas, identify the gas leakage source based on the shape information, determine a third spatial position of the leakage source, and map the third spatial position to the digital twin model; Step S4: Establish a gas diffusion model, simulate the diffusion path of the gas leakage based on the environmental data of the target area and the gas diffusion model, and superimpose it into the digital twin model to form a visualized diffusion path.

[0008] In combination with the first aspect, in a first implementation of the first aspect of the present application, obtaining a first spatial position of each target object in the target area includes: Extracting multiple first cross-sections of multiple target objects from the three-dimensional point cloud data, obtaining a center point of each first cross-section, calculating the position coordinates of the center point, the radius distance of the first cross-section, and the normal vector, taking the center point as the starting point, extending along the normal vector to obtain a first reference line of each first cross-section, determining whether a distance between the first reference lines corresponding to two adjacent first cross-sections is less than a first threshold, and whether an angle between the two first reference lines is less than a second threshold, and if so, connecting the center points corresponding to the two first reference lines to obtain a target straight line, according to this step, integrating the first cross-sections corresponding to the center points located on the same target straight line into the same target object, and defining the two first cross-sections located at the endpoints of the same target object as reference cross-sections; The center points corresponding to the two reference sections are defined as the endpoint coordinates of the target object, and the angle between the line segment generated by the two endpoint coordinates and the preset reference coordinate axis and the average value of the radius distance of the two reference sections are defined as the azimuth and radius value of the target object respectively.

[0009] In combination with the first aspect, in a second implementation of the first aspect of the present application, obtaining the center point of each first cross section includes: A representative point is set on each first section, and the point cloud data within a preset range around the representative point is defined as a node set. Two point clouds are selected from the node set as first candidate nodes of the first section, and the inverse normals of the two first candidate nodes are obtained. The vertical line segment between the two inverse normals is connected, and the center point of the vertical line segment is used as the center point of the first section.

[0010] In combination with the first aspect, in a third implementation of the first aspect of the present application, identifying the second spatial position of each target component based on the reference data includes: Establish a machine learning model, obtain a first historical image containing a target component, mark the target component type and plane coordinates in the first historical image, and input them into the machine learning model for learning. The machine learning model identifies multiple component position coordinates of the target component in the area image, projects the component position coordinates onto the three-dimensional point cloud data, obtains the device coordinate position of the camera that takes the area image, and generates multiple second reference lines from the device coordinate position. The second reference lines pass through the component position coordinates of the target component, and the point closest to each second reference line in the three-dimensional point cloud data is defined as a second candidate node. The center point of multiple second candidate nodes is defined as the second spatial position of the target component.

[0011] In combination with the first aspect, in a fourth implementation of the first aspect of the present application, determining whether a gas leakage occurs includes: Acquire a target image within a preset range around the first spatial position and the second spatial position, the target image including a first category image and a second category image, establish a first detection model based on a deep learning model, acquire a second historical image containing multiple categories of gas, input the second historical image into the first detection model for training, and input the target image into the trained first detection model, the first detection model outputs a first detection result of gas leakage, if no gas leakage is detected, perform frequency conversion on the second category image to obtain frequency information, input the frequency information into the second detection model, the second detection model is established based on a statistical model, identifies whether there is frequency information related to gas in the frequency information, the second detection model outputs a second detection result, if the second detection result is that there is no gas leakage, then inputs the frequency information into a third detection model, the third detection model is established based on a support vector machine model, and outputs a third detection result.

[0012] In combination with the first aspect, in a fifth implementation of the first aspect of the present application, obtaining shape information of the gas includes: The target image in which gas is detected is defined as a reference image, multiple frames of the reference image are obtained within a preset window, a moving average of the gas in the multiple frames of the reference image is calculated, the reference image is smoothed based on the moving average to obtain a background model, and whenever a new reference image is obtained, the shape information of the gas is extracted from the new reference image based on the background model.

[0013] In combination with the first aspect, in a sixth implementation of the first aspect of the present application, identifying the gas leakage source based on the shape information includes: Obtain shape information of the gas in multiple consecutive frames of reference images from the first frame reference image to the current frame reference image, identify contour information of the gas based on the shape information, determine the position center point of the gas in each frame reference image, calculate the motion vector of the gas in adjacent reference images based on the position center point, add the motion vectors corresponding to all consecutive frames to obtain a sum vector, use the starting point of the sum vector as the gas leakage source, the direction of the sum vector represents the flow direction of the leakage source, and the magnitude of the sum vector is used as the flow velocity of the leakage source.

[0014] In combination with the first aspect, in a seventh implementation of the first aspect of the present application, determining the third spatial position of the leakage source includes: Obtain device information of a camera that captures the reference image, the device information including a shooting position, angle, and orientation; obtain a first distance from a leakage source to the camera that captures the image; convert the three-dimensional coordinates of the leakage source into first coordinates based on a motion vector of the leakage source, the first distance, and the device information of the camera; the first coordinates include a radius, an azimuth, and a pitch angle of the leakage source; and convert the first coordinates into third spatial coordinates in a Cartesian coordinate system and perform correction.

[0015] In conjunction with the first aspect, in an eighth implementation of the first aspect of the present application, simulating the diffusion path of the gas leakage includes: A gas diffusion model is established based on fluid dynamics, and the environmental data of the target area and the third spatial position of the leakage source are input into the gas diffusion model. The gas diffusion model calculates the diffusion speed and diffusion direction of the gas based on the gas characteristics and the environmental data, and calculates the diffusion range of the gas leakage at different time points to form a diffusion path.

[0016] In a second aspect, the present application provides a digital twin-based gas leakage accurate early warning system, the system comprising: Establish a module for obtaining reference data of a target area, the reference data including three-dimensional point cloud data, regional images of multiple target objects in the target area, and environmental data, and generating a digital twin model of the target area based on the three-dimensional point cloud data; a position determination module, configured to obtain a first spatial position of each target object in the target area, each target object including a plurality of target components, identify a second spatial position of each target component based on the reference data, and map the first spatial position and the second spatial position to the digital twin model; an analysis module, configured to determine whether a gas leak occurs within a preset range around the first spatial position and the second spatial position; if so, obtain shape information of the gas, identify a gas leakage source based on the shape information, determine a third spatial position of the leakage source, and map the third spatial position to the digital twin model; The simulation module is used to establish a gas diffusion model, simulate the diffusion path of the gas leakage based on the environmental data of the target area and the gas diffusion model, and superimpose it on the digital twin model to form a visual diffusion path.

[0017] Compared with the prior art, the beneficial effects of the present invention are at least as follows: This application generates a digital twin model by acquiring three-dimensional point cloud data and regional images of the target area, which can provide a solid foundation for the precise positioning of the gas leakage source; uses a machine learning model to identify the spatial position of the target component and maps it to the three-dimensional model, thereby realizing real-time monitoring and tracking of the gas leakage source; this application realizes efficient detection and prediction of gas leaks through deep learning, statistical models, fluid dynamics and other technologies; through methods such as frequency conversion and support vector machines, the model further improves the accuracy of detection and reduces the risk of false alarms and missed alarms; on this basis, a gas diffusion model is established, combined with environmental data, which can simulate the diffusion path of gas leakage, predict the impact range of gas leakage, help quickly judge the danger of leakage and conduct emergency response. This application not only enhances the accuracy and reliability of gas leak detection, but also provides real-time decision support and improves overall safety through multi-dimensional fusion analysis and simulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 This is a schematic diagram of an embodiment of a gas leakage accurate early warning method based on digital twins in an embodiment of the present application; Figure 2 is a schematic diagram of multiple first cross sections in an embodiment of the present application; Figure 3 A first cross-sectional schematic diagram of a target object in an embodiment of the present application; Figure 4 is a schematic diagram of a second spatial position in an embodiment of the present application; Figure 5 This is a schematic diagram of an embodiment of a gas leakage accurate early warning system based on digital twins in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The embodiments of the present application provide a method and system for accurate early warning of gas leaks based on digital twins. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products or devices.

[0021] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In the embodiments of the present application, an embodiment of a gas leakage accurate early warning method based on digital twins includes: Step S1: Obtain reference data of the target area, where the reference data includes three-dimensional point cloud data, regional images of multiple target objects in the target area, and environmental data, and generate a digital twin model of the target area based on the three-dimensional point cloud data.

[0022] Specifically, reference data is acquired from a target area, such as a factory or an industrial site. This data includes 3D point cloud data of the factory, regional images of multiple target objects within the factory, and environmental data. A laser scanner is used to comprehensively scan the factory interior to obtain 3D point cloud data. Electro-optical cameras, optical gas imaging cameras, and other devices are used to obtain regional images of multiple target objects within the factory, such as pipelines, equipment, tanks, and valves. Various environmental sensors, such as gas sensors, temperature and humidity sensors, pressure sensors, and wind speed sensors, are deployed within the factory to collect environmental data in real time. Modeling software such as Blender is used to convert the 3D point cloud data into a digital twin model. The digital twin model accurately reflects the 3D position of all equipment, pipelines, and structures within the factory, providing basic data support for subsequent monitoring, management, and maintenance.

[0023] Step S2: Obtain the first spatial position of each target object in the target area, where each target object includes multiple target components. Identify the second spatial position of each target component based on the reference data, and map the first spatial position and the second spatial position to the digital twin model.

[0024] Specifically, the first spatial position of each target object is obtained based on the three-dimensional point cloud data. The first spatial position includes the endpoint coordinates, azimuth angle, and radius value of the target object. The target component is the subassembly that makes up the target object, such as the joints and valves of the pipe. The second spatial position is the precise coordinate in the three-dimensional space. The recognition process will be explained later. The first spatial position of the target object is mapped to the corresponding part of the digital twin model, and the geometric shape of the target object is formed by connecting. The target component on the target object is marked based on the second spatial position and connected to the three-dimensional coordinates of the target object to display the complete structure of the target object. By obtaining the spatial position of the target object and its components, the accurate position of each component in the digital twin model is ensured, which facilitates subsequent simulation, monitoring, analysis and other operations.

[0025] Step S3: Determine whether gas leakage occurs within a preset range around the first spatial position and the second spatial position. If yes, obtain the shape information of the gas, identify the leakage source of the gas based on the shape information, determine the third spatial position of the leakage source, and map the third spatial position to the digital twin model.

[0026] Specifically, with the first spatial position and the second spatial position as the center, a preset range is set, and the target object and target component are photographed within the preset range to obtain a target image. The preset range is a location where the probability of gas leakage is higher. Whether a gas leakage occurs is judged based on the target image. The specific judgment method will be explained later. Once a gas leakage is confirmed, the shape information of the gas is obtained based on the target image, the source of the gas leakage is located based on the gas shape information, and the third spatial position of the leakage source is determined. The third spatial position is the three-dimensional position information of the gas leakage source relative to the panoramic image. The third spatial position of the leakage source is mapped to the digital twin model. In the digital twin model, the leakage source will be displayed as a graphic mark, for example, the position of the leakage source is marked in red. Through gas shape information and image analysis, the source of the gas leakage can be accurately determined, further improving the accuracy and response speed of gas leakage detection.

[0027] Step S4: Establish a gas diffusion model, simulate the diffusion path of the gas leakage based on the environmental data of the target area and the gas diffusion model, and superimpose it on the digital twin model to form a visual diffusion path.

[0028] Specifically, environmental data within the factory is obtained, such as wind speed and direction, gas density, temperature, and humidity. The speed and direction of gas diffusion are affected by wind speed and direction. Higher wind speeds accelerate gas diffusion and change the direction of gas diffusion, while higher temperatures accelerate the movement of gas molecules and change their diffusion pattern. The gas diffusion model is based on a CFD (computational fluid dynamics) model and is used to predict the process of gas diffusion starting from the leak source. The CFD model takes into account the dynamic characteristics of airflow and the influence of environmental factors, simulates the diffusion path of gas starting from the leak source, and superimposes the diffusion path into the digital twin model. The digital twin model displays the leak source and diffusion path, and uses color gradients, such as red for high concentration and blue for low concentration, to indicate areas of different concentrations. Through visual display, operators can intuitively see the trend of gas diffusion and the scope of impact, providing real-time gas leakage dynamics, and quickly make early warnings and adjustment decisions.

[0029] In a specific embodiment, obtaining the first spatial position of each target object in the target area specifically includes the following steps: Extract multiple first sections of multiple target objects from the three-dimensional point cloud data, obtain the center point of each first section, calculate the position coordinates of the center point, the radius distance of the first section, and the normal vector, take the center point as the starting point, and extend along the normal vector to obtain a first reference line of each first section, and determine whether the distance between the first reference lines corresponding to two adjacent first sections is less than a first threshold, and whether the angle between the two first reference lines is less than a second threshold. If so, connect the center points corresponding to the two first reference lines to obtain a target straight line. According to this step, the first sections corresponding to the center points located on the same target straight line are integrated into the same target object, and the two first sections located at the endpoints of the same target object are defined as reference sections; The center points corresponding to the two reference sections are defined as the endpoint coordinates of the target object, and the angle between the line segment generated by the two endpoint coordinates and the preset reference coordinate axis and the average value of the radius distance of the two reference sections are defined as the azimuth and radius value of the target object respectively.

[0030] Specifically, if Figure 2As shown, it is a schematic diagram of multiple first sections, where P1, P2, P3 and P4 are four first sections of the pipeline. The first section refers to the cross section of a target object such as a pipeline. For each first section, the center points Q1, Q2, Q3 and Q4 are calculated. The specific calculation method is described later. The coordinates of each midpoint and the radius r1, r2, r3 and r4 of the first section and the normal vector are obtained. The first reference lines w1, w2, w3 and w4 of the first sections P1, P2, P3 and P4 are obtained by extending along the normal vector through the center point as the starting point. It is determined whether the distance between the first reference lines w1 and w2 corresponding to two adjacent first sections, such as P1 and P2, is less than a first threshold, and whether the angle between the two first reference lines is less than a second threshold. The first threshold and the second The thresholds are determined based on actual conditions. If the distance is less than the first threshold and the angle is less than the second threshold, the two sections are considered to be along similar directions and belong to the same pipeline. The center points of the first sections corresponding to the two first reference lines are connected to form a target line. The areas corresponding to multiple first sections along the same target line are integrated into the same pipeline object. In the integrated target object, the sections at the two endpoints are defined as reference sections, such as P1 and P4. The endpoint coordinates of the target object are determined by the center point coordinates Q1 and Q4 of the two reference sections. The azimuth of the target object is calculated based on the angle between the line segment generated from the two endpoint coordinates and a preset reference coordinate axis, such as the north direction of the factory. The average of the radii r1 and r4 of the two reference sections is used as the radius of the target object. By accurately calculating the first and second sections of each target object, the endpoint coordinates, azimuth, and radius of the target object are accurately obtained, thereby accurately positioning the target object in space.

[0031] In a specific embodiment, obtaining the center point of each first cross section specifically includes the following steps: A representative point is set on each first section, and the point cloud data within a preset range around the representative point is defined as a node set. Two point clouds are selected in the node set as the first candidate nodes of the first section, and the inverse normals of the two first candidate nodes are obtained. The vertical line segment between the two inverse normals is connected, and the midpoint of the vertical line segment is used as the center point of the first section.

[0032] Specifically, if Figure 3As shown, it is a schematic diagram of the first section of the target object, wherein P1 represents the first section, A1 is a representative point, and a preset range is set around A1 from C1. Multiple point cloud data within C1 form a node set, and two point cloud data are selected from the node combination, preferably points closer to the target object such as the edge contour of the pipeline as the first candidate nodes, such as A2 and A3, and the inverse normals L1 and L2 of the two first candidate nodes are obtained. The inverse normal is the opposite direction of the normal of the two candidate nodes, that is, the normal pointing to the inside of the pipeline, and the vertical line segment L3 between the two inverse normals and the midpoint Q of L3 are obtained as the center point of the first section P1. By selecting multiple representative points and node sets, the present application makes the center point generated in complex or curved pipelines more accurate, avoids the influence of noise, missing data or incomplete data when directly extracting the center point of the first section from the point cloud data, and improves the stability and reliability of the calculation results.

[0033] In a specific embodiment, identifying the second spatial position of each target component based on the reference data specifically includes the following steps: A machine learning model is established to obtain a first historical image containing a target component, the target component type and plane coordinates in the first historical image are marked, and the result is input into the machine learning model for learning. The machine learning model identifies multiple component position coordinates of the target component in the regional image, projects the component position coordinates onto the three-dimensional point cloud data, obtains the device coordinate position of the camera that shoots the regional image, generates multiple second reference lines based on the device coordinate position, and the second reference lines pass through the component position coordinates of the target component. The point closest to each second reference line in the three-dimensional point cloud data is defined as a second candidate node, and the center point of the multiple second candidate nodes is defined as the second spatial position of the target component.

[0034] Specifically, a machine learning model is established based on a convolutional neural network, and a first historical image containing a target component is obtained as training data for the machine learning model. The first historical image contains image data of the target component at different angles and conditions. These images are marked with the type and plane coordinates of the target component. The model learns these image features, identifies the target component in the regional image, and automatically outputs the two-dimensional coordinates of the target component. The two-dimensional coordinates obtained from the machine learning model are projected onto the three-dimensional point cloud data through the camera model and the known camera position, such as Figure 4The figure shows a schematic diagram of the second spatial position. The device coordinate position M of the camera is obtained. Starting from M, multiple second reference lines L3, L4, and L5 are generated. Each reference line passes through the component position coordinates of the target component. For example, the second reference line L3 passes through a coordinate k1 of the target component, the second reference line L4 passes through the coordinates k2 and k3 of the target component, and the second reference line L4 passes through a coordinate k4 of the target component. The point closest to each second reference line is selected from the point cloud data corresponding to point q in the three-dimensional point cloud data and defined as the second candidate node, such as v1, v2, v3, and v4. The center point B of v1, v2, v3, and v4 is defined as the second spatial position of the target component. By establishing a machine learning model to identify the target component in the image and combining it with the three-dimensional point cloud data to accurately locate the target component, the system can automatically calculate the second spatial position of the target component, which not only improves positioning accuracy but also enhances automation, supports positioning in complex environments, and can update the digital twin model of the factory in real time.

[0035] In a specific embodiment, determining whether a gas leak occurs specifically includes the following steps: Obtain target images within a preset range around a first spatial position and a second spatial position, the target images including first category images and second category images, establish a first detection model based on a deep learning model, obtain a second historical image containing multiple categories of gas, input the second historical image into the first detection model for training, and input the target image into the trained first detection model, the first detection model outputs a first detection result of gas leakage, if no gas leakage is detected, perform frequency conversion on the second category image to obtain frequency information, input the frequency information into the second detection model, the second detection model is established based on a statistical model, identifies whether there is frequency information related to gas in the frequency information, the second detection model outputs a second detection result, if the second detection result is that there is no gas leakage, the frequency information is input into a third detection model, the third detection model is established based on a support vector machine model, and outputs a third detection result.

[0036] Specifically, images within a preset range around a first spatial position and a second spatial position in the target area are obtained. The target image may include two types of images of the target area: the first category of images is a visible light image obtained by an electro-optical camera (EO camera), and the second category of images is an infrared image obtained by an optical gas imaging camera (OGI camera). The first detection model is established based on a convolutional neural network to identify gas leaks in images. First, a second historical image containing multiple gases is obtained. The second historical image contains an image set of various gas leakage scenes. The second historical image is input into a deep learning model for training. During the training process, the model learns gas features in the image, such as gas shape, color, size, and position. After the training is completed, the first detection model analyzes the new target image and determines whether there is a gas leak in the image by extracting features of gas leakage from the image. If a gas leak is detected, the first detection model will output the monitored gas, as well as the coordinates and category information of the detected gas.

[0037] If the first detection model fails to detect a gas leak, the second category of images, namely the infrared images, are subjected to frequency conversion. Frequency conversion uses techniques such as Fourier transform to convert the time domain signal into the frequency domain, and converts visible changes in the image into frequency domain data. The frequency information obtained after frequency conversion can capture subtle changes in the image, such as small fluctuations in certain gases or temperature changes, to help compensate for details that the first model failed to identify.

[0038] A second detection model is established through the likelihood ratio test model. The likelihood ratio test is a statistical method used to evaluate the probability ratio between two hypotheses, usually the null hypothesis and the alternative hypothesis, to evaluate whether there is a target object in the regional image. The null hypothesis: there is no gas leakage in the image, and the frequency pattern conforms to the normal background or non-abnormal distribution. The alternative hypothesis: there is a gas leakage in the image, and the frequency pattern matches the characteristics of the gas leakage. The training data, namely the images of gas leakage and the images without leakage, are used to calculate the frequency feature distribution of the leakage image and the normal image in the frequency domain, and the likelihood ratio under the two hypothesis states is calculated. The likelihood ratio is the likelihood ratio of the data observed under the two hypotheses. When the likelihood ratio is greater than the preset threshold, the second detection model outputs that a gas leakage exists; if it is less than the preset threshold, it is considered that there is no gas leakage. By combining the frequency information likelihood ratio test, the second detection model can supplement the shortcomings of the deep learning model, provide additional detection angles and more detailed analysis, and improve the sensitivity and accuracy of detection.

[0039] If the second detection model also fails to detect a gas leak, a third detection model is used. This model is built based on a support vector machine (SVM) model, and the frequency information is used as a feature input into the SVM for training. During training, the SVM model maximizes the margin to find the optimal decision boundary, effectively distinguishing between images with gas leaks and those without. If the first, second, and third detection models all fail to detect a gas leak, it is assumed that no gas leak exists in that frame. By combining multiple models, each layer of the detection model focuses on different data features or dimensions, increasing the accuracy and reliability of gas leak detection and reducing the likelihood of false positives and negatives.

[0040] In a specific embodiment, obtaining the shape information of the gas includes the following steps: The target image in which gas is detected is defined as the reference image. Multiple frames of continuous reference images are obtained within a preset window. The moving average of the gas in the multiple frames of continuous reference images is calculated. The reference image is smoothed based on the moving average to obtain a background model. Whenever a new reference image is obtained, the shape information of the gas is extracted from the new reference image based on the background model.

[0041] Specifically, the image in which gas is detected is defined as a reference image. The reference image represents the gas leakage image in the target area at a certain moment. Multiple frames of reference images are continuously collected within a predetermined time window. The size of the time window can be set according to actual conditions. A moving average calculation is performed on the multiple frames of continuous reference images, that is, the average value of each pixel in each frame of the image in multiple images is calculated. The moving average calculation can eliminate noise and irregular fluctuations in the image and improve the stability and smoothness of the image. The moving average is used as the background model of the reference image. The background model represents the background part of the object in the area when there is no gas leakage. The current reference image is smoothed using the background model to remove noise and irrelevant dynamic changes in the image. The system will perform foreground extraction based on the previously constructed background model to extract the changed part of the reference image. This part usually corresponds to the gas leakage area. The extracted shape information includes the boundary and diffusion shape of the gas. The dynamic changes of gas leakage are captured by multiple frames of images. By calculating the moving average and background modeling, it helps to more accurately detect the morphological information of the leakage and improve the detection accuracy of gas leakage.

[0042] In a specific embodiment, identifying the gas leakage source based on shape information specifically includes the following steps: The shape information of the gas in multiple consecutive reference images from the first frame reference image to the current frame reference image is obtained, the contour information of the gas is identified based on the shape information, the position center point of the gas in each frame reference image is determined, the motion vector of the gas in adjacent reference images is calculated based on the position center point, the motion vectors corresponding to all consecutive frames are added together to obtain a sum vector, the starting point of the sum vector is used as the gas leakage source, the direction of the sum vector represents the flow direction of the leakage source, and the magnitude of the sum vector is used as the flow velocity of the leakage source.

[0043] Specifically, the shape information of the gas in multiple frames of continuous reference images from the first frame reference image to the current frame reference image is obtained. The first frame reference image is the initial image, and the current frame reference image is an image obtained over time. Image processing is performed on each frame of the image. The contour information of the gas is extracted using an edge detection algorithm such as the Canny algorithm. The gas contour in each frame of the reference image is analyzed, and the center point of the gas position is calculated. The center point is calculated by averaging the coordinates of the gas contour points. The motion vector of the gas is obtained by calculating the change in the position of the gas center point between consecutive frames. The motion vector reflects the position change of the gas over a period of time. The calculation formula of the motion vector is: ,in, is the coordinate of the center point of the gas in the tth frame, is the coordinate of the center point of the gas in the t+1 frame, is the motion vector of the gas, which represents the moving direction and displacement of the gas between two frames. The motion vectors of all consecutive frames are added together to obtain the sum vector. The starting point of the sum vector is the position of the gas leakage source. The sum vector represents the overall movement direction and displacement of the gas. Its starting point is the gas leakage source. The direction of the sum vector represents the flow direction of the gas, that is, the diffusion direction of the leakage source. The size of the sum vector represents the flow speed of the gas. By tracking the movement of the gas through consecutive frame images, the diffusion of the gas can be captured in real time, so as to more accurately locate the leakage source and track the changes in the leakage.

[0044] In a specific embodiment, determining the third spatial position of the leakage source specifically includes the following steps: Obtain device information of the camera that captured the reference image, including the shooting position, angle, and orientation; obtain a first distance from the leakage source to the shooting camera; convert the three-dimensional coordinates of the leakage source into first coordinates based on the motion vector of the leakage source, the first distance, and the device information of the camera; the first coordinates include the radius, azimuth, and pitch angle of the leakage source; and convert the first coordinates into third spatial coordinates in a Cartesian coordinate system and perform correction.

[0045] Specifically, the device information of the camera that captured the reference image is obtained, including the camera's actual position coordinates, the camera's shooting angle, and its orientation. The first distance refers to the actual distance from the leak source to the camera, which can be obtained using a distance measuring device such as a laser rangefinder. The first coordinates of the leak source are calculated based on the motion vector of the leak source, the first distance, and the camera's device information. The first coordinates include the radius, azimuth, and pitch angle of the leak source, where the radius is the first distance between the leak source and the camera, the azimuth is the angle of the leak source on the horizontal plane, reflecting the direction of the leak source relative to the camera's line of sight, and the pitch angle is the vertical angle of the leak source relative to the horizontal plane, i.e., the height angle of the gas leak source. The polar coordinates determined by the radius, azimuth, and pitch angle of the leak source are converted into third-space coordinates in a Cartesian coordinate system. The Cartesian coordinates obtained from the polar coordinate conversion are further corrected. The correction process may involve considering the camera's calibration error, position offset, or environmental factors to ensure that the final leak source location is more accurate.

[0046] In a specific embodiment, simulating the diffusion path of gas leakage specifically includes the following steps: A gas diffusion model is established based on fluid dynamics. The environmental data of the target area and the third-space position of the leakage source are input into the gas diffusion model. The gas diffusion model calculates the diffusion speed and diffusion direction of the gas based on the gas characteristics and environmental data, and calculates the diffusion range of the gas leakage at different time points to form a diffusion path.

[0047] Specifically, a gas diffusion model is established based on the principles of fluid dynamics. The gas diffusion model simulates the propagation behavior of gas in the air. The diffusion of gas is usually affected by environmental factors such as the velocity, density, temperature, and pressure of the fluid. Environmental data is input into the gas diffusion model, which calculates the diffusion velocity of the gas and determines the diffusion direction of the gas by combining wind direction data with the physical properties of the gas. Based on the diffusion velocity and direction of the gas, the gas diffusion model will calculate the diffusion path of the gas at different time points. The diffusion path describes the diffusion process and diffusion direction of the gas starting from the leak source. The gas diffusion model accurately predicts the diffusion path, diffusion velocity, and impact range of gas leaks, which helps to obtain the impact range of the leak source in real time after a gas leak occurs, providing the decision-making basis required for emergency response.

[0048] The above describes a gas leakage accurate early warning method based on digital twin in the embodiment of the present application. The following describes a gas leakage accurate early warning system based on digital twin in the embodiment of the present application. Figure 5 In the embodiments of the present application, an accurate gas leakage warning system based on digital twins is provided in one embodiment, including: Establish a module for obtaining reference data of the target area, the reference data including 3D point cloud data, regional images of multiple target objects in the target area, and environmental data, and generate a digital twin model of the target area based on the 3D point cloud data; a position determination module, configured to obtain a first spatial position of each target object in the target area, each target object including a plurality of target components, identify a second spatial position of each target component based on the reference data, and map the first spatial position and the second spatial position to the digital twin model; An analysis module is configured to determine whether a gas leak occurs within a preset range around the first spatial position and the second spatial position. If so, the analysis module obtains shape information of the gas, identifies the source of the gas leak based on the shape information, determines a third spatial position of the leak source, and maps the third spatial position to the digital twin model. The simulation module is used to establish a gas diffusion model. It simulates the diffusion path of gas leakage based on the environmental data of the target area and the gas diffusion model, and superimposes it on the digital twin model to form a visual diffusion path.

[0049] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0050] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0051] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A gas leakage accurate early warning method based on digital twin, characterized by: The method comprises: Step S1: Acquire reference data of a target area, the reference data including three-dimensional point cloud data, regional images of multiple target objects in the target area, and environmental data, and generate a digital twin model of the target area based on the three-dimensional point cloud data; Step S2: obtaining a first spatial position of each target object in the target area, where each target object includes a plurality of target components, identifying a second spatial position of each target component based on the reference data, and mapping the first spatial position and the second spatial position to the digital twin model; Step S3: Determine whether gas leakage occurs within a preset range around the first spatial position and the second spatial position. If so, obtain shape information of the gas, identify the gas leakage source based on the shape information, determine a third spatial position of the leakage source, and map the third spatial position to the digital twin model; Step S4: Establish a gas diffusion model, simulate the diffusion path of the gas leakage based on the environmental data of the target area and the gas diffusion model, and superimpose it into the digital twin model to form a visualized diffusion path.

2. The method according to claim 1, characterized in that Obtaining a first spatial position of each target object in the target area includes: Extracting multiple first cross-sections of multiple target objects from the three-dimensional point cloud data, obtaining a center point of each first cross-section, calculating the position coordinates of the center point, the radius distance of the first cross-section, and the normal vector, taking the center point as the starting point, extending along the normal vector to obtain a first reference line of each first cross-section, determining whether a distance between the first reference lines corresponding to two adjacent first cross-sections is less than a first threshold, and whether an angle between the two first reference lines is less than a second threshold, and if so, connecting the center points corresponding to the two first reference lines to obtain a target straight line, according to this step, integrating the first cross-sections corresponding to the center points located on the same target straight line into the same target object, and defining the two first cross-sections located at the endpoints of the same target object as reference cross-sections; The center points corresponding to the two reference sections are defined as the endpoint coordinates of the target object, and the angle between the line segment generated by the two endpoint coordinates and the preset reference coordinate axis and the average value of the radius distance of the two reference sections are defined as the azimuth and radius value of the target object respectively.

3. The method according to claim 2, characterized in that Get the center point of each first section, including: A representative point is set on each first section, and the point cloud data within a preset range around the representative point is defined as a node set. Two point clouds are selected from the node set as first candidate nodes of the first section, and the inverse normals of the two first candidate nodes are obtained. The vertical line segment between the two inverse normals is connected, and the center point of the vertical line segment is used as the center point of the first section.

4. The method according to claim 1, wherein Identifying a second spatial position of each target component based on the reference data includes: Establish a machine learning model, obtain a first historical image containing a target component, mark the target component type and plane coordinates in the first historical image, and input them into the machine learning model for learning. The machine learning model identifies multiple component position coordinates of the target component in the area image, projects the component position coordinates onto the three-dimensional point cloud data, obtains the device coordinate position of the camera that takes the area image, and generates multiple second reference lines from the device coordinate position. The second reference lines pass through the component position coordinates of the target component. The point closest to each second reference line in the three-dimensional point cloud data is defined as a second candidate node, and the center point of the multiple second candidate nodes is defined as the second spatial position of the target component.

5. The method according to claim 1, wherein Determine whether a gas leak has occurred, including: Acquire a target image within a preset range around the first spatial position and the second spatial position, the target image including a first category image and a second category image, establish a first detection model based on a deep learning model, acquire a second historical image containing multiple categories of gas, input the second historical image into the first detection model for training, and input the target image into the trained first detection model, the first detection model outputs a first detection result of gas leakage, if no gas leakage is detected, perform frequency conversion on the second category image to obtain frequency information, input the frequency information into the second detection model, the second detection model is established based on a statistical model, identifies whether there is frequency information related to gas in the frequency information, the second detection model outputs a second detection result, if the second detection result is that there is no gas leakage, then inputs the frequency information into a third detection model, the third detection model is established based on a support vector machine model, and outputs a third detection result.

6. The method according to claim 5, characterized in that Obtain gas shape information, including: The target image in which gas is detected is defined as a reference image, multiple frames of the reference image are obtained within a preset window, a moving average of the gas in the multiple frames of the reference image is calculated, the reference image is smoothed based on the moving average to obtain a background model, and whenever a new reference image is obtained, the shape information of the gas is extracted from the new reference image based on the background model.

7. The method according to claim 6, characterized in that Identifying a gas leakage source based on the shape information includes: Obtain shape information of the gas in multiple consecutive frames of reference images from the first frame reference image to the current frame reference image, identify contour information of the gas based on the shape information, determine the position center point of the gas in each frame reference image, calculate the motion vector of the gas in adjacent reference images based on the position center point, add the motion vectors corresponding to all consecutive frames to obtain a sum vector, use the starting point of the sum vector as the gas leakage source, the direction of the sum vector represents the flow direction of the leakage source, and the magnitude of the sum vector is used as the flow velocity of the leakage source.

8. The method according to claim 6, characterized in that Determining the third spatial position of the leakage source includes: Obtain device information of a camera that captures the reference image, the device information including a shooting position, angle, and orientation; obtain a first distance from a leakage source to the camera that captures the image; convert the three-dimensional coordinates of the leakage source into first coordinates based on a motion vector of the leakage source, the first distance, and the device information of the camera; the first coordinates include a radius, an azimuth, and a pitch angle of the leakage source; and convert the first coordinates into third spatial coordinates in a Cartesian coordinate system and perform correction.

9. The method according to claim 1, characterized in that Simulate the diffusion path of gas leaks, including: A gas diffusion model is established based on fluid dynamics, and the environmental data of the target area and the third spatial position of the leakage source are input into the gas diffusion model. The gas diffusion model calculates the diffusion speed and diffusion direction of the gas based on the gas characteristics and the environmental data, and calculates the diffusion range of the gas leakage at different time points to form a diffusion path.

10. A digital twin-based gas leakage accurate early warning system, used to implement a digital twin-based gas leakage accurate early warning method according to any one of claims 1 to 9, characterized in that: The system comprises: Establish a module for obtaining reference data of a target area, the reference data including three-dimensional point cloud data, regional images of multiple target objects in the target area, and environmental data, and generating a digital twin model of the target area based on the three-dimensional point cloud data; a position determination module, configured to obtain a first spatial position of each target object in the target area, each target object including a plurality of target components, identify a second spatial position of each target component based on the reference data, and map the first spatial position and the second spatial position to the digital twin model; an analysis module, configured to determine whether a gas leak occurs within a preset range around the first spatial position and the second spatial position; if so, obtain shape information of the gas, identify a gas leakage source based on the shape information, determine a third spatial position of the leakage source, and map the third spatial position to the digital twin model; The simulation module is used to establish a gas diffusion model, simulate the diffusion path of the gas leakage based on the environmental data of the target area and the gas diffusion model, and superimpose it on the digital twin model to form a visual diffusion path.

Images