Gas flow rate measuring method, equipment, program product and medium

Abstract

The invention discloses a gas flow velocity measurement method and device, a program product and a medium. The method comprises the following steps: acquiring a current frame image; generating multi-channel space-time image data based on the current frame image; acquiring current distance information; based on the multi-channel space-time image data and the current distance information, utilizing a multi-modal fusion network in a pre-trained gas flow rate measurement model to obtain multi-modal feature data; and outputting the current gas flow rate by using a flow rate detection network in the gas flow rate measurement model based on the multi-modal characteristic data.

Description

Technical Field

[0001] This invention relates to the field of gas detection technology, and in particular to a gas flow rate measurement method, a computing device cluster, a computer program product, and a computer-readable storage medium. Background Technology

[0002] In recent years, the need for quantitative analysis of gas leaks has become increasingly urgent in the fields of industrial production safety, environmental monitoring, and scientific research. Traditional gas flow rate measurement usually relies on contact-type equipment, which needs to be installed on pipelines, cannot be used for unknown leaks in open spaces, and does not have the capability for remote, non-invasive measurement.

[0003] Existing optical detection methods, such as infrared imaging-based gas detection, primarily focus on the qualitative identification and spatial localization of gas clouds, i.e., determining the presence of gas leaks and outlining their visible contours. However, these methods often struggle to provide crucial quantitative information about gas leak rates. Accurately determining gas leak flow rates is essential for assessing the severity of leak incidents, guiding emergency responses, and optimizing industrial processes. Summary of the Invention

[0004] To address the existing technical problems, this invention provides a gas flow velocity measurement method, a computing device cluster, a computer program product, and a computer-readable storage medium, which can accurately measure flow velocity.

[0005] In a first aspect, a gas velocity measurement method is provided, comprising: acquiring a current frame image; generating multi-channel spatiotemporal image data based on the current frame image; acquiring current distance information; obtaining multi-modal feature data based on the multi-channel spatiotemporal image data and the current distance information, using a multi-modal fusion network in a pre-trained gas velocity measurement model; and outputting the current gas velocity based on the multi-modal feature data and a velocity detection network in the gas velocity measurement model.

[0006] In a second aspect, a computing device cluster is provided, comprising at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, such that the computing device cluster executes the gas flow rate measurement method provided in the first aspect of this application.

[0007] Thirdly, a computer program product is provided, including a computer program that, when executed, implements a gas flow rate measurement method as described in the first aspect of this application.

[0008] Fourthly, a computer-readable storage medium is provided, including computer program instructions that, when executed by a cluster of computing devices, execute the gas flow rate measurement method provided in the first aspect of this application.

[0009] This application identifies the current frame image containing a gas region from continuous scene images. Based on the current frame image, multi-channel spatiotemporal image data is generated and combined with key physical parameters such as distance to form the input of a multimodal fusion network in a pre-trained gas velocity measurement model, resulting in multimodal feature data. This multimodal feature data includes image information and distance information, deeply integrating image and physical information to effectively address different application scenarios. Based on the multimodal feature data, the velocity detection network in the gas velocity measurement model outputs the current gas velocity. This method achieves efficient integration and collaborative analysis of different data modalities through the design of a multimodal fusion network. The method employs an end-to-end deep learning framework, which makes it more robust to image jitter, background temperature changes, and sensor noise generated during equipment use, ensuring stable and accurate velocity measurement even in real and complex industrial environments. Attached Figure Description

[0010] Figure 1 This is a diagram illustrating the application environment of a gas flow rate measurement method in one embodiment; Figure 2 This is a flowchart of a gas flow rate measurement method in one embodiment; Figure 3 This is a flowchart illustrating how multimodal feature data is obtained based on an early fusion method in one embodiment; Figure 4 This is a network structure diagram of a gas velocity measurement model based on an early fusion method in one embodiment; Figure 5 This is a network structure diagram of a gas velocity measurement model based on a late fusion approach in one embodiment; Figure 6 This is a schematic diagram of a gas flow rate measuring device in one embodiment; Figure 7 This is a schematic block diagram of a computing device provided in one embodiment; Figure 8 This is a schematic diagram of the structure of a computing device cluster provided in one embodiment. Detailed Implementation

[0011] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0013] In the following description, the expression “some embodiments” refers to a subset of all possible embodiments. However, it should be understood that “some embodiments” can be the same subset or different subsets of all possible embodiments and can be combined with each other without conflict.

[0014] See Figure 1 This diagram illustrates the application environment of a gas velocity measurement method in one embodiment. The environment includes an image acquisition device 12, a computing device 10, and a ranging device 16. The image acquisition device 12 acquires images of a gas detection scenario. The ranging device 16 measures the distance between a target point and a laser ranging device. The gas velocity measurement method is applied in the computing device 10. The computing device 10 stores a gas velocity measurement model, which is used to measure the velocity of gas leakage in the current frame image containing a gas region. Figure 2 As shown in application scenario two, the image acquisition device 12 can exist independently of the computing device 10. As shown in application scenario one, the image acquisition device 12 and the ranging device 16 can also be integrated into the computing device 10; that is, the computing device 10 can also include the image acquisition device 12 and the ranging device 16, for example, the computing device 10 is an infrared imaging device. The image acquisition device 12 is communicatively connected to the computing device 10, and the image acquisition device 12 can transmit data to the computing device 10. The ranging device 16 is also communicatively connected to the computing device 10, and the ranging device 16 can transmit data to the computing device 10. It is understood that the image acquisition device 12 and the ranging device 16 can be the same sensor device; for example, when using image-based distance measurement, the ranging device 16 is the same as the image acquisition device 12.

[0015] The computing device 10 includes a processor 13 and a memory 14. The processor 13 is used to execute program instructions corresponding to the gas flow rate measurement method. The memory 14 is used to store the program and data corresponding to the implementation of the gas flow rate measurement method. The computing device 10 can be used to train a gas flow rate measurement model and can also be used to apply the gas flow rate measurement model.

[0016] The computing device 10 includes, but is not limited to, handheld detection devices, non-handheld autonomously movable detection devices, and non-handheld, non-autonomous detection devices. Handheld detection devices include, but are not limited to, handheld imaging devices with infrared thermal imaging and / or visible light imaging capabilities. Non-handheld, autonomously movable detection devices include, but are not limited to, autonomously movable detection devices with infrared thermal imaging and / or visible light imaging capabilities. Non-handheld, non-autonomous detection devices include, but are not handheld and cannot be autonomously moved, devices with infrared thermal imaging and / or visible light imaging capabilities. Using handheld and non-handheld, autonomously movable detection devices, users can detect the flow velocity of gas targets in a moving scene or in a fixed scene during movement. Non-handheld, non-autonomous detection devices can detect the flow velocity of gas targets in a fixed scene. Therefore, the gas flow velocity measurement method provided in this application embodiment can be applied to various complex moving and changing scenes as well as fixed scenes. The computing device 10 can be a server, computer equipment, or terminal device. It can be a device with video processing or image processing capabilities, an infrared imaging device, a monitoring device with image capture capabilities, an autonomous mobile device, etc. Examples of autonomous mobile devices include, but are not limited to: vehicles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobile devices, airplanes, drones, ships, or robots, etc. The computing device 10 can also be a fixed electronic device mounted on a certain device or at a fixed location in a fixed scene. Terminal devices include, but are not limited to, mobile phones, tablets, wearable electronic devices, etc.

[0017] The image acquisition device 12 can be one or more sensors, such as monocular vision sensors or multi-view vision sensors. For example, it can be composed of one or more sensors, such as infrared thermal imaging sensors or depth sensors. It can also be a handheld computing device or gimbal equipped with a high-resolution infrared image acquisition module to obtain clear image data. It can also be a dual-light camera. The ranging device 16 can be composed of vision-based sensors (such as monocular vision sensors or multi-view vision sensors), or it can be composed of one or more sensors, such as laser ranging sensors, millimeter-wave sensors, or depth sensors. That is, the ranging can be performed by image principle, laser principle, millimeter-wave principle, or a combination of multiple technologies.

[0018] The processor 13 can be one or more. When there are multiple processors 13, the multiple processors can be integrated on one chip or set independently on each chip.

[0019] The computing device 10 may also include other sensor modules, including but not limited to environmental perception sensors and motion attitude sensors. Environmental perception sensors include, but are not limited to, one or more of the following: brightness sensors, temperature sensors, haze sensors, etc. Motion attitude sensors include, but are not limited to, one or more of the following combinations: inertial measurement units (IMUs), velocity sensors, acceleration sensors, gyroscope sensors, geomagnetic sensors, rotation vector sensors, steering wheel angle sensors, level sensors, tilt sensors, vibration sensors, displacement sensors, and gravity sensors, etc. The computing device 10 may use a high-resolution infrared thermal imager, such as a handheld or pan-tilt type, to acquire clear image data. It may also be equipped with ranging devices such as laser rangefinders or depth cameras to accurately measure the physical distance between the acquisition device and the gas leak source. Similar to the image acquisition device 12, other sensor modules may not be included in the computing device 10 and may exist independently of it, but may be able to communicate with it.

[0020] The computing device 10 may also include a display terminal for displaying images.

[0021] Please see Figure 2 This is a flowchart of a gas flow rate measurement method provided in an embodiment of this application. Figure 2 This is a flowchart illustrating the application of a gas flow rate measurement model. A gas flow rate measurement method is applied in a computing device, and this method includes the following steps: S10. Obtain the current frame image.

[0022] In this embodiment, the current frame image represents the latest frame in a series of acquired scene images. The current frame image can be an infrared image, a visible light image, or an image fused from both infrared and visible light images. Optionally, a pre-trained gas target detection model can be used to detect gas regions in the series of scene images. The gas target detection model takes the series of scene images as input, extracts gas features, and outputs gas regions based on these features. Simultaneously, data processing is performed based on the gas features to obtain gas heatmap data. The gas heatmap data represents the gas distribution in the current frame image.

[0023] S11. Generate multi-channel spatiotemporal image data based on the current frame image.

[0024] In this embodiment, the multi-channel spatiotemporal image data includes the current consecutive frame images and the current gas mask image. The current consecutive frame images refer to the consecutive frame images adjacent to the current frame image. The current gas mask image represents the image in the current frame image that identifies the regions where gas exists; for example, in the current gas mask image, the pixel value of pixels in the gas regions is marked as 1, and the pixel value of pixels in the non-gas regions is marked as 0. The current consecutive frame images can be continuous grayscale images.

[0025] This method integrates N consecutive current frame images with a single current gas mask image, stitching the selected N consecutive current frame images along the channel dimension to construct a multi-channel spatiotemporal image data. This operation merges N independent (1,H,W) shaped tensors with a single (1,H,W) shaped mask into a single (N+1,H,W) shaped input tensor, where H represents the height and W represents the width. Through this operation, the original N dynamically changing tensors along the time dimension and the tensor indicated by the current gas mask image are effectively encoded into a new tensor, reflecting local correlations along the channel dimension. This allows for subsequent processing using efficient 2D CNNs, avoiding the computationally expensive 3D CNN approach.

[0026] S12. Obtain current distance information.

[0027] In this embodiment, the current distance information can be a distance scalar obtained by a laser ranging device. For example, this distance scalar represents the distance between the target point corresponding to the center point of the current frame image and the laser ranging device. Alternatively, the current distance information can be a pixel-level depth map obtained using a depth camera or LiDAR. This depth map represents the depth value corresponding to each pixel.

[0028] S13. Based on multi-channel spatiotemporal image data and current distance information, multi-modal feature data is obtained by using the multi-modal fusion network in the pre-trained gas flow velocity measurement model.

[0029] In this embodiment, the gas velocity measurement model is trained on a training dataset. The input to the model is multi-channel spatiotemporal image data and current distance information. A multimodal fusion network is used to fuse the multi-channel spatiotemporal image data and current distance information to obtain multimodal data. Then, multimodal feature data is obtained based on this multimodal data. This process involves first fusing the data and then extracting multimodal features from the fused data. Alternatively, the multimodal fusion network can extract gas feature data from the multi-channel spatiotemporal image data and distance feature data from the current distance information separately, and then fuse these two sets of data to obtain multimodal feature data. Since the gas velocity measurement model is trained on a training dataset, distance is also introduced during training. Distance is a key physical parameter because it establishes the scale relationship between two-dimensional image pixels and three-dimensional physical space. At different measurement distances, the actual physical area represented by a single pixel in the image is different; the greater the distance, the larger the physical area corresponding to a unit pixel. Therefore, during training, the gas velocity measurement model can learn the relationship between distance and velocity in the sample dataset. It is necessary to introduce distance information into the training samples to calibrate the visual features. In this way, the trained gas velocity measurement model can accurately distinguish between small-velocity leaks at close range and large-velocity leaks at long distance, thereby improving the accuracy of the measurement results.

[0030] S14. Based on multimodal feature data, the current gas flow rate is output using the flow rate detection network in the gas flow rate measurement model.

[0031] In this embodiment, the multimodal fusion network is connected to the flow velocity detection network. The flow velocity detection network is a regression prediction head. Regardless of the fusion method used, it will ultimately output a value representing the predicted flow velocity. The current gas flow velocity represents the predicted gas leakage velocity based on the current frame image. The flow velocity detection network can be a common type of neural network, such as a feedforward neural network, a recurrent neural network and its variants, or a convolutional neural network.

[0032] In the above embodiments, this application identifies the current frame image containing a gas region from continuous scene images. Based on the current frame image, multi-channel spatiotemporal image data can be generated and combined with key physical parameters such as distance to form the input of a multimodal fusion network in a pre-trained gas velocity measurement model, obtaining multimodal feature data. The multimodal feature data includes image information and distance information. Deep integration of image information and physical information can effectively address different application scenarios. Based on the multimodal feature data, the current gas velocity is output using the velocity detection network in the gas velocity measurement model. This method achieves efficient integration and collaborative analysis of different data modes by designing a multimodal fusion network. This method adopts an end-to-end deep learning framework, which makes this method more robust to image jitter, background temperature changes, and sensor noise generated during equipment use, ensuring stable and accurate velocity measurement even in real and complex industrial environments.

[0033] In some embodiments, the multi-channel spatiotemporal image data includes the current consecutive frame images and the current gas mask image, and generating the multi-channel spatiotemporal image data based on the current frame images includes: Based on the current frame image, obtain the current consecutive frame images; Generate the current gas mask image based on the current consecutive frame images; The current consecutive frame image and the current gas mask image are stitched together along the channel dimension to obtain the multi-channel spatiotemporal image data.

[0034] In this embodiment, a specially trained YOLOv5-based gas target detection model based on video sequences is used to detect the input continuous frame scene images. Gas feature maps extracted during the detection process can be analyzed to obtain gas heat maps. These heat maps are then used to analyze the acquired continuous frame scene images frame by frame to automatically identify gas regions within the continuous frame scene images. When the gas target detection model successfully detects a gas leak in a certain frame of the video, that frame is designated as the current frame. The current frame and its adjacent N-1 frames are combined to form a continuous image sequence containing N frames, i.e., the current continuous frame image. For the current frame, a mask image is generated using the gas region location data output by the gas target detection model and the gas heat map. This mask image is then binarized to obtain the current gas mask image. This current gas mask image has the same size as the current frame image, thus providing a precise spatial prior for the gas location for the subsequent gas velocity measurement model.

[0035] This method integrates N consecutive current frame images with a single current gas mask image, stitching the selected N consecutive current frame images along the channel dimension to construct a multi-channel spatiotemporal image data. This operation merges N independent (1,H,W) shaped tensors with a single (1,H,W) shaped mask into a single (N+1,H,W) shaped input tensor, where H represents the height and W represents the width. Through this operation, the original N dynamically changing tensors along the time dimension and the tensor indicated by the current gas mask image are effectively encoded into a new tensor, reflecting local correlations along the channel dimension. This allows for subsequent processing using efficient 2D CNNs, avoiding the computationally expensive 3D CNN approach.

[0036] In the above embodiments, a strategy of localization followed by analysis is introduced. By accurately locking the region of the gas cloud using an initial gas target detection model, large areas of irrelevant background in the image can be effectively ignored. The gas velocity measurement model no longer needs to expend its own learning capacity to search for and identify gas regions from the entire image. This strategy not only reduces the learning difficulty of the model but also enables the model to achieve higher accuracy with lower complexity, optimizing overall performance and efficiency.

[0037] In some embodiments, the multi-channel spatiotemporal image data includes current consecutive frame images and current gas mask images, and the multimodal fusion network includes a data fusion network, a multimodal feature extraction network, and a multimodal fully connected network, such as... Figure 3 As shown, Figure 3 This is a flowchart illustrating how multimodal feature data is obtained based on an early fusion method in one embodiment; such as... Figure 4 As shown, Figure 4 This is a network structure diagram of a gas flow velocity measurement model based on an early fusion method in one embodiment, combined with... Figure 3 and Figure 4 The description of S13 may also include: S31. Based on the current distance information and multi-channel spatiotemporal image data, perform fusion processing through a data fusion network to obtain multimodal data.

[0038] In this embodiment, multi-channel spatiotemporal image data and current distance information are used as two input sources for the gas velocity measurement model. This is to allow the gas velocity measurement model to simultaneously understand the correlation between image dynamics and distance information. Figure 4 As shown, this multimodal fusion network is an early fusion method, which involves first fusing the data and then extracting features from the fused data.

[0039] Optionally, the step of performing fusion processing through a data fusion network based on the current distance information and the multi-channel spatiotemporal image data to obtain the multimodal data includes: Based on the current distance information, obtain the current distance image indicating the distance information; By multiplying the current distance image with the current gas mask image, a target gas mask image containing distance information is obtained; The target gas mask image and the multi-channel spatiotemporal image data are stitched together along the channel dimension using a data fusion network to obtain the multimodal data.

[0040] In this embodiment, the pixel value of each pixel in the current distance image represents the distance value. The current distance image and the current gas mask image are the same size. Multiplying the current distance image and the current gas mask image yields a target gas mask image containing distance information. Since the current gas mask image indicates gas regions, non-gas regions have pixels of 0 after binarization. This multiplication method ensures that distance information is activated only in gas regions. The data fusion network includes, but is not limited to, cascading operations. Through cascading operations, the target gas mask image and multi-channel spatiotemporal image data are stitched together along the channel dimension to obtain multimodal data.

[0041] Optionally, obtaining the current distance image indicating the distance information based on the current distance information includes at least one of the following: Obtain the current distance value represented by the current distance information and the weight matrix representing the distance weights; multiply the current distance value by the weight corresponding to each pixel in the weight matrix to obtain the current distance image; or Obtain the depth map represented by the current distance information, and normalize the depth map to obtain the current distance image.

[0042] In this embodiment, the acquired current distance information is first spatially encoded. All pixel values ​​in the weight matrix can be the same or different. For example, starting from the center pixel of the weight matrix, the values ​​decrease linearly outwards in a radial pattern. Specifically, when the distance information is a scalar obtained through a laser ranging device, i.e., a current distance value, a weight matrix with the same image resolution as the current gas mask image is created. All pixel values ​​in the weight matrix can be set to 1, or the weight matrix can be set to a radial linear decay, i.e., the center value of the weight matrix is ​​set to 1, and the values ​​decrease linearly outwards to 0, thus assigning a higher weight to the distance information in the central region of the image. This weight matrix is ​​then multiplied by the distance scalar to obtain a single-channel tensor, resulting in the current distance image. Subsequently, the current distance image is multiplied by the current gas mask image. If the distance information is not a scalar but a pixel-level depth map obtained using a depth camera or LiDAR, the depth map can be directly normalized and multiplied element-wise with the current gas mask image. This approach ensures that distance information is activated only in the gas region. The final tensor shape of the multimodal data remains (N+1, H, W), and features will be extracted directly through a multimodal feature extraction network.

[0043] S32. Based on multimodal data, multidimensional gas feature data is extracted through a multimodal feature extraction network.

[0044] In this embodiment, the multimodal feature extraction network can be a CNN network. Different sensors exhibit different response patterns to the same interfering gas and also different response patterns to the target gas. Early fusion allows the multimodal feature extraction network to simultaneously observe the original response patterns of all sensors, making it easier to learn joint feature representations of the target gas and interfering gas at different distances, as well as different mixing ratios at different distances. The information provided by different sensor modalities is complementary. Early fusion ensures that this complementary original information is preserved and utilized to the greatest extent when constructing the final feature representation, avoiding the loss of key details during their respective feature extraction processes.

[0045] S33. Based on the multidimensional gas feature data, the multimodal feature data is obtained through a multimodal fully connected network.

[0046] In the above embodiments, based on the current distance information and the multi-channel spatiotemporal image data, a data fusion network is first used to perform fusion processing to obtain multimodal data. This involves fusing image information and distance information, maximizing the preservation and utilization of these two original information types and avoiding the loss of key details during their respective feature extraction processes. Then, based on the multimodal data, a multimodal feature extraction network is used to extract multidimensional gas feature data. Finally, based on the multidimensional gas feature data, a multimodal fully connected network is used to obtain multimodal feature data, adaptable to different sensor configurations and application requirements. This flexible architecture design deeply integrates infrared image information with physical information, effectively addressing various application scenarios.

[0047] In some embodiments, the multi-channel spatiotemporal image data includes the current consecutive frame image and the current gas mask image; the multimodal fusion network includes a gas feature extraction network, a distance feature extraction network, a feature fusion network, and a fully connected network; and the multimodal feature data obtained based on the multi-channel spatiotemporal image data and the current distance information, using the multimodal fusion network in a pre-trained gas velocity measurement model, includes: Based on the multi-channel spatiotemporal image data, gas feature data is obtained using the gas feature extraction network. Based on the current distance information, distance feature data is obtained using the distance feature extraction network; The gas feature data and the distance feature data are fused using the feature fusion network to obtain fused gas data. Based on the fused gas data, the multimodal feature data is obtained by processing it using the fully connected network.

[0048] In this embodiment, as Figure 5 As shown, Figure 5 This diagram illustrates the network architecture of a gas velocity measurement model based on a late-stage fusion approach in one embodiment. The gas feature extraction network can be a CNN network, and the distance feature extraction network can be a Multi-Layer Perceptron (MLP) network. Gas feature data is represented by a gas feature vector (X,1,1), where X represents the dimension of the vector. Distance feature data is represented by a distance feature vector (Y,1,1), where Y represents the dimension of the vector. The feature fusion network includes cascaded operations.

[0049] This method extracts high-level features from multi-channel spatiotemporal image data and distance data separately through dedicated sub-networks, and then concatenates these two types of features at a deeper layer of the network. This method is mainly suitable for cases where the distance information is a one-dimensional scalar. The actual process is shown in the attached figure. Figure 5As shown. For the gas feature extraction network, a validated, high-performance CNN network is used to extract abstract spatial and temporal features layer by layer. After the last convolutional layer of the backbone network, Global Average Pooling (GAP) is applied to compress the high-dimensional feature map in the spatial dimension, outputting a gas feature vector. For the distance feature extraction network, a small Multilayer Perceptron (MLP) is used to extract a more expressive distance feature vector. The purpose is to match the dimension and abstraction level of the distance features with the gas feature vector, because the image features, after being extracted by the CNN network, are already high-dimensional and highly abstract gas feature vectors. The distance feature extraction network also outputs a distance feature vector. Subsequently, the feature vectors obtained from the two sub-networks are concatenated to form a multimodal fusion feature vector, i.e., fused gas data.

[0050] In the above embodiments, feature extraction is performed independently to ensure that the unique physical characteristics of each modality are fully explored. Distance information collected by the distance sensor can capture dynamic physical information such as gas concentration gradient, diffusion velocity, and spatial distribution. Combining this distance information with the gas information allows for more accurate extraction of the distance to the leak point and the concentration diffusion direction vector. Image features can be used to extract the thermal radiation profile of the gas plume. After fusing the features extracted from both methods, and combining the distance information with the gas profile, the coordinates of the leak source can be accurately located. Multi-channel spatiotemporal image data can be used to extract visual morphological features such as the shape, texture, and trajectory of the gas cloud. Extracting each feature individually before fusing them reduces cross-interference from noise or irrelevant information in the original data, thereby improving the accuracy of flow velocity detection.

[0051] In some embodiments, the method further includes: Obtain a training dataset, wherein each training sample in the training dataset includes a multi-channel spatiotemporal sample image, sample distance data, and a gas velocity label corresponding to the training sample; Based on the multi-channel spatiotemporal sample map and the sample distance data, multi-modal sample feature data is obtained by using the multi-modal fusion network in the gas flow velocity measurement model currently being trained. Based on the multimodal sample feature data, the gas flow rate of the current training is output using the flow rate detection network in the gas flow rate measurement model currently being trained; The goal is to train the gas velocity measurement model based on the premise that the error between the gas velocity and the gas velocity label in the current training is less than a preset error value.

[0052] In this embodiment, the multi-channel spatiotemporal sample map includes consecutive frame sample images and gas mask sample images. The consecutive frame sample images include the current frame sample image and multiple adjacent frame images. The method for obtaining the gas mask sample image is similar to that for obtaining the current gas mask image, and will not be described again here. The sample distance data represents the distance at the time of acquiring the training samples.

[0053] The training dataset includes data collected from different types of gases or the same type of gas at various distances, scenarios, and flow rates. For example, for the same gas leak, samples of gas leaks at different flow rates can be collected at the same distance, and samples of gas leaks at the same flow rate can be collected at different distances. The above method can also be used to collect data from different scenarios. A large amount of video data needs to be collected, including gas leaks of different types of gases at various distances, scenarios, and flow rates. During data acquisition, the distance between the gas leak location and the shooting location, as well as the gas flow rate measured by the flow meter, are measured in advance. The flow rate and distance are then continuously adjusted. Simultaneously, the actual flow rate value and the distance between the camera and the gas source are accurately recorded for each video segment; the actual flow rate value serves as the gas flow rate label. Secondly, during data acquisition, the optical axis of the image acquisition device should be orthogonal to the gas leak source, and the core area of ​​the gas dynamics should be precisely focused to suppress geometric distortions introduced by perspective and tilted viewpoints, thereby ensuring that the recorded distance accurately reflects the visual scale in the image. In addition, the collected data should be preprocessed, including frame sampling and resolution adjustment, to facilitate subsequent model training and evaluation. The training process can be performed on at least one computing device with high-performance computing capabilities, such as GPUs or cloud computing platforms, to perform complex image processing and deep learning computations.

[0054] Specifically, firstly, a large amount of video data (such as infrared video data) recorded under different gas types and flow rates and distances is collected. For each data segment, a series of sample images are generated, and each sample image is precisely labeled with its corresponding real flow rate and measurement distance. Distance is a key physical parameter in this method because it establishes the scale relationship between two-dimensional image pixels and three-dimensional physical space. At different measurement distances, the real physical area represented by a single pixel in the image is different; the farther the distance, the larger the physical area corresponding to a unit pixel. Therefore, distance information is introduced to calibrate visual features; otherwise, the model will have difficulty distinguishing between low-velocity leaks at close range and high-velocity leaks at long distances, leading to significant deviations in measurement results. By superimposing distance information, the model can learn the flow rate characteristics of gas leaks at different distances during training. Thus, the trained model can accurately distinguish between low-velocity leaks at close range and high-velocity leaks at long distances.

[0055] Optionally, the training dataset can be formed from multiple training samples collected for one gas or from a mixture of training samples collected for multiple gases.

[0056] In this embodiment, for example, proprietary models can be trained separately for gases with different physical properties, such as methane and sulfur hexafluoride, to address the differences in physical properties and visual characteristics of different gases.

[0057] In this embodiment, the model employs end-to-end supervised training, using real flow velocity values ​​as labels and providing supervision signals through mean squared error calculation. During training, appropriate hyperparameters, learning rates, and batch sizes need to be set, and data augmentation and normalization operations are performed to improve the model's generalization ability and robustness. To address the differences in the physical properties of different gases, this method trains dedicated models for different target gases such as methane and sulfur hexafluoride separately to ensure the accuracy and reliability of predictions.

[0058] Optionally, similar to the application steps described above, an early fusion approach can also be used during training. Based on the multi-channel spatiotemporal sample map and the sample distance data, multi-modal sample feature data can be obtained using the multi-modal fusion network in the currently trained gas flow velocity measurement model, including: Based on the sample distance data and the multi-channel spatiotemporal sample map, a fusion process is performed through a data fusion network to obtain multimodal sample data; Based on the multimodal sample data, multidimensional gas feature sample data is extracted through the multimodal feature extraction network in training; Based on the multidimensional gas feature sample data, the multimodal sample feature data is obtained through the multimodal fully connected network during training.

[0059] Optionally, similar to the application steps described above, a late-stage fusion approach can also be used during training. Based on the multi-channel spatiotemporal sample map and the sample distance data, multi-modal sample feature data can be obtained using the multi-modal fusion network in the currently trained gas flow velocity measurement model, including: Based on the multi-channel spatiotemporal sample map, gas feature sample data is obtained using the gas feature extraction network under training. Based on the sample distance data, distance feature sample data is obtained using the distance feature extraction network in training; The gas feature sample data and the distance feature sample data are fused using the feature fusion network during training to obtain fused gas sample data. Based on the fused gas sample data, the multimodal sample feature data is obtained by processing the fully connected network during training.

[0060] During training, the network structure of the gas velocity measurement model is the same as that of the pre-trained gas velocity measurement model; however, the model parameters are not yet fixed during training and require iterative training. Furthermore, to improve the algorithm's prediction accuracy and generalization ability, parameter tuning and optimization are necessary. For example, adjusting the optimal number of frames N for constructing the spatiotemporal feature map. Simultaneously, models should be trained separately for different gases to adapt to their unique physical characteristics.

[0061] In the above embodiments, the actual flow velocity is used as the label, and a supervisory signal is provided by calculating the error. During the training process, the goal is to train the gas flow velocity measurement model so that the error between the current gas flow velocity and the gas flow velocity label is less than a preset error value, thereby improving the model's generalization ability and robustness.

[0062] In some embodiments, the method further includes at least one of the following: The current gas flow rate is displayed in real time on the display image of the user interface; An alarm will be triggered when the current gas flow rate is greater than or equal to the preset flow rate. Based on the current leakage level indicated by the current gas flow rate, execute the alarm warning corresponding to the current leakage level.

[0063] In this embodiment, the displayed image can be the current frame image. The preset flow rate can be configured with different flow rate thresholds according to the scene type to adapt to different scene requirements. Different gas flow rate ranges corresponding to different leakage levels can be pre-configured; for example, a leakage greater than 0 and less than or equal to 1 unit is considered minor, greater than 1 unit but less than or equal to 5 units is considered toxic, and greater than 5 units is considered severe. Different leakage levels correspond to different alarm methods. The more severe the leakage, the more alarm methods or the more intense the alarms, etc.

[0064] In the above embodiments, when the current gas flow rate is predicted, a warning is displayed on the screen or a warning is given when the warning conditions are met, or different warnings are executed, in order to reduce the harm caused by gas leakage.

[0065] In one application scenario, the gas velocity measurement method includes: data sample acquisition and preprocessing, gas leak area detection and gas mask sample image generation, multi-channel spatiotemporal sample map construction, multi-modal sample feature fusion based on multi-channel spatiotemporal sample map and sample distance data, obtaining a trained gas velocity measurement model after model training, acquiring continuous frame scene images, acquiring the current frame image based on the continuous frame scene images; generating multi-channel spatiotemporal image data based on the current frame image; acquiring current distance information; obtaining multi-modal feature data based on the multi-channel spatiotemporal image data and the current distance information using the multi-modal fusion network in the pre-trained gas velocity measurement model; and outputting the current gas velocity based on the multi-modal feature data using the velocity detection network in the gas velocity measurement model.

[0066] The steps of the above method have been described in one or more embodiments and will not be repeated here. The collected data is analyzed, and the gas cloud region is automatically identified and located using a target detection model. A spatial mask for the gas is then generated by combining this with a heatmap. This mask delineates the gas region, providing prior knowledge for subsequent processing that ignores irrelevant background information. Next, multiple consecutive frames of images are integrated with this mask to construct a spatiotemporal feature map that simultaneously represents gas motion and morphology. Then, an effective multimodal fusion strategy is used to effectively integrate the spatiotemporal features extracted from the images with distance information. This scheme covers two multimodal data fusion paths: concatenating feature vectors from different sources at a deep network layer and structurally integrating distance information into the image at the input stage. Finally, the fused multimodal features are input into a regression network, which outputs the predicted gas velocity.

[0067] The embodiments provided in this application enable remote, non-contact gas flow rate measurement in a controlled environment. This application combines time-series image analysis with multimodal deep learning techniques to improve the accuracy and robustness of gas flow rate measurement.

[0068] After training and deployment, the model can be used for real-time flow velocity prediction. First, a video stream is acquired in real-time from an infrared thermal imager. The current frame and the previous N-1 frames are captured from the video stream to form a continuous N-frame image sequence. The aforementioned heatmap method is then used to analyze this image sequence to generate a mask indicating the gas location in real time. Subsequently, the continuous N-frame images are stitched together with this mask along the channel to form the model's image input. Distance information for the current frame is obtained through a distance measurement device. Finally, both the image and distance information are input into the flow velocity prediction model, achieving end-to-end, non-contact real-time measurement. This method, through data acquisition, spatiotemporal feature map construction, multimodal feature fusion, and end-to-end regression prediction, combines time-series image analysis and multimodal deep learning techniques to achieve remote, non-contact quantitative measurement of gas leakage flow velocity. This method has broad application prospects and can play an important role in industrial, environmental, and safety monitoring fields.

[0069] One or more embodiments of this application, when combined, have at least the following characteristics: Data-Driven Fast Approximation of Physical Processes: This method offers an efficient alternative to traditional physical modeling. Traditional fluid dynamics methods require precise geometric models and boundary conditions, and data acquisition and computation are time-consuming and labor-intensive. This method constructs a deep learning framework from visual observation to physical results through end-to-end learning, enabling fast approximation of complex gas dynamic physical processes.

[0070] High robustness and interference resistance: Traditional non-deep learning methods in the vision domain often rely on manually extracted features, such as optical flow methods. These features are highly sensitive to camera shake and background noise, resulting in poor robustness. This method employs an end-to-end deep learning framework, enabling the model to automatically learn the deep features most relevant to flow velocity from raw pixel data, eliminating the need for tedious and limited manual feature engineering. This makes the method more robust to image shake, background temperature changes, and sensor noise generated during equipment use, ensuring stable and accurate measurements even in real and complex industrial environments.

[0071] Guided Feature Learning and Model Optimization: This method introduces a strategy of localization followed by analysis. Localization initially identifies the gas region, resulting in a mask representing the gas region. The initial object detection model accurately locates the gas cloud region, effectively ignoring large areas of irrelevant background in the image. The flow velocity measurement network no longer needs to expend its learning capacity to search and identify gas regions across the entire image. This strategy not only reduces the learning difficulty of the model but also enables it to achieve higher accuracy with lower complexity, optimizing overall performance and efficiency.

[0072] Flexible Multimodal Fusion Architecture: This method designs a flexible multimodal fusion architecture, including both early and late fusion paths, to adapt to different sensor configurations and application requirements. This flexible architecture design enables deep integration of infrared image information and physical information, effectively addressing various application scenarios.

[0073] Comprehensive application to multiple scenarios and distances: The design of this method takes into account the influence of different gas types and multiple scenarios. During the data collection phase, classification was carried out, and corresponding models were trained for different gas types, which can improve the adaptability to different application scenarios.

[0074] Safety and Application Expansion of Remote Non-Contact Measurement: The non-contact measurement characteristics of this method bring safety advantages. Operators and equipment can maintain a safe distance from potential hazardous gas leak sources, reducing personnel risks. Furthermore, this method expands its application scope, enabling its use in scenarios where traditional contact sensors cannot be deployed.

[0075] Please see Figure 6 One embodiment of this application provides a gas flow rate measurement method and apparatus, which is exemplary. Figure 6 A schematic diagram of a gas flow rate measuring device provided in an exemplary embodiment of this application is shown. This gas flow rate measuring device is applied in a computing device, or it can be a computing device. The gas flow rate measuring device includes: an acquisition module 90 for acquiring a current frame image; a generation module 91 for generating multi-channel spatiotemporal image data based on the current frame image; the acquisition module 90 is also used to acquire current distance information; a fusion module 92 for obtaining multi-modal feature data based on the multi-channel spatiotemporal image data and the current distance information, using a multi-modal fusion network in a pre-trained gas flow rate measuring model; and a prediction module 93 for outputting the current gas flow rate based on the multi-modal feature data, using a flow rate detection network in the gas flow rate measuring model.

[0076] Optionally, the generation module 91 is also used for: Based on the current frame image, obtain the current consecutive frame images; Generate the current gas mask image based on the current consecutive frame images; The current consecutive frame image and the current gas mask image are stitched together along the channel dimension to obtain the multi-channel spatiotemporal image data.

[0077] Optionally, the fusion module 92 is also used for: Based on the current distance information and the multi-channel spatiotemporal image data, the multimodal data is obtained by performing fusion processing through a data fusion network. Based on the multimodal data, multidimensional gas feature data is extracted through the multimodal feature extraction network; Based on the multidimensional gas feature data, the multimodal feature data is obtained through the multimodal fully connected network.

[0078] Optionally, the fusion module 92 is also used for: Based on the current distance information, obtain the current distance image indicating the distance information; By multiplying the current distance image with the current gas mask image, a target gas mask image containing distance information is obtained; The target gas mask image and the multi-channel spatiotemporal image data are stitched together along the channel dimension using a data fusion network to obtain the multimodal data.

[0079] Optionally, the fusion module 92 is also used for: Obtain the current distance value represented by the current distance information and the weight matrix representing the distance weights; multiply the current distance value by the weight corresponding to each pixel in the weight matrix to obtain the current distance image; or Obtain the depth map represented by the current distance information, and normalize the depth map to obtain the current distance image.

[0080] Optionally, the fusion module 92 is also used for: Based on the multi-channel spatiotemporal image data, gas feature data is obtained using the gas feature extraction network. Based on the current distance information, distance feature data is obtained using the distance feature extraction network; The gas feature data and the distance feature data are fused using the feature fusion network to obtain fused gas data. Based on the fused gas data, the multimodal feature data is obtained by processing it using the fully connected network.

[0081] Optionally, a training module 94 is also included for: Obtain a training dataset, wherein each training sample in the training dataset includes a multi-channel spatiotemporal sample image, sample distance data, and a gas velocity label corresponding to the training sample; Based on the multi-channel spatiotemporal sample map and the sample distance data, multi-modal sample feature data is obtained by using the multi-modal fusion network in the gas flow velocity measurement model currently being trained. Based on the multimodal sample feature data, the gas flow rate of the current training is output using the flow rate detection network in the gas flow rate measurement model currently being trained; The goal is to train the gas velocity measurement model based on the premise that the error between the gas velocity and the gas velocity label in the current training is less than a preset error value.

[0082] Optionally, the training dataset is formed from multiple training samples collected from one gas or from a mixture of training samples collected from multiple gases.

[0083] Optionally, the prediction module 93 is also used for: The current gas flow rate is displayed in real time on the display image of the user interface; An alarm will be triggered when the current gas flow rate is greater than or equal to the preset flow rate. Based on the current leakage level indicated by the current gas flow rate, execute the alarm warning corresponding to the current leakage level.

[0084] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the gas flow rate measurement method apparatus includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0085] This application embodiment can, based on the above method, exemplarily divide the gas flow rate measurement method device into functional modules. For example, the gas flow rate measurement method device may include functional modules corresponding to each functional division, or two or more functions may be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0086] like Figure 7As shown, computing device 10 includes a processor 13, a memory 14, and a communication interface 15. The processor 13, memory 14, and communication interface 15 communicate via a bus. Computing device 10 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 10. The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in the figure, but this does not indicate that there is only one bus or one type of bus. Bus 104 can include a path for transmitting information between various components of computing device 10 (e.g., memory 14, processor 13, communication interface 15). Processor 13 can include any one or more processors such as a central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0087] Memory 14 may include volatile memory, such as random access memory (RAM). Processor 13 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0088] The memory 14 stores executable program code, and the processor 13 executes this executable program code to implement the functions of the aforementioned modules, thereby realizing the gas flow rate measurement method. That is, the memory 14 stores instructions for executing the gas flow rate measurement method. Alternatively, the memory 14 stores executable code, and the processor 13 executes this executable code to implement the functions of the aforementioned gas flow rate measuring device, thereby realizing the gas flow rate measurement method. That is, the memory 14 stores instructions for executing the gas flow rate measurement method.

[0089] The communication interface 15 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 10 and other devices or communication networks.

[0090] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone. Figure 8 As shown, the computing device cluster includes at least one computing device 10. The memory 14 of one or more computing devices 10 in the computing device cluster may store the same instructions for executing the gas flow rate measurement method. In some possible implementations, the memory 14 of one or more computing devices 10 in the computing device cluster may also each store partial instructions for executing the gas flow rate measurement method. In other words, a combination of one or more computing devices 10 can jointly execute the instructions for executing the gas flow rate measurement method.

[0091] It should be noted that the memory 14 in different computing devices 10 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the gas flow rate measuring device. That is, the instructions stored in the memory 14 of different computing devices 10 can implement the functions of one or more modules.

[0092] In another aspect, this application provides a computer-readable non-volatile storage medium storing a computer program. When the computer program is executed by a processor, it causes the processor to perform the steps of a gas flow rate measurement method provided in any of the above embodiments of this application.

[0093] In another aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of a gas flow rate measurement method as described in any embodiment of this application and / or a training method based on a joint model.

[0094] Those skilled in the art will understand that all or part of the processes in the methods provided in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0095] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method of measuring the flow rate of a gas, characterized by, The method comprises: acquiring a current frame image; based on the current frame image, generating multi-channel spatio-temporal image data; acquiring current distance information; based on the multi-channel spatio-temporal image data and the current distance information, using a multi-modal fusion network in a pre-trained gas flow rate measurement model to obtain multi-modal feature data; based on the multi-modal feature data, using a flow rate detection network in the gas flow rate measurement model to output a current gas flow rate.

2. The gas flow rate measurement method of claim 1, wherein, The multi-channel spatio-temporal image data includes a current continuous frame image and a current gas mask image, and the generation of the multi-channel spatio-temporal image data based on the current frame image comprises: based on the current frame image, acquiring the current continuous frame image; based on the current continuous frame image, generating a current gas mask image; splicing the current continuous frame image and the current gas mask image along the channel dimension to obtain the multi-channel spatio-temporal image data.

3. The method of claim 1, wherein, The multi-channel spatio-temporal image data includes a current continuous frame image and a current gas mask image, and the multi-modal fusion network includes a data fusion network, a multi-modal feature extraction network, and a multi-modal fully connected network. Based on the multi-channel spatio-temporal image data and the current distance information, using a multi-modal fusion network in a pre-trained gas flow rate measurement model to obtain multi-modal feature data comprises: based on the current distance information and the multi-channel spatio-temporal image data, performing fusion processing through a data fusion network to obtain the multi-modal data; based on the multi-modal data, extracting multi-dimensional gas feature data through the multi-modal feature extraction network; based on the multi-dimensional gas feature data, obtaining the multi-modal feature data through the multi-modal fully connected network.

4. The method of claim 3, wherein, Based on the current distance information and the multi-channel spatio-temporal image data, performing fusion processing through a data fusion network to obtain the multi-modal data comprises: based on the current distance information, acquiring a current distance image indicating distance information; based on the multiplication of the current distance image and the current gas mask image, obtaining a target gas mask image containing distance information; through a data fusion network, splicing the target gas mask image and the multi-channel spatio-temporal image data along the channel dimension to obtain the multi-modal data.

5. The method of claim 4, wherein, Based on the current distance information, acquiring a current distance image indicating distance information comprises at least one of the following: acquiring a current distance value represented by the current distance information and a weight matrix representing distance weights, multiplying the current distance value and the weight corresponding to each pixel point in the weight matrix to obtain the current distance image; or acquiring a depth map represented by the current distance information, and performing normalization processing on the depth map to obtain the current distance image.

6. The method of claim 1, wherein, The multi-channel spatio-temporal image data includes a current continuous frame image and a current gas mask image, the multi-modal fusion network includes a gas feature extraction network, a distance feature extraction network, a feature fusion network, and a full connection network, and based on the multi-channel spatio-temporal image data and the current distance information, the multi-modal fusion network in the pre-trained gas flow rate measurement model is used to obtain multi-modal feature data, which includes: Based on the multi-channel spatio-temporal image data, the gas feature extraction network is used to obtain gas feature data; Based on the current distance information, the distance feature extraction network is used to obtain distance feature data; The gas feature data and the distance feature data are fused by using the feature fusion network to obtain fused gas data; Based on the fused gas data, the full connection network is used for processing to obtain the multi-modal feature data.

7. The method of claim 1, wherein, The method further includes: Obtaining a training data set, each training sample in the training data set including a multi-channel spatio-temporal sample image, sample distance data, and a gas flow rate label corresponding to the training sample; Based on the multi-channel spatio-temporal sample image and the sample distance data, the multi-modal fusion network in the current training gas flow rate measurement model is used to obtain multi-modal sample feature data; Based on the multi-modal sample feature data, the flow rate detection network in the current training gas flow rate measurement model is used to output the current training gas flow rate; Based on the error between the current training gas flow rate and the gas flow rate label being less than a preset error value as the target, the current training gas flow rate measurement model is trained.

8. The method of claim 7, wherein, The training data set is formed by collecting multiple training samples of one gas or mixed by collecting training samples of multiple gases.

9. The method of claim 1, wherein, The method further includes at least one of the following: Real-time display of the current gas flow rate on the display image of the user interface; When the current gas flow rate is greater than or equal to a preset flow rate, an alarm warning is executed; According to the current leakage degree indicated by the current gas flow rate, the alarm warning corresponding to the current leakage degree is executed.

10. A cluster of computing devices, characterized in that, At least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute the instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method of any one of claims 1-9.

11. A computer readable storage medium, characterized in that, Computer program instructions, when executed by a computing device cluster, the computing device cluster executes the method of any one of claims 1-9.

12. A computer program product comprising instructions, characterized in that, When the instructions are executed by the computing device cluster, the computing device cluster executes the method of any one of claims 1-9.

Images