Gas edge enhancement method, training method of gas segmentation model, equipment and program product

Abstract

The invention discloses a gas edge enhancement method, equipment and a program product. The method comprises the following steps: acquiring a collected current continuous frame image; input image data of a pre-trained gas segmentation model is formed based on the current continuous frame images, the gas segmentation model comprises a gas feature extraction network and a gas segmentation network, and the gas feature extraction network is obtained based on migration of a gas feature generation network in a pre-trained gas target detection model; based on the input image data, utilizing the gas feature extraction network to extract gas feature data, and based on the gas feature data, performing segmentation through the gas segmentation network to obtain gas region data.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a gas edge enhancement method, a gas segmentation model training method, a gas detection device, a computing device, and a computer program product. Background Art

[0002] In recent years, with the rapid development of industry and the acceleration of urbanization, the demand for gas detection and monitoring has continued to increase in the fields of industry, environment, and safety monitoring. Traditional gas edge enhancement methods typically rely on specific sensors, but their effectiveness is limited in complex backgrounds and diverse scenarios. Therefore, a gas edge enhancement algorithm suitable for complex backgrounds and diverse scenarios is needed. It can accurately and quickly detect multiple gases and intuitively present the leakage range and gas plume shape.

[0003] Gas detection technology currently faces numerous challenges. First, sensors are prone to aging and drift over long periods of use, resulting in inaccurate detection results and requiring frequent calibration or replacement. Second, in multi-gas detection scenarios, the mixing of different gases and complex environmental conditions increase the difficulty of detection. Traditional methods struggle to effectively distinguish and locate multiple gases, resulting in inaccurate detected gas areas or unclear gas area boundaries. Summary of the Invention

[0004] In order to solve the existing technical problems, the present invention provides a gas edge enhancement method, a gas segmentation model training method, a gas detection device, a computing device and a computer program product, which can achieve accurate segmentation of the gas leakage range.

[0005] In a first aspect, a gas edge enhancement method is provided, comprising: acquiring a current continuous frame image acquired; forming input image data of a pre-trained gas segmentation model based on the current continuous frame image, wherein the gas segmentation model comprises a gas feature extraction network and a gas segmentation network, and the gas feature extraction network is obtained by migrating a gas feature generation network in a pre-trained gas target detection model; based on the input image data, extracting gas feature data using the gas feature extraction network, and based on the gas feature data, segmenting the gas region data through the gas segmentation network.

[0006] In a second aspect, an intelligent path planning method based on a physical information neural network is provided, the method comprising: obtaining a first training data set and a second training data set; constructing an initial gas target detection model, the gas target detection model comprising a gas feature generation network and a gas detection network; based on the first training data, iteratively training the gas target detection model to obtain a pre-trained gas target detection model; constructing an initial gas segmentation model, the gas segmentation model comprising a gas feature extraction network and a gas segmentation network, wherein the gas feature extraction network has the same network structure as the gas feature generation network; migrating the weight coefficient of the gas feature generation network in the pre-trained gas target detection model to the gas feature extraction network to obtain an updated gas segmentation model; based on the second training data set, iteratively training the updated gas segmentation model to obtain a pre-trained gas segmentation model.

[0007] In a third aspect, a gas detection device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the gas edge enhancement method provided in the first aspect of the present application.

[0008] In a fourth aspect, a computing device is provided, comprising a storage device and a processing device, wherein the storage device stores a computer program, and when the computer program is executed by the processing device, the processing device executes the training method for the gas segmentation model provided in the second aspect.

[0009] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed, implements the gas edge enhancement method provided in the first aspect of this application, and / or the gas segmentation model training method provided in the second aspect.

[0010] This application migrates the parameters of the gas feature generation network in the pre-trained gas target detection model to the gas feature extraction network, and then trains the gas segmentation model based on the second training data set, with the gas mask label as the training target, so that the gas segmentation model can learn the precise position coordinate features in the gas mask label. After the training is completed, the gas segmentation model can output the gas mask position data more accurately, thereby realizing the precise segmentation of the gas leakage range. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A diagram showing an application environment of a gas edge enhancement method in one embodiment;

[0012] Figure 2 is a flow chart of a gas edge enhancement method in one embodiment;

[0013] Figure 3is a flow chart of a method for training a gas segmentation model in one embodiment;

[0014] Figure 4 A schematic diagram of a gas edge enhancement method and apparatus according to an embodiment;

[0015] Figure 5 is a schematic diagram of a training device for a gas segmentation model in one embodiment;

[0016] Figure 6 A schematic diagram of a computing device according to an embodiment. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the scope of protection of the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0019] In the following description, reference is made to “some embodiments” which describe a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0020] See Figure 1 , which illustrates the application environment of a gas edge enhancement method according to one embodiment. The gas edge enhancement method is applied to a gas detection device 10, which includes an image acquisition device 12, a processor 13, and a memory 14. The image acquisition device 12 is used to capture continuous frame images in front of the gas detection device 10. The processor 13 detects gas regions within the continuous frame images captured by the image acquisition device 12. The memory 14 is used to store programs and data corresponding to the gas edge enhancement method.

[0021] The gas detection device 10 includes but is not limited to a handheld detection device, a non-handheld autonomously movable detection device, and a non-handheld non-autonomously movable detection device. The handheld detection device includes but is not limited to a handheld imaging device with an infrared thermal imaging function and a handheld imaging device with a visible light imaging function. The non-handheld autonomously movable detection device includes but is not limited to an autonomously movable detection device with an infrared thermal imaging function and an autonomously movable detection device with a visible light imaging function. The non-handheld non-autonomously movable detection device includes but is not limited to a non-handheld and non-autonomously movable device with an infrared thermal imaging function and a non-handheld and non-autonomously movable device with a visible light imaging function. The handheld detection device and the non-handheld autonomously movable detection device can be used by the user to perform gas detection on the current scene in motion during exercise, or to perform gas detection on a fixed scene. The non-handheld non-autonomously movable detection device can be used to perform gas detection on a fixed scene. Therefore, the gas edge enhancement method provided in the present application can be applied to various complex motion-changing scenes and fixed scenes.

[0022] The image acquisition device 12 can be a combination of one or more sensors. It can be a monocular vision sensor or a multi-camera vision sensor. For example, it can be a combination of one or more of a thermal imaging sensor, a visible light image sensor, a millimeter wave sensor, a lidar sensor, an infrared thermal imaging sensor, and a depth sensor. For example, the image acquisition device 12 uses a high-resolution infrared image acquisition device, such as a handheld infrared imaging device or a pan / tilt head, to acquire clear image data.

[0023] The processor 13 may be one or more processors. When there are multiple processors 13, the multiple processors may be integrated on a single chip or independently provided on each chip. The gas detection device 10 is a device installed on any type of mobile object, such as a vehicle, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobile device, an airplane, a drone, a ship, or a robot. The gas detection device 10 may also be fixedly installed on a fixed device or at a fixed location in a fixed scene.

[0024] The gas detection device 10 may also include other sensor modules, including but not limited to environmental perception sensors and motion posture sensors. Environmental perception sensors include but are not limited to one or more combinations of the following sensors: brightness sensors, temperature sensors, haze sensors, and other environmental sensors. Motion posture sensors include but are not limited to one or more combinations of the following: inertial measurement units (IMUs), velocity sensors, acceleration sensors, gyroscope sensors, geomagnetic sensors, rotation vector sensors, steering wheel angle sensors, level sensors, tilt sensors, vibration sensors, displacement sensors, and gravity sensors.

[0025] The gas detection device 10 may further include a display terminal for displaying images.

[0026] Current gas detection technology still faces many challenges in existing technologies. First, sensors are prone to aging and drift during long-term use, resulting in inaccurate detection results and requiring frequent calibration or replacement. Second, in multi-gas detection scenarios, the mixture of different gases and complex environmental conditions increase the difficulty of detection, and traditional methods are difficult to effectively distinguish and locate multiple gases. In addition, it is difficult to obtain a large number of real gas leakage data sets, especially in complex backgrounds and multiple scenarios. This limits the training and generalization capabilities of the model. When combining deep learning, simulation technology and traditional image processing methods, compatibility and integration issues between different technologies also need to be resolved. Finally, issues such as system architecture optimization, signal chain design, power consumption considerations, and electromagnetic compatibility also pose challenges to the performance and cost of gas detection systems.

[0027] See also Figure 2 , is a flow chart of a gas edge enhancement method provided in one embodiment of the present application. A gas edge enhancement method is applied to a gas detection device, and the gas edge enhancement method includes the following steps:

[0028] S11. Acquire the currently acquired continuous frame images.

[0029] In this embodiment, the current continuous frame images are images captured for the current scene. The current scene can be a static scene or a dynamic scene. The current continuous frame images can be continuous frame infrared images, continuous frame infrared images and continuous frame visible light images, or an image fused from continuous frame infrared images and continuous frame visible light images.

[0030] S12. Form input image data of a pre-trained gas segmentation model based on the current continuous frame images, wherein the gas segmentation model includes a gas feature extraction network and a gas segmentation network, and the gas feature extraction network is obtained by migrating the gas feature generation network in the pre-trained gas target detection model.

[0031] In this embodiment, the gas segmentation model is used to segment the gas region's location data and the gas region's mask area from consecutive image frames. The gas target detection model is capable of identifying the gas region's location data from consecutive image frames. First, a target detection model is trained as a pre-trained model. Based on this pre-trained model, a gas segmentation model is further trained to accurately segment the gas leak range.

[0032] The gas signature generation network and the gas signature extraction network share the same network structure and are both used to extract gas signature data. First, a gas target detection model is trained to obtain a pre-trained gas target detection model. The parameters of the gas signature generation network in the pre-trained gas target detection model are then transferred to the gas signature extraction network. The gas segmentation model is then trained to obtain a pre-trained gas segmentation model. The parameters include, but are not limited to, weight coefficients.

[0033] S13. Based on the input image data, the gas feature extraction network is used to extract the gas feature data. Based on the gas feature data, the gas segmentation network is used to segment and obtain the gas region data.

[0034] In this embodiment, a gas feature extraction network extracts gas feature data from input image data. This gas feature data is then used as input to a gas segmentation network, which outputs gas region data indicating the gas plume. The gas region data includes gas region location data and gas mask location data. During gas detection, sequential image frames are first fused into a multi-channel image to capture the motion characteristics of the gas. A pre-trained gas segmentation model and adaptive learning methods are then used to detect and locate gases based on their characteristics. The gas segmentation model is then used to segment the edges of the gas, more accurately highlighting its location.

[0035] In the above embodiment, the current continuous frame images collected are obtained, and input image data of a pre-trained gas segmentation model is formed based on the current continuous frame images. Gas feature data is extracted through the gas feature extraction network in the pre-trained gas segmentation model. Based on the gas feature data, gas area data is segmented through the gas segmentation network in the pre-trained gas segmentation model; the parameters of the gas feature generation network in the pre-trained gas target detection model are transferred to the gas feature extraction network, and the gas segmentation model is trained again, which can improve the accuracy of precise segmentation of the gas leakage range.

[0036] In some embodiments, the gas target detection model is trained based on a first training dataset, and the gas segmentation model is trained based on a second training dataset. Each training sample in the first training dataset includes a sample image and a gas target label corresponding to the sample image. Each training sample in the second training dataset includes a training image and a segmentation label corresponding to the training image. The segmentation label includes a gas region label and a gas mask label corresponding to the gas region. The gas region label and the gas target label include location data of the gas region. The gas target label is used to indicate the location data of the gas region, such as the center point of the boundary of the gas region.

[0037] In this embodiment, the segmentation label has one more label than the gas target label, namely the gas mask label, which can indicate the precise edge of the gas area. The segmentation label can be the label used by yolov5-seg, which can segment the contour at the pixel level to accurately locate the gas edge position. For example, the segmentation label is represented as follows: 0 0.5 0.5 0.31250.3125 0.4375 0.4167 0.5625 0.4167 0.56250.625 0.4375 0.625, where 0 represents the target category index (for example, category 0 represents "gas") and [0.5 0.5] represents the normalized coordinates of the center point of the bounding box. [0.31250.3125] represents the normalized width and height of the bounding box. [0.4375 0.4167 0.5625 0.4167 0.5625 0.625 0.4375 0.625]: Coordinates of the polygon vertices corresponding to the gas mask.

[0038] In this embodiment, a target detection model is first trained based on the first training data set as a pre-training model. On the basis of the pre-training model, the parameters of the gas feature generation network in the pre-trained gas target detection model are transferred to the gas feature extraction network, and then the gas segmentation model is trained based on the second training data set. This can improve the accuracy of precise segmentation of the gas leakage range. Moreover, in the process of training the gas segmentation model based on the second training data set, the gas mask label is used as the training target, so that the gas segmentation model can learn the precise position coordinate features in the gas mask label. After the training is completed, the gas segmentation model can output the gas mask position data more accurately, thereby realizing precise segmentation of the gas leakage range.

[0039] In the above embodiment, the parameters of the gas feature generation network in the pre-trained gas target detection model are migrated to the gas feature extraction network, and then the gas segmentation model is trained based on the second training data set, with the gas mask label as the training target. This allows the gas segmentation model to learn the precise position coordinate features in the gas mask label. After the training is completed, the gas segmentation model can output the gas mask position data more accurately, thereby achieving accurate segmentation of the gas leakage range.

[0040] In some embodiments, the training samples in the first training data set are obtained based on scene images collected in real scenes, and the training samples in the second training data set are obtained based on simulation software.

[0041] In this example, a large number of real-world datasets are first collected to generate a first training dataset, which is then used to train an object detection model as a pre-trained model. Simulation software is then used to generate a large amount of gas data and generate corresponding segmentation labels to obtain a second training dataset. Next, based on the pre-trained model, a gas segmentation model is further trained to achieve accurate segmentation of gas leak areas.

[0042] In this example, we first need to collect a large amount of real-world video data. This data should cover a variety of scenarios in industrial, environmental, and safety monitoring to ensure data diversity and representativeness. The data should include the movement and distribution of various gases to enable comprehensive model training. The collected data requires preprocessing, including sampling, resolution adjustment, and annotation, to ensure data quality and consistency, preparing for subsequent model training and evaluation.

[0043] Leveraging the leading real-time 3D engine UE5 and professional scientific computing software Ansys Fluent, large-scale, high-fidelity simulation data is generated. This simulated data is richer in diversity, volume, and complexity than real-world data, effectively complementing the shortcomings of real-world data. Leveraging the inherent features of these software, segmentation labels can be automatically generated, significantly reducing costs and time. This creates a highly diverse secondary training dataset, providing a rich data resource for subsequent model training. The advantage of this dataset construction method lies in its automatic, rapid, zero-cost, and completely accurate labeling. In contrast, real-world labeling requires manual labor, which is expensive, time-consuming, and prone to errors. Therefore, simulation allows for the acquisition of more high-quality data at a low cost.

[0044] In the above embodiment, by training the gas target detection model with data from real scenes, the gas feature generation network in the gas target detection model can learn more gas features from real scenes, and subsequently more accurately extract gas features from real scenes. By migrating the parameters of the gas feature generation network in the pre-trained gas target detection model to the gas feature extraction network, and then training the gas segmentation model based on a second training dataset, using gas mask labels as training targets, the gas segmentation model can learn the precise position coordinate features in the gas mask labels. After training, the gas segmentation model can more accurately extract gas areas from real scenes and more accurately output gas mask position data, achieving precise segmentation of the gas leakage range.

[0045] In some embodiments, the gas region data includes gas mask position data, and the method further comprises:

[0046] Gas target edge data is determined based on the gas mask position data and the current continuous frame images.

[0047] In this embodiment, the gas mask position data includes but is not limited to the position coordinates of the mask edge. Based on the gas mask position data and the current continuous frame image, the gas edge is further accurately extracted to obtain the target edge data.

[0048] Optionally, the determining gas target edge data based on the gas mask position data and the current continuous frame image includes:

[0049] Based on the gas mask position data, acquiring a gas region image of a current frame and a gas region image in a preset frame before the current frame from the current continuous frame images;

[0050] The gas region image in the current frame and the gas region image in the previous preset frame are differentially processed using a frame difference method to obtain frame difference image data, and the gas target edge data is obtained based on the frame difference image data.

[0051] In this embodiment, the position of the gas region image in the current frame and the position of the gas region image in the previous preset frame are identical to the positions indicated by the gas mask position data. Gas diffuses, and the frame difference method can be used to further accurately extract the motion characteristics of the gas. Based on these motion characteristics, the gas diffusion region can be precisely determined. In other words, the frame difference method can supplement the gas mask position data segmented by the gas segmentation model. The motion of the diffused gas typically varies little between frames, while the motion of the background rigid body typically varies significantly. This frame difference method can generate frame difference image data containing the gas motion characteristics. By setting a grayscale threshold, the gas diffusion region can be extracted from the frame difference image data while ignoring the background rigid body and noise, resulting in a more accurate gas distribution and a more precise boundary.

[0052] Optionally, the determining gas target edge data based on the gas mask position data and the current continuous frame image includes:

[0053] Based on the gas mask position data, acquiring a current frame gas region image from a current frame in the current continuous frame images;

[0054] Calculating gas grayscale distribution data based on the current frame gas region image;

[0055] Based on the gas grayscale distribution data, a void area is determined, and the void area is filtered out from the current gas area to obtain the gas target edge data.

[0056] In this embodiment, a grayscale statistics method is used to calculate the grayscale data in the gas region image of the current frame to obtain gas grayscale distribution data. This gas grayscale distribution data includes, but is not limited to, the gas grayscale range and the average grayscale of the gas region image of the current frame. Gases are spatially disturbed and are inevitably non-uniform, with regions of very low concentration within them. A minimum and maximum grayscale thresholds can be configured based on the gas grayscale range to filter out grayscale regions below the minimum grayscale threshold to eliminate low-concentration regions. Gases typically absorb energy. When the background is uniform, the grayscale distribution of the gas may be concentrated in a single region. Using a histogram, a relatively concentrated gas distribution can be obtained. Regions exceeding this distribution value are likely not areas of high gas concentration. Therefore, grayscale regions above the maximum grayscale threshold can be filtered out to obtain more accurate texture information. For a uniform background, the grayscale statistics method can be a histogram.

[0057] It is understood that when the steps of filtering the void regions and further extracting the gas edge positions using the frame difference method are performed simultaneously in one embodiment, they can be performed in no particular order, or one can be performed before the other. Alternatively, only one of them can be performed in one embodiment.

[0058] In the above embodiment, a gas segmentation model is used to obtain gas mask position data for the gas plume and leakage range. The gas leak location in the current frame is then extracted based on this gas mask position data. In the area corresponding to the gas mask position data, the gas and its edge features are clearly visible, and the gas outer edge can be identified by the edge of the gas mask. To further accurately extract the gas edge and highlight the gas, the gas motion and color characteristics can be further extracted. The gas motion characteristics are extracted using the frame difference method. By statistically analyzing the grayscale value distribution of the gas and background, gas voids within the gas mask are identified, resulting in a clearer gas distribution mask and, consequently, a more defined gas region edge.

[0059] In some embodiments, the method further comprises:

[0060] Based on the gas target edge data or the gas mask position data, a current target gas region image is determined, and the current target gas region image is enhanced to obtain an enhanced gas region in a current frame.

[0061] In this embodiment, based on the gas target edge data, an image of the gas region indicated by the gas target edge data is acquired from the current frame to obtain a current target gas region image. The size of the current target gas region image is the same as the size of the region indicated by the gas target edge data.

[0062] Optionally, the enhancing the current target gas region image to obtain an enhanced gas region in the current frame includes:

[0063] Calculating background distribution data and gas grayscale distribution data based on the current target gas area image;

[0064] When a result indicating that the grayscale of the current target gas region image is white is determined based on the background distribution data and the gas grayscale distribution data, a first gain coefficient is calculated, and the grayscale value in the current target gas region image is increased according to the first gain coefficient to obtain an enhanced gas region in the current frame;

[0065] Based on the background distribution data and the gas grayscale distribution data, when it is determined that the grayscale of the current target gas area image is dark, a second gain coefficient is calculated. According to the second gain coefficient, the grayscale value in the current target gas area image is reduced to obtain the enhanced gas area in the current frame.

[0066] In this embodiment, background grayscale data can be extracted based on the position of the current target gas region image in the current frame, and background distribution data can be statistically obtained using a histogram. Using the histogram, gas grayscale distribution data is statistically obtained for the current target gas region image. Background distribution data includes, but is not limited to, the background grayscale range and the background average grayscale. Gas grayscale distribution data includes, but is not limited to, the gas grayscale range and the gas average grayscale. When the gas average grayscale is greater than or equal to the background average grayscale, the grayscale of the current target gas region image is determined to be white. When the gas average grayscale is less than the background average grayscale, the grayscale of the current target gas region image is determined to be dark. The first gain coefficient is greater than the second gain coefficient, and the first gain coefficient is greater than 1, while the second gain coefficient is less than 1. The first and second gain coefficients can be determined based on the current grayscale difference between the gas average grayscale and the background average grayscale. For example, a gain coefficient corresponding to each grayscale difference range can be preconfigured, and a matching grayscale difference range can be determined based on the current grayscale difference, and then the corresponding gain coefficient can be determined. Of course, other methods of calculating the gain coefficients are also possible. If the grayscale of the current target gas region image is too white, the grayscale of each pixel in the current target gas region image is multiplied by the first gain coefficient based on the first gain coefficient to obtain the enhanced gas region in the current frame. If the grayscale of the current target gas region image is too dark, the grayscale of each pixel in the current target gas region image is multiplied by the second gain coefficient based on the second gain coefficient to obtain the enhanced gas region in the current frame. The entire current target gas region image is multiplied by a single coefficient, thereby highlighting the entire gas region position, thereby naturally highlighting the gas edge.

[0067] Because there are many types of gases to be detected, the gas characteristics are diverse, and the scenes are diverse, different grayscale features are easily presented in the video. For example, the absorption characteristics of the gas for infrared radiation in a specific band usually make it appear in thermal imaging with a grayscale value lower than the surrounding background; however, when the gas temperature is higher than the ambient background, based on the principle of blackbody radiation, its thermal radiation intensity is higher than the background. At this time, the gas will appear as a bright white thermal anomaly area in the infrared image. In order to better accurately determine the range of the gas, the gas grayscale distribution can be used to further refine the gas characteristics. In addition, to highlight the gas edge, when the grayscale of the current target gas area image is dark, the grayscale value in the current target gas area image is reduced, so that the darker gas area is darker. When the grayscale of the current target gas area image is white, the grayscale value in the current target gas area image is increased, so that the whiter gas area is whiter, further enhancing the contrast between the gas and the environment.

[0068] In the above embodiment, taking into account that the gas may present different grayscale value distributions due to factors such as temperature, the average grayscale distribution of the background and the gas is statistically analyzed to determine whether the gas grayscale is white or black, and a gain coefficient is calculated. According to the gain coefficient, the grayscale of the pixels in the gas area is changed to enhance the gas edge, and an image after gas edge enhancement is obtained, which makes the gas edge clearer, making it easier for users to intuitively understand the gas distribution and leakage range.

[0069] like Figure 3 As shown, Figure 3 Flowchart of a method for training a gas segmentation model in one embodiment. The method for training a gas segmentation model is applied to a computing device. The method includes the following steps:

[0070] S31: Obtain a first training data set and a second training data set.

[0071] In this embodiment, the first training data set is a labeled image data set containing different gases, which is used to train the gas target detection model. The second training data set is also an image data set containing different gases and multiple labels, which is used to train the gas segmentation model. For the first training data set or the second training data set, it can be divided into a training set and a validation set to prepare for model training. The gas target detection model is used for training, and appropriate hyperparameters, learning rates and batch sizes are set to ensure that the model can effectively learn the features in the data. During the training process, data enhancement and regularization operations are performed to improve the generalization ability and robustness of the model. Continuous multi-frame data is used as model input, and time enhancement and grayscale enhancement are used to improve the diversity and robustness of the model, providing a pre-training basis for subsequent segmentation models. The data enhancement and regularization operations include but are not limited to grayscale transformation, moasic, etc. These methods can greatly increase the training effect. The sample images are continuous frame sample images, and the training images are continuous frame training images. Time enhancement can be achieved through continuous frames.

[0072] Furthermore, parameter tuning and optimization are necessary to improve the efficiency and accuracy of the algorithm. For example, cross-validation is used to determine hyperparameters for the gas target detection and gas segmentation models, such as convolution kernel size and number of layers. Furthermore, data augmentation techniques, such as translation, rotation, and scaling, are applied to expand the training dataset and improve the model's robustness and generalization capabilities.

[0073] S32. Construct an initial gas target detection model, which includes a gas feature generation network and a gas detection network.

[0074] S33. Iteratively train the gas target detection model based on the first training data to obtain a pre-trained gas target detection model.

[0075] In this embodiment, each training sample in the first training dataset includes a sample image and a corresponding gas target label. During training, the gas target detection model in the current iteration outputs current gas identification data corresponding to the sample image. Based on the loss between the current gas identification data and the corresponding gas target label, if the loss value does not meet the iteration termination condition, further sample images are acquired for training until a gas target detection model that meets the iteration termination condition, i.e., a pre-trained gas target detection model, is obtained. The iteration termination condition includes, but is not limited to, the number of iterations being less than a preset number and the loss value being less than a preset loss threshold.

[0076] S34. Construct an initial gas segmentation model, which includes a gas feature extraction network and a gas segmentation network, wherein the network structure of the gas feature extraction network is the same as that of the gas feature generation network.

[0077] S35. Migrate the weight coefficients of the gas feature generation network in the pre-trained gas target detection model to the gas feature extraction network to obtain an updated gas segmentation model.

[0078] In this embodiment, the network results of the gas feature extraction network and the gas feature generation network may include a backbone and neck structure. Specifically, a gas segmentation model with the same backbone and neck structure as the gas target detection model is constructed, and the detection head of the gas target detection model is replaced with a segmentation head to meet the requirements of the segmentation task, thereby obtaining a gas segmentation model. The weights of the gas feature generation network in the trained target detection model are assigned to the gas feature extraction network to obtain an initial gas segmentation model. This enables the gas segmentation model to have a good extraction effect on gas features in real-world scenarios, thereby improving the model's initial performance and training efficiency. The feature extraction part has the same network structure, so that during model migration, the network structure of the gas target detection model can be migrated to the gas segmentation model. The training labels of the gas target detection model and the gas segmentation model are different.

[0079] S36. Iteratively train the updated gas segmentation model based on the second training data set to obtain a pre-trained gas segmentation model.

[0080] In this example, the gas segmentation model is trained using the second training dataset, employing a similar training approach to the object detection model to ensure that the model effectively learns the features in the data. Methods such as temporal and grayscale enhancement are used to improve the model's diversity and robustness. Multi-frame input enhances the model's ability to extract motion features, thereby improving its performance in complex scenarios.

[0081] Each training sample in the second training data set includes a training image and a segmentation label corresponding to the training image, wherein the segmentation label includes a gas region label and a gas mask label corresponding to the gas region. The gas region label identifies the location data of the gas region. The gas mask label indicates the edge data of the gas region. During the training process, the current gas segmentation data corresponding to the training image is output by the gas segmentation model in the current iteration. Based on the first loss between the gas position data in the current gas segmentation data and the gas region label corresponding to the training image, and based on the second loss between the gas mask position data in the current gas segmentation data and the gas mask label corresponding to the training image, a total loss value is calculated based on the first loss and the second loss. When the total loss value does not meet the iterative training condition, training images are continuously acquired for training until a gas segmentation model that meets the iterative termination condition is obtained, that is, a pre-trained gas segmentation model. The iterative training condition includes but is not limited to the number of iterations being less than a preset number and the total loss value being less than a preset loss threshold.

[0082] In some embodiments, for the first training data set or the second training data set, the data set can be classified according to the scene to obtain different scene data sets. For each different scene data set, a gas target detection model is trained separately to obtain a gas target detection model for each different scene. Then, the gas target detection models for each different scene are weighted fused to obtain the gas target detection model of the present application. Similarly, for the gas segmentation model, a gas segmentation model is trained separately for each different scene data set to obtain a gas segmentation model for each different scene. Then, the gas segmentation models for each different scene are weighted fused to obtain the gas segmentation model of the present application. Taking into account the influence of different scenes and complex backgrounds, classification is performed in the data collection stage, and a corresponding model is trained for each scene. By using real data sets and simulated data sets, the adaptability and robustness of the model in different scenes are ensured. In addition, through multi-model fusion and data enhancement strategies, the generalization ability of the model is further improved, so that the model can better adapt to gas detection needs in complex backgrounds and multiple scenes.

[0083] In some embodiments, the pixels in the gas area in the current frame gas area image can also be colored according to the grayscale of the pixels in the current frame gas area image and a preconfigured grayscale-color mapping relationship to facilitate intuitive display of the gas area.

[0084] In some embodiments, a gas edge enhancement method may include: S1: Acquisition and preprocessing of a real gas dataset; S2: Training a gas target detection model; S3: Production of a simulated gas dataset; S4: Construction and migration of a gas segmentation model; S5: Training the gas segmentation model; S6: Extraction of gas features; and S7: Gas edge enhancement. The specific steps of S1-S7 have been described in one or more of the above embodiments and will not be repeated here. The present embodiments implement gas detection and edge enhancement in complex backgrounds and various scenarios. The main idea is to train a gas target detection model using a large number of real datasets as a pre-trained model. Simulation software is then used to generate a large amount of gas data and corresponding segmentation labels. Based on the pre-trained model, a gas segmentation model is further trained to achieve accurate segmentation of the gas leak range. After segmentation, based on the gas detection results, gas edges are extracted using traditional image processing methods, such as frame difference, filtering, and histograms. The edges of the gas regions are then enhanced by multiplying the grayscale of the pixels in the gas region by a gain coefficient. Through the above steps, the present method can achieve gas detection and edge enhancement in complex backgrounds and various scenarios. It combines deep learning, simulation technology and traditional image processing methods to improve the accuracy and robustness of gas detection, and through intuitive edge-enhanced display, enables users to clearly understand the distribution and leakage range of gas.

[0085] In combination with one or more embodiments of the present application, the present application has at least the following features:

[0086] 1. First, a large number of real-world datasets are collected to train an object detection model as a pre-trained model. Then, using simulation software such as UE5 and Ansys Fluent, a large amount of gas data is generated and corresponding segmentation labels are generated. Based on the pre-trained model, a gas segmentation model is further trained to accurately segment the gas leak area. After segmentation, traditional methods such as image processing are used to extract gas edges. The edges of the gas area are enhanced by multiplying the grayscale of the pixels in the gas area by a gain factor. This method effectively highlights the boundaries of the gas area, facilitates clear definition of the gas leak area, and improves the accuracy and robustness of gas detection.

[0087] 2. By combining preprocessing, feature extraction, gas detection and location, and colorization, and combining image processing and deep learning techniques, we developed a multi-gas detection and colorization algorithm suitable for complex backgrounds and diverse scenarios. This method has broad application prospects and can play an important role in fields such as industrial, environmental, and safety monitoring.

[0088] On the other hand, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the gas edge enhancement method described in any embodiment of the present application.

[0089] Among them, in the computer program product, the optional implementation form of the program module architecture of the computer program that implements each step of the gas edge enhancement method can be a gas edge enhancement method device. Figure 4 An embodiment of the present application provides a gas edge enhancement method and device, including: an acquisition module 41, used to acquire the current continuous frame image collected; a segmentation module 42, used to form input image data of a pre-trained gas segmentation model based on the current continuous frame image, wherein the gas segmentation model includes a gas feature extraction network and a gas segmentation network, and the gas feature extraction network is obtained by migrating the gas feature generation network in the pre-trained gas target detection model; the segmentation module 42 is also used to extract gas feature data based on the input image data using the gas feature extraction network, and based on the gas feature data, obtain gas area data through segmentation through the gas segmentation network.

[0090] Optionally, the gas target detection model is trained based on a first training data set, and the gas segmentation model is trained based on a second training data set, each training sample in the first training data set includes a sample image and a gas target label corresponding to the sample image, and each training sample in the second training data set includes a training image and a segmentation label corresponding to the training image, and the segmentation label includes a gas area label and a gas mask label corresponding to the gas area.

[0091] Optionally, the training samples in the first training data set are obtained based on scene images collected in a real scene, and the training samples in the second training data set are obtained based on simulation software.

[0092] Optionally, an edge enhancement module 43 is further included, configured to:

[0093] Gas target edge data is determined based on the gas mask position data and the current continuous frame images.

[0094] Optionally, the edge enhancement module 43 is further configured to:

[0095] Based on the gas mask position data, acquiring a gas region image of a current frame and a gas region image in a preset frame before the current frame from the current continuous frame images;

[0096] The gas region data of the current frame and the gas region data in the previous preset frame are differentially processed by using a frame difference method to obtain frame difference image data, and the gas target edge data is obtained based on the frame difference image data.

[0097] Optionally, the edge enhancement module 43 is further configured to:

[0098] Based on the gas mask position data, acquiring a current frame gas region image from a current frame in the current continuous frame images;

[0099] Calculating gas grayscale distribution data based on the current frame gas region image;

[0100] Based on the gas grayscale distribution data, a void area is determined, and the void area is filtered out from the current gas area to obtain the gas target edge data.

[0101] Optionally, the edge enhancement module 43 is further configured to:

[0102] Based on the gas target edge data or the gas mask position data, a current target gas region image is determined, and the current target gas region image is enhanced to obtain an enhanced gas region in a current frame.

[0103] Optionally, the edge enhancement module 43 is further configured to:

[0104] Calculating background distribution data and gas grayscale distribution data based on the current target gas area image;

[0105] When a result indicating that the grayscale of the current target gas region image is white is determined based on the background distribution data and the gas grayscale distribution data, a first gain coefficient is calculated, and the grayscale value in the current target gas region image is increased according to the first gain coefficient to obtain an enhanced gas region in the current frame;

[0106] Based on the background distribution data and the gas grayscale distribution data, when it is determined that the grayscale of the current target gas area image is dark, a second gain coefficient is calculated. According to the second gain coefficient, the grayscale value in the current target gas area image is reduced to obtain the enhanced gas area in the current frame.

[0107] In another aspect, the present application provides a computer program product, including a computer program, which, when executed by a processing device, implements the gas segmentation model training method described in any embodiment of the present application. In the computer program product, an optional implementation form of the program module architecture of the computer program that implements each step of the gas segmentation model training method may be a gas edge enhancement method device. Figure 5An embodiment of the present application provides a method and device for training a gas segmentation model, including: a data set acquisition module 51, used to acquire a first training data set and a second training data set; a construction module 52, used to construct an initial gas target detection model, wherein the gas target detection model includes a gas feature generation network and a gas detection network; a training module 53, used to iteratively train the gas target detection model based on the first training data to obtain a pre-trained gas target detection model; the construction module 52 is also used to construct an initial gas segmentation model, wherein the gas segmentation model includes a gas feature extraction network and a gas segmentation network, wherein the gas feature extraction network has the same network structure as the gas feature generation network; a migration module 54 is used to migrate the weight coefficient of the gas feature generation network in the pre-trained gas target detection model to the gas feature extraction network to obtain an updated gas segmentation model; the training module 53 is also used to iteratively train the updated gas segmentation model based on the second training data set to obtain a pre-trained gas segmentation model.

[0108] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, the gas edge enhancement method device and / or the gas segmentation model training method device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0109] The embodiments of the present application can, according to the above method, exemplarily divide the gas edge enhancement method device and / or the gas segmentation model training method device into functional modules. For example, the gas edge enhancement method device and / or the gas segmentation model training method device may include various functional modules corresponding to the various functional divisions, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0110] See also Figure 6On the other hand, an embodiment of the present application further provides a computing device 30, including a processing device 303 and a storage device 304. The storage device 304 stores a computer program. When the computer program is executed by the processor, the processing device 303 executes the steps of an intelligent path planning method based on a physical information neural network provided in any of the above embodiments of the present application.

[0111] The processing device 303 is the control center, connecting the various components of the gas detection device using various interfaces and lines. By running or executing software programs and / or modules stored in the storage device 304 and accessing data stored in the storage device 304, the processing device 303 performs various functions and processes data. Optionally, the processing device 303 may include one or more processing cores; preferably, the processing device 303 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interfaces, and application programs, and the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processing device 303.

[0112] The storage device 304 can be used to store software programs and modules. The processing device 303 executes various functional applications and data processing by running the software programs and modules stored in the storage device 304. The storage device 304 may mainly include a program storage area and a data storage area. The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created based on the use of a gas detection device, etc. In addition, the storage device 304 may include a high-speed random access memory and a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the storage device 304 may also include a memory processor to provide the processing device 303 with access to the storage device 304.

[0113] On the other hand, an embodiment of the present application further provides a computer-readable non-volatile storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of a gas edge enhancement method provided in any of the above embodiments of the present application, and / or the steps of a gas segmentation model training method provided in any of the embodiments.

[0114] On the other hand, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of a gas edge enhancement method as described in any embodiment of the present application, and / or a gas segmentation model training method provided in any embodiment.

[0115] On the other hand, an embodiment of the present application provides a vehicle device, including a controller and a storage unit, wherein the storage unit stores a computer program. When the computer program is executed by the controller, the controller executes the training method of the gas segmentation model provided in the second aspect of the present application.

[0116] Those skilled in the art will appreciate that all or part of the processes in the methods provided in the above embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0117] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. The scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A gas edge enhancement method, characterized in that: include: Get the current continuous frame image acquired; Forming input image data of a pre-trained gas segmentation model based on the current continuous frame images, wherein the gas segmentation model includes a gas feature extraction network and a gas segmentation network, wherein the gas feature extraction network is obtained by migrating a gas feature generation network in a pre-trained gas target detection model; Based on the input image data, the gas feature extraction network is used to extract gas feature data, and based on the gas feature data, the gas segmentation network is used to segment and obtain gas region data.

2. The gas edge enhancement method according to claim 1, wherein: The gas target detection model is trained based on a first training data set, and the gas segmentation model is trained based on a second training data set. Each training sample in the first training data set includes a sample image and a gas target label corresponding to the sample image. Each training sample in the second training data set includes a training image and a segmentation label corresponding to the training image. The segmentation label includes a gas area label and a gas mask label corresponding to the gas area.

3. The gas edge enhancement method according to claim 2, wherein: The training samples in the first training data set are obtained based on scene images collected in real scenes, and the training samples in the second training data set are obtained based on simulation software.

4. The gas edge enhancement method according to claim 1, wherein: The gas region data includes gas mask position data, and the method further includes: Gas target edge data is determined based on the gas mask position data and the current continuous frame images.

5. The gas edge enhancement method according to claim 4, wherein: The determining of gas target edge data based on the gas mask position data and the current continuous frame image includes: Based on the gas mask position data, acquiring a gas region image of a current frame and a gas region image in a preset frame before the current frame from the current continuous frame images; The gas region data of the current frame and the gas region data in the previous preset frame are differentially processed by using a frame difference method to obtain frame difference image data, and the gas target edge data is obtained based on the frame difference image data.

6. The gas edge enhancement method according to claim 4, wherein: The determining of gas target edge data based on the gas mask position data and the current continuous frame image includes: Based on the gas mask position data, acquiring a current frame gas region image from a current frame in the current continuous frame images; Calculating gas grayscale distribution data based on the current frame gas region image; Based on the gas grayscale distribution data, a void area is determined, and the void area is filtered out from the current gas area to obtain the gas target edge data.

7. The gas edge enhancement method according to claim 1 or 4, characterized in that: The method further comprises: Based on the gas target edge data or the gas mask position data, a current target gas region image is determined, and the current target gas region image is enhanced to obtain an enhanced gas region in a current frame.

8. The gas edge enhancement method according to claim 7, wherein: The enhancing the current target gas region image to obtain the enhanced gas region in the current frame includes: Calculating background distribution data and gas grayscale distribution data based on the current target gas area image; When a result indicating that the grayscale of the current target gas region image is white is determined based on the background distribution data and the gas grayscale distribution data, a first gain coefficient is calculated, and the grayscale value in the current target gas region image is increased according to the first gain coefficient to obtain an enhanced gas region in the current frame; Based on the background distribution data and the gas grayscale distribution data, when it is determined that the grayscale of the current target gas area image is dark, a second gain coefficient is calculated. According to the second gain coefficient, the grayscale value in the current target gas area image is reduced to obtain the enhanced gas area in the current frame.

9. A method for training a gas segmentation model, characterized in that: include: Obtaining a first training data set and a second training data set; Constructing an initial gas target detection model, wherein the gas target detection model includes a gas feature generation network and a gas detection network; Iteratively training the gas target detection model based on the first training data to obtain a pre-trained gas target detection model; Constructing an initial gas segmentation model, wherein the gas segmentation model includes a gas feature extraction network and a gas segmentation network, wherein the gas feature extraction network has the same network structure as the gas feature generation network; Migrating the weight coefficients of the gas feature generation network in the pre-trained gas target detection model to the gas feature extraction network to obtain an updated gas segmentation model; Based on the second training data set, the updated gas segmentation model is iteratively trained to obtain a pre-trained gas segmentation model.

10. The gas segmentation model training method according to claim 9, characterized in that: Each training sample in the first training data set includes a sample image and a gas target label corresponding to the sample image, and each training sample in the second training data set includes a training image and a segmentation label corresponding to the training image, and the segmentation label includes a gas area label and a gas mask label corresponding to the gas area.

11. The gas segmentation model training method according to claim 9, wherein: The training samples in the first training data set are obtained based on scene images collected in real scenes, and the training samples in the second training data set are obtained based on simulation software.

12. A gas detection device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the gas edge enhancement method according to any one of claims 1 to 8.

13. A computing device, characterized in that The method comprises a storage device and a processing device, wherein the storage device stores a computer program, and when the computer program is executed by the processing device, the processing device executes the training method of the gas segmentation model as claimed in any one of claims 9 to 11.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the gas edge enhancement method according to any one of claims 1 to 8 and / or the gas segmentation model training method according to any one of claims 9 to 11 are implemented.

Images