Gas hidden danger identification method and device

Abstract

The invention discloses a gas hidden danger identification method and device which are applied to the field of image processing, and the method comprises the steps: collecting a to-be-identified image of gas equipment; performing semantic segmentation on the to-be-recognized image to obtain a segmented to-be-recognized image; performing noise reduction processing on the segmented to-be-recognized image to obtain a noise-reduced to-be-recognized image; performing angle correction on the to-be-recognized image after noise reduction to obtain a corrected to-be-recognized image; performing image preprocessing on the corrected to-be-recognized image to obtain a preprocessed to-be-recognized image; and inputting the preprocessed to-be-recognized image into the visual model to obtain a hidden danger recognition result. The to-be-recognized image of the gas equipment is recognized through the visual model, a large number of gas equipment can be quickly detected, the detection time is remarkably shortened, and new hidden dangers are found in time.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for identifying gas hidden dangers. Background Art

[0002] Gas is widely used in households and industrial production, becoming a vital energy source. However, gas safety issues have also become prominent. Gas leaks and equipment failures can cause explosions, fires, and other accidents, threatening people's lives and property. Ensuring gas safety has become a key concern in society.

[0003] Currently, the identification of gas hazards primarily relies on manual inspections. While this method can detect gas safety hazards to a certain extent, it suffers from inefficiencies and poor accuracy, making it difficult to meet the safety requirements of large-scale areas. This is especially true in cities, where hundreds of thousands of residents and businesses are present. The time and manpower required to manually inspect each gas device individually is enormous, resulting in lengthy inspection cycles and a failure to identify new hazards in a timely manner.

[0004] Therefore, how to discover new hidden dangers in a timely manner has become an urgent problem to be solved in this field. Summary of the Invention

[0005] The present application provides a method and device for identifying gas hidden dangers, the purpose of which is to promptly discover new hidden dangers.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] A method for identifying hidden gas hazards, comprising:

[0008] Collect images of gas equipment to be identified;

[0009] Performing semantic segmentation on the image to be identified to obtain a segmented image to be identified;

[0010] performing noise reduction processing on the segmented image to be identified to obtain a noise-reduced image to be identified;

[0011] Performing angle correction on the image to be identified after noise reduction to obtain a corrected image to be identified;

[0012] Performing image preprocessing on the corrected image to be identified to obtain a preprocessed image to be identified;

[0013] The preprocessed image to be identified is input into a visual model to obtain a hidden danger identification result; the visual model is pre-trained based on a target sample image; the target sample image is pre-generated based on a sample gas equipment image and a rare sample image.

[0014] Optionally, performing semantic segmentation on the image to be identified to obtain a segmented image to be identified includes:

[0015] Extracting a device area in the image to be identified;

[0016] A segmented image to be recognized is generated based on the device area.

[0017] Optionally, performing image preprocessing on the corrected image to be identified to obtain a preprocessed image to be identified includes:

[0018] Performing image grayscale processing on the corrected image to be identified to obtain a grayscale image;

[0019] Performing filtering on the grayscale image to obtain a filtered grayscale image;

[0020] Normalization processing is performed on the filtered grayscale image to obtain a normalized grayscale image, and the normalized grayscale image is determined as the pre-processed image to be recognized.

[0021] Optionally, inputting the pre-processed image to be identified into a visual model to obtain a hidden danger identification result includes:

[0022] Inputting the preprocessed image to be recognized into the convolutional layer of the visual model to obtain a feature map;

[0023] Inputting the feature map into the pooling layer in the visual model to obtain a feature map after dimensionality reduction;

[0024] The feature map after dimensionality reduction is input into the fully connected layer in the visual model to obtain a hidden danger identification result.

[0025] Optionally, also include:

[0026] Obtain inspection text and gas usage;

[0027] Extracting the image to be identified using a residual network to obtain image features;

[0028] Input the inspection text into the BERT model to obtain a semantic vector of the inspection text;

[0029] Extracting the gas usage using a recurrent neural network to obtain time series features;

[0030] Mapping the image features, the semantic vector of the inspection text, and the temporal features into the same space to obtain multimodal data;

[0031] The graph attention network is used to infer the multimodal data according to the preset knowledge graph to obtain the probability of hidden danger chain reaction.

[0032] Optionally, the process of obtaining the visual model by pre-training based on the target sample image includes:

[0033] Acquire target sample image;

[0034] Inputting the target sample image into the visual model to obtain a sample hidden danger recognition result;

[0035] Calculating a loss function between the sample hidden danger identification result and the actual hidden danger identification result corresponding to the target sample image; the loss function at least includes a cross entropy loss function;

[0036] When the loss function does not converge, adjusting the model parameters of the visual model, and returning to the step of inputting the target sample image into the visual model to obtain a sample hidden danger identification result;

[0037] When the loss function converges, it is determined that the visual model training is completed.

[0038] Optionally, the process of pre-generating a target sample image based on the sample gas equipment image and the rare sample image includes:

[0039] Acquire sample gas equipment images; the sample gas equipment images include images of multiple hidden danger types; the hidden danger types include high-risk hidden dangers, medium-risk hidden dangers, and low-risk hidden dangers;

[0040] Setting a loss weight for the sample gas equipment image according to the hidden danger type;

[0041] After the loss weight is set, a conditional generative adversarial network is used to generate rare sample images based on the sample gas equipment images;

[0042] A target sample image is generated based on the sample gas appliance image and the rare sample image.

[0043] A device for identifying hidden gas hazards, comprising:

[0044] An acquisition unit, used for acquiring images of gas equipment to be identified;

[0045] a segmentation unit, configured to perform semantic segmentation on the image to be identified to obtain a segmented image to be identified;

[0046] a noise reduction unit, configured to perform noise reduction processing on the segmented image to be identified to obtain a noise-reduced image to be identified;

[0047] a correction unit, configured to perform angle correction on the image to be identified after noise reduction to obtain a corrected image to be identified;

[0048] a processing unit, configured to perform image preprocessing on the corrected image to be identified to obtain a preprocessed image to be identified;

[0049] The recognition unit is used to input the preprocessed image to be recognized into a visual model to obtain a hidden danger recognition result; the visual model is pre-trained based on a target sample image; the target sample image is pre-generated based on a sample gas equipment image and a rare sample image.

[0050] Optionally, the segmentation unit is specifically configured to:

[0051] Extracting a device area in the image to be identified;

[0052] A segmented image to be recognized is generated based on the device area.

[0053] Optionally, the processing unit is specifically configured to:

[0054] Performing image grayscale processing on the corrected image to be identified to obtain a grayscale image;

[0055] Performing filtering on the grayscale image to obtain a filtered grayscale image;

[0056] Normalization processing is performed on the filtered grayscale image to obtain a normalized grayscale image, and the normalized grayscale image is determined as the pre-processed image to be recognized.

[0057] The technical solution provided in this application involves collecting images of gas equipment to be identified; performing semantic segmentation on the images to be identified to obtain segmented images to be identified; performing noise reduction processing on the segmented images to be identified to obtain noise-reduced images to be identified; performing angle correction on the noise-reduced images to be identified to obtain corrected images to be identified; performing image preprocessing on the corrected images to be identified to obtain preprocessed images to be identified; and inputting the preprocessed images to be identified into a visual model to obtain hidden danger identification results. By using a visual model to identify images of gas equipment to be identified, a large number of gas equipment can be quickly detected, significantly shortening detection time and allowing for the timely discovery of new hidden dangers. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 A flow chart of a method for identifying gas hazards provided in an embodiment of the present application;

[0060] Figure 2 A flowchart of a target sample image training method provided in an embodiment of the present application;

[0061] Figure 3 A flow chart of a target sample image generation method provided in an embodiment of the present application;

[0062] Figure 4 A flowchart of a method for predicting the probability of a chain reaction of hidden dangers provided in an embodiment of the present application;

[0063] Figure 5 A schematic diagram of the architecture of a gas hazard identification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0065] In this application, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0066] like Figure 1 FIG. 1 is a flow chart of a method for identifying a gas hazard provided by the present application, comprising the following steps:

[0067] S101: Collect images of gas equipment to be identified.

[0068] The image to be identified must include key parts of the gas appliance and be clear and complete. For example, for a gas stove, the image must clearly show the burner, flameout protection device, and ignition device.

[0069] It is understandable that, at preset time intervals or when a detection instruction is received, an image to be identified of the gas equipment is collected by a collection device (such as a camera, etc., all with autofocus and anti-shake functions).

[0070] It should be noted that the installation method of the data collection device varies for different gas appliances. When the gas appliance is a gas water heater, the data collection device is installed on the side or below to ensure that the overall appearance of the water heater and the flue interface can be clearly photographed (for example, 50-80 cm from the side, flush with the flue interface, and supports angle adjustment); when the gas appliance is a gas meter, the data collection device is installed in a direction that can capture the gas meter dial and protective buckle position (for example, installed 1 meter in front, fixed with a magnetic base, and supports automatic white balance); when the gas appliance is a gas stove, the data collection device is installed above to fully capture the burner, flameout protection device, and other parts of the stove (for example, installed 30-50 cm below the wall cabinet, and vertically downward to capture the burner and flameout protection device).

[0071] In addition, low-quality image enhancement technology was used for the images to be identified, and a deblurring and deocclusion algorithm based on a generative adversarial network was developed to address misreadings of gas cooker meters caused by factors such as backlighting, reflections, and occlusion. Furthermore, a welding defect detection algorithm was developed, using a U-Net++ segmentation model to identify defects such as incomplete weld penetration, porosity, and cracks in outdoor pipeline welding images.

[0072] It should be noted that the image of the gas equipment to be identified is transmitted via wireless (such as Wi-Fi, 5G) or wired (such as Ethernet) transmission methods for subsequent image processing. During the transmission process, the image data is encrypted to prevent data leakage and tampering, ensuring the security of data transmission.

[0073] S102: Perform semantic segmentation on the image to be recognized to obtain a segmented image to be recognized.

[0074] The segmented image to be identified is an image in which background interference is filtered out and only contains gas equipment.

[0075] Optionally, the image to be identified can be input into the DeepLabv3+ model, and the image to be identified can be semantically segmented for gas equipment to generate a pixel-level mask of the gas equipment, thereby obtaining the segmented image to be identified.

[0076] Optionally, in another embodiment of the present application, the specific implementation process of step S102 is: extracting the device area in the image to be identified; and generating a segmented image to be identified based on the device area.

[0077] The equipment area is the area containing gas equipment.

[0078] S103: performing noise reduction processing on the segmented image to be recognized to obtain a noise-reduced image to be recognized.

[0079] It can be understood that the segmented image to be identified is subjected to denoising processing. Specifically, when the segmented image to be identified is an RGB image, bilateral filtering is used to smooth the B channel of the segmented image to be identified, retaining the metallic gloss texture of the flue of the gas equipment, and obtaining the denoised image to be identified; when the segmented image to be identified is a grayscale image, adaptive median filtering is used to perform denoising processing on the segmented image to be identified, and the window size is dynamically adjusted according to the local noise density to obtain the denoised image to be identified.

[0080] S104: performing angle correction on the image to be recognized after noise reduction to obtain a corrected image to be recognized.

[0081] Specifically, by detecting edge feature points of the gas equipment in the de-noised image to be identified and using a homography to perform angle correction, the tilted image can be corrected to a normal view. The angle measurement error of the flue in the corrected image does not exceed 1.5°.

[0082] In addition, the denoised image to be recognized can be angle-corrected using a spatial transformer network based on deformable convolution. Specifically, the network architecture of the spatial transformer network based on deformable convolution mainly includes a positioning network, a deformable convolution layer, and a loss function.

[0083] Localization Network: This network consists of five convolutional layers, each of which includes convolution, batch normalization, and ReLU activation functions. The output of the network is the deformation parameter θ, which includes a 6-dimensional affine transformation matrix and local offsets to describe the spatial transformation.

[0084] Deformable Convolution Layer: Deformable convolution is introduced in the grid generator of the spatial transformer network. Deformable convolution can dynamically learn local deformations, allowing the network to adjust the sampling position according to the characteristics of the input, thereby performing spatial transformation more flexibly.

[0085] Loss Function: To prevent excessive deformation distortion, a deformation smoothness constraint is incorporated into the loss function, and the deformation offset is constrained using an L2 regularization term. Furthermore, a text region alignment loss is employed to align text regions. Specifically, a pre-trained text detection model (such as DB-Net) is used to extract a binary image of the text region. The IoU loss is then calculated between the corrected binary image and the ideal rectangle to ensure accurate alignment of the text region.

[0086] S105: performing image preprocessing on the corrected image to be recognized to obtain a preprocessed image to be recognized.

[0087] Among them, preprocessing the corrected image to be identified can improve image quality, enhance the clarity of key features, reduce noise interference, and thus improve feature extraction effects.

[0088] Optionally, in another embodiment of the present application, the specific implementation of step S105 includes process A1 to process A3.

[0089] A1: Perform grayscale processing on the corrected image to be recognized to obtain a grayscale image.

[0090] Among them, converting the corrected image to be identified from a color image to a grayscale image can reduce the amount of data, thereby improving the subsequent processing speed.

[0091] A2: Filter the grayscale image to obtain a filtered grayscale image.

[0092] Optionally, the grayscale image may be filtered using Gaussian filtering or median filtering.

[0093] A3: performing normalization processing on the filtered grayscale image to obtain a normalized grayscale image, and determining the normalized grayscale image as the pre-processed image to be recognized.

[0094] It is understandable that adjusting the pixel values ​​of an image to a certain range facilitates model processing and learning. For example, the image pixel values ​​are normalized to the interval [0, 1] or [-1, 1].

[0095] S106: Input the pre-processed image to be identified into the visual model to obtain a hidden danger identification result.

[0096] Among them, the visual model is pre-trained based on the target sample image; the target sample image is pre-generated based on the sample gas equipment image and the rare sample image.

[0097] It should be noted that the hidden danger identification result indicates whether the gas equipment has various hidden dangers. For example, if there is no area that meets the characteristics of a flue in the image of the gas equipment to be identified, it is determined that the gas equipment may have a hidden danger of no flue.

[0098] It's important to note that hidden danger identification results can also be provided as feedback, enabling users to take appropriate action based on the findings. If the hidden danger identification results indicate a hidden danger in the gas equipment, an alarm mechanism is triggered, such as an audible or visual alarm signal, to prompt the user to take timely action. Furthermore, the hidden danger identification results are recorded in a database for subsequent query and analysis.

[0099] In addition, users can provide feedback on hidden danger identification results. If they find that the hidden danger identification results do not match the actual situation, they can provide relevant information. This feedback data can be used to further optimize the visual model and improve its accuracy. For example, if a user reports that a gas water heater is actually installed with a flue, but the hidden danger identification result mistakenly identifies it as having no flue, the image and user feedback data can be recorded and the relevant features can be re-learned and optimized in subsequent model training.

[0100] Optionally, in another embodiment of the present application, the specific implementation of step S106 includes process B1 to process B3.

[0101] B1: Input the preprocessed image to be recognized into the convolutional layer of the visual model to obtain a feature map.

[0102] Among them, the preprocessed image to be recognized is input into the convolution layer in the visual model. The convolution layer slides on the image through different convolution kernels to extract local features of the image such as edges, textures, and shapes, thereby obtaining a feature map.

[0103] B2: Input the feature map into the pooling layer in the visual model to obtain the feature map after dimensionality reduction.

[0104] Among them, the feature map is input into the pooling layer in the visual model, and the pooling layer downsamples the extracted features to reduce the number of features and reduce the computational complexity while retaining important feature information, thereby obtaining a feature map after dimensionality reduction.

[0105] B3: Input the reduced feature map into the fully connected layer of the visual model to obtain the hidden danger identification result.

[0106] It can be understood that the feature map after dimensionality reduction is input into the fully connected layer in the visual model. The fully connected layer classifies the extracted features to determine whether there are various hidden dangers in the gas equipment, thereby obtaining the hidden danger identification results.

[0107] Optionally, in another embodiment of the present application, a specific implementation method of obtaining a visual model based on pre-training of target sample images is as follows: Figure 2 As shown, the following steps are included:

[0108] S201: Acquire a target sample image.

[0109] S202: Input the target sample image into the visual model to obtain the sample hidden danger recognition result.

[0110] The sample hidden danger identification result indicates the hidden danger identification result corresponding to the target sample image.

[0111] Optionally, vision models are built on convolutional neural network architectures (e.g., VGG, ResNet, Inception), recurrent neural networks, and their variants.

[0112] Furthermore, during model construction, parameters such as the number of network layers, convolution kernel size, and pooling method should be appropriately set to balance model complexity and performance. For example, in the task of identifying gas hazards, the network depth can be appropriately increased to enhance the model's ability to extract complex features, but care must also be taken to avoid overfitting.

[0113] S203: Calculating a loss function between the sample hidden danger identification result and the actual hidden danger identification result corresponding to the target sample image.

[0114] Among them, the loss function at least includes the cross entropy loss function.

[0115] It should be noted that the actual hidden danger identification results are manually annotated for target sample images, clearly marking each component of the gas equipment in the image and whether it has hidden dangers. For example, in a gas water heater image, the location, shape, and whether the flue is missing are annotated; in a gas meter image, the status of the protective buckle is annotated; and in a gas stove image, the type of flameout protection device and whether it is working properly are annotated. Professional image annotation tools are used in the annotation process to ensure accuracy and consistency.

[0116] S204: When the loss function has not converged, adjust the model parameters of the visual model and return to step S202.

[0117] It is understandable that the model parameters of the visual model can be adjusted through the back-propagation algorithm so that the loss function is gradually reduced, thereby improving the accuracy of the model.

[0118] It should be noted that by simultaneously training the equipment classification (3 categories), component positioning (5 key parts) and hidden danger detection (12 states) tasks, and using shared feature layers (such as the first 10 layers of ResNet-50) to achieve joint parameter optimization, the number of parameters in the visual model was reduced by 25%, while the overall performance was improved by 18%.

[0119] Furthermore, during training, data augmentation techniques such as rotation, scaling, cropping, and brightness adjustment can be employed to increase the diversity of training data and prevent model overfitting. Furthermore, by properly setting hyperparameters such as the training batch size and learning rate, the training process can be optimized and efficiency improved. For example, a larger batch size can leverage more data for parallel computation, accelerating training speed, but memory limitations must be considered. An appropriate learning rate can help the model converge to the optimal solution more quickly.

[0120] S205: When the loss function converges, it is determined that the visual model training is completed.

[0121] It's important to emphasize that during training, the visual model should be regularly evaluated using a validation dataset, using metrics such as accuracy, recall, and F1 score. Based on the evaluation results, the model can be optimized, such as by adjusting the model structure and hyperparameters or adding more training data. When the model's performance on the validation dataset reaches a predetermined threshold or the loss function converges, the visual model training is considered complete.

[0122] Furthermore, the trained visual model is tested on a test dataset to ensure its accuracy and generalization in real-world applications. For example, if the visual model achieves over 95% accuracy in identifying the presence of a gas water heater flue and over 90% recall in identifying the status of a gas meter's protective buckle, the model is considered to meet the requirements for real-world applications.

[0123] Optionally, in another embodiment of the present application, a specific implementation method of generating a target sample image based on a sample gas device image and a rare sample image is described in detail below. Figure 3 As shown, the following steps are included:

[0124] S301: Acquire a sample gas equipment image.

[0125] The sample gas equipment images include images of various hidden danger types, including high-risk hidden dangers, medium-risk hidden dangers, and low-risk hidden dangers.

[0126] Optionally, sample gas equipment images can include images of different brands, models, and types of gas equipment with potential hazards. For example, images of gas water heaters with different flue styles (direct exhaust, forced exhaust, etc.) and images of faulty water heaters without flues can be collected; images of gas meters with and without protective buckles can be collected; and images of gas stoves with and without flameout protection devices can be collected.

[0127] For example, the absence of a flue and missing protective buckles are high-risk hazards, a damaged flue is a medium-risk hazard, and surface stains are low-risk hazards.

[0128] S302: Setting a loss weight for the sample gas equipment image according to the hidden danger type.

[0129] Among them, the sample gas equipment image with high-risk hidden danger type is set with a loss weight of 3 times, the sample gas equipment image with medium-risk hidden danger type is set with a loss weight of 2 times, and the sample gas equipment image with low-risk hidden danger type is set with a loss weight of 1 times.

[0130] S303: After the loss weight is set, a conditional generative adversarial network is used to generate rare sample images based on the sample gas equipment images.

[0131] It's understandable that using a conditional generative adversarial network to generate rare hidden danger samples (for example, a chimney tilted at 45° or a partially obscured protective buckle) is relatively rare in the training set, so generating them can help the model better address overfitting to low-probability hidden dangers. Every 50 iterations, the enhancement pool is updated, with newly added rare hidden danger samples accounting for 15% of the training set. This method can effectively reduce the recognition error of low-probability hidden dangers (occurrence rate < 0.1%).

[0132] S304: Generate a target sample image based on the sample gas equipment image and the rare sample image.

[0133] Optionally, many hidden dangers of gas equipment do not exist in isolation. One hidden danger may trigger a series of chain reactions, leading to more serious consequences. By predicting the probability of chain reactions of hidden dangers, the subsequent risks that may be caused by hidden dangers can be known in advance. Therefore, in another embodiment of the present application, a method for predicting the probability of chain reactions of hidden dangers is provided, such as Figure 4 As shown, the following steps are included:

[0134] S401: Obtain inspection text and gas usage.

[0135] Among them, if the inspection text has problems such as distorted shooting angle, illegible handwriting, and seal coverage, perspective transformation and spatial transformation networks are used to correct tilted and curved table images to improve the readability of the text area.

[0136] It's worth noting that for mixed printed and handwritten OCR, the Transformer and CRNN models are integrated to achieve simultaneous recognition of printed fields (such as user IDs and addresses) and handwritten notes (such as hazard descriptions and signatures) in inspection text, with a character accuracy exceeding 98%. Furthermore, through semantic information extraction, an entity recognition model (such as BERT-BiLSTM-CRF) for the gas field is constructed to extract structured data such as "hazard type," "treatment recommendations," and "re-inspection deadline" from the text, automatically generating electronic work orders.

[0137] Specifically, the model architecture of this hybrid print / handwritten OCR technology consists of the following components: A ViT-CRNN hybrid model: First, ViT is used as a feature extractor. The input image is segmented into 16×16 pixel blocks and then converted into a 768-dimensional embedding vector through linear projection. Next, it is processed by a 12-layer Transformer encoder, each layer consisting of an 8-head multi-head self-attention mechanism and a 2048-dimensional MLP hidden layer. The final output is a sequence of global context feature vectors. A CRNN decoder: The decoding part uses a bidirectional LSTM (256 units) to encode the feature sequence output by ViT to capture bidirectional temporal dependencies. A connectionist temporal classification (CTC) loss function is used to align sequences of variable length and output a character probability matrix. A font classification branch: To distinguish between print and handwritten characters, a classification head is designed in parallel with the CRNN to predict the probability of each character belonging to print or handwritten. This classification head uses a cross-entropy loss function with a weight coefficient of λ = 0.3.

[0138] S402: Utilize the residual network to extract the image to be recognized and obtain image features.

[0139] Among them, the residual network includes but is not limited to ResNet-50.

[0140] S403: Input the inspection text into the BERT model to obtain the semantic vector of the inspection text.

[0141] S404: Extracting gas usage using a recurrent neural network to obtain time series features.

[0142] Among them, the recurrent neural network includes but is not limited to LSTM.

[0143] S405: Map the image features, the semantic vector of the inspection text, and the temporal features into the same space to obtain multimodal data.

[0144] Among them, the image features, semantic vectors of inspection texts and time series features are mapped into the same space through a multi-layer perceptron to obtain multimodal data. Specifically, the specific form of multimodal data is: Among them, Fi mg is the image feature, F text is the semantic vector of the inspection text, F ts is the time series feature, is a learnable parameter matrix with an output dimension of 512. text is the embedding layer output of text features, E ts is the embedding layer output of the temporal feature.

[0145] It's important to note that in contrastive learning optimization, positive sample pairs consist of multimodal data from the same hidden danger event, such as an image (e.g., "hose crack") and text (e.g., "replacement recommended"). Negative sample pairs are constructed by randomly sampling data from different events. The loss function uses the InfoNCE loss to maximize the similarity between positive sample pairs.

[0146] Specifically, the specific expression of the nfoNCE loss function is shown in formula (1).

[0147] (1)

[0148] In formula (1), is the cosine similarity, is the temperature coefficient, which is used to control the smoothness of the distribution. sim(zi,zj) represents the sample z i and z j The cosine similarity between them, N is the batch size, including 1 positive sample pair and N-1 negative sample pairs, is the similarity score of the positive sample pair, The logarithmic operation converts the probability into a loss value, and the negative sign is taken to make the optimization objective minimize the loss.

[0149] It can be understood that by adjusting the temperature coefficient τ, the sensitivity of the model to similarity can be controlled, thereby improving the accuracy of hidden danger identification.

[0150] S406: Use the graph attention network to infer the multimodal data based on the preset knowledge graph to obtain the probability of hidden danger chain reaction.

[0151] When building a pre-defined knowledge graph for the gas sector, entities are first defined, including hazards (such as cracks and leaks), equipment (such as stoves and meters), users (such as addresses and gas usage habits), and specifications (such as GB standards). Next, relationships between entities are defined, such as the causal chain (e.g., hose aging → leak → explosion, with a 0.2% probability) and disposal rules (e.g., rusted meters must be replaced within three days, according to GB50028-202X). This information is stored in a Neo4j graph database and can be queried in real time using the Cypher language, supporting traversal of associated paths to improve the efficiency of hazard monitoring and handling.

[0152] Optionally, the node embedding update formula of the graph attention network is: .in, is the attention weight, is the LeakyReLU activation function, is the updated embedding vector of node i, To aggregate the information of all neighbor nodes j of node i, is the learnable weight matrix, is the embedding vector of neighbor node j.

[0153] It's important to note that during reasoning, the Graph Attention Network can simultaneously process the features of multimodal data and the structured information in the pre-set knowledge graph. Through the three-layer Graph Attention Network, information about neighboring nodes can be aggregated. For example, multimodal data can be input into the Graph Attention Network as node features, while the causal chains and processing rules in the pre-set knowledge graph guide the Graph Attention Network's reasoning process, resulting in more accurate predictions of the probability of hidden danger chain reactions.

[0154] In summary, by using visual models to identify images of gas equipment to be identified, a large number of gas equipment can be quickly inspected, significantly shortening the inspection time and promptly discovering new hidden dangers.

[0155] like Figure 5 , which is a schematic diagram of the architecture of a gas hazard identification device provided in the present application, the identification device includes: an acquisition unit 100, a segmentation unit 200, a noise reduction unit 300, a correction unit 400, a processing unit 500 and an identification unit 600.

[0156] The acquisition unit 100 is used to acquire the image to be identified of the gas equipment.

[0157] The segmentation unit 200 is used to perform semantic segmentation on the image to be identified to obtain a segmented image to be identified.

[0158] The segmentation unit 200 is specifically configured to: extract a device region in the image to be identified; and generate a segmented image to be identified based on the device region.

[0159] The denoising unit 300 is configured to perform denoising on the segmented image to be identified to obtain a denoised image to be identified.

[0160] The correction unit 400 is configured to perform angle correction on the image to be identified after noise reduction to obtain a corrected image to be identified.

[0161] The processing unit 500 is configured to perform image preprocessing on the corrected image to be identified to obtain a preprocessed image to be identified.

[0162] The processing unit 500 is specifically used to: perform image grayscale processing on the corrected image to be identified to obtain a grayscale image; perform filtering processing on the grayscale image to obtain a filtered grayscale image; perform normalization processing on the filtered grayscale image to obtain a normalized grayscale image, and determine the normalized grayscale image as the pre-processed image to be identified.

[0163] The recognition unit 600 is used to input the preprocessed image to be recognized into the visual model to obtain the hidden danger recognition result; the visual model is pre-trained based on the target sample image; the target sample image is pre-generated based on the sample gas equipment image and the rare sample image.

[0164] The recognition unit 600 is specifically used to: input the preprocessed image to be recognized into the convolution layer in the visual model to obtain a feature map; input the feature map into the pooling layer in the visual model to obtain a feature map after dimensionality reduction; input the feature map after dimensionality reduction into the fully connected layer in the visual model to obtain a hidden danger recognition result.

[0165] The recognition unit 600 is specifically used to: obtain a target sample image; input the target sample image into the visual model to obtain a sample hidden danger recognition result; calculate the loss function between the sample hidden danger recognition result and the actual hidden danger recognition result corresponding to the target sample image; the loss function at least includes a cross-entropy loss function; when the loss function has not converged, adjust the model parameters of the visual model, and return to execute the step of inputting the target sample image into the visual model to obtain the sample hidden danger recognition result; when the loss function converges, it is determined that the visual model training is completed.

[0166] The recognition unit 600 is specifically used to: obtain a sample gas equipment image; the sample gas equipment image contains images of multiple hidden danger types; the hidden danger types include high-risk hidden dangers, medium-risk hidden dangers and low-risk hidden dangers; set loss weights for the sample gas equipment images according to the hidden danger types; after the loss weights are set, use the conditional generative adversarial network to generate rare sample images based on the sample gas equipment images; generate target sample images based on the sample gas equipment images and the rare sample images.

[0167] In summary, by using visual models to identify images of gas equipment to be identified, a large number of gas equipment can be quickly inspected, significantly shortening the inspection time and promptly discovering new hidden dangers.

[0168] Combine Figure 5 The content shown, the identification device also includes:

[0169] The acquisition unit is used to obtain inspection text and gas usage.

[0170] The first extraction unit is used to extract the image to be identified by using a residual network to obtain image features.

[0171] The input unit is used to input the inspection text into the BERT model to obtain the semantic vector of the inspection text.

[0172] The second extraction unit is used to extract the gas usage using a recurrent neural network to obtain time series features.

[0173] The mapping unit is used to map image features, semantic vectors of inspection texts, and temporal features into the same space to obtain multimodal data.

[0174] The reasoning unit is used to use the graph attention network to reason about multimodal data based on the preset knowledge graph to obtain the probability of hidden danger chain reaction.

[0175] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Ordinary technicians in this field can understand and implement it without expending creative work.

[0176] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0177] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying hidden gas hazards, characterized in that: include: Collect images of gas equipment to be identified; Performing semantic segmentation on the image to be identified to obtain a segmented image to be identified; performing noise reduction processing on the segmented image to be identified to obtain a noise-reduced image to be identified; Performing angle correction on the image to be identified after noise reduction to obtain a corrected image to be identified; Performing image preprocessing on the corrected image to be identified to obtain a preprocessed image to be identified; Inputting the pre-processed image to be identified into a visual model to obtain a hidden danger identification result; the visual model is pre-trained based on the target sample image; The target sample image is generated in advance based on the sample gas appliance image and the rare sample image.

2. The method according to claim 1, characterized in that The performing semantic segmentation on the image to be identified to obtain the segmented image to be identified includes: Extracting a device area in the image to be identified; A segmented image to be recognized is generated based on the device area.

3. The method according to claim 1, characterized in that The performing image preprocessing on the corrected image to be identified to obtain a preprocessed image to be identified includes: Performing image grayscale processing on the corrected image to be identified to obtain a grayscale image; Performing filtering on the grayscale image to obtain a filtered grayscale image; Normalization processing is performed on the filtered grayscale image to obtain a normalized grayscale image, and the normalized grayscale image is determined as the pre-processed image to be recognized.

4. The method according to claim 1, wherein The step of inputting the pre-processed image to be identified into a visual model to obtain a hidden danger identification result includes: Inputting the preprocessed image to be recognized into the convolutional layer of the visual model to obtain a feature map; Inputting the feature map into the pooling layer in the visual model to obtain a feature map after dimensionality reduction; The feature map after dimensionality reduction is input into the fully connected layer in the visual model to obtain a hidden danger identification result.

5. The method according to claim 1, characterized in that Also includes: Obtain inspection text and gas usage; Extracting the image to be identified using a residual network to obtain image features; Input the inspection text into the BERT model to obtain a semantic vector of the inspection text; Extracting the gas usage using a recurrent neural network to obtain time series features; Mapping the image features, the semantic vector of the inspection text, and the temporal features into the same space to obtain multimodal data; The graph attention network is used to infer the multimodal data according to the preset knowledge graph to obtain the probability of hidden danger chain reaction.

6. The method according to claim 1, characterized in that The process of obtaining the visual model based on the target sample image through pre-training includes: Acquire target sample image; Inputting the target sample image into the visual model to obtain a sample hidden danger recognition result; Calculating a loss function between the sample hidden danger identification result and the actual hidden danger identification result corresponding to the target sample image; the loss function at least includes a cross entropy loss function; When the loss function does not converge, adjusting the model parameters of the visual model, and returning to the step of inputting the target sample image into the visual model to obtain a sample hidden danger identification result; When the loss function converges, it is determined that the visual model training is completed.

7. The method according to claim 1, characterized in that The process of generating a target sample image based on a sample gas equipment image and a rare sample image in advance includes: Acquire sample gas equipment images; the sample gas equipment images include images of multiple hidden danger types; the hidden danger types include high-risk hidden dangers, medium-risk hidden dangers, and low-risk hidden dangers; Setting a loss weight for the sample gas equipment image according to the hidden danger type; After the loss weight is set, a conditional generative adversarial network is used to generate rare sample images based on the sample gas equipment images; A target sample image is generated based on the sample gas appliance image and the rare sample image.

8. A device for identifying hidden dangers of gas, characterized in that: include: An acquisition unit, used for acquiring images of gas equipment to be identified; a segmentation unit, configured to perform semantic segmentation on the image to be identified to obtain a segmented image to be identified; a noise reduction unit, configured to perform noise reduction processing on the segmented image to be identified to obtain a noise-reduced image to be identified; a correction unit, configured to perform angle correction on the image to be identified after noise reduction to obtain a corrected image to be identified; a processing unit, configured to perform image preprocessing on the corrected image to be identified to obtain a preprocessed image to be identified; A recognition unit, configured to input the pre-processed image to be recognized into a visual model to obtain a hidden danger recognition result; the visual model is pre-trained based on a target sample image; The target sample image is generated in advance based on the sample gas appliance image and the rare sample image.

9. The device according to claim 8, characterized in that The segmentation unit is specifically used for: Extracting a device area in the image to be identified; A segmented image to be recognized is generated based on the device area.

10. The device according to claim 8, characterized in that The processing unit is specifically configured to: Performing image grayscale processing on the corrected image to be identified to obtain a grayscale image; Performing filtering on the grayscale image to obtain a filtered grayscale image; Normalization processing is performed on the filtered grayscale image to obtain a normalized grayscale image, and the normalized grayscale image is determined as the pre-processed image to be recognized.

Images