Gas equipment classification detection method and device based on spatio-temporal feature fusion and dynamic graph attention reasoning and medium

Abstract

The application provides a gas equipment classification detection method and device based on space-time feature fusion and dynamic graph attention reasoning and a medium. In view of the problems that existing single-frame detection features are easy to be missing, confidence calibration is poor, and topological rules are weak in generalization, the method acquires a security video stream to extract multiple frames of images, performs space-time feature extraction and time sequence fusion to output a focused feature map; the focused feature map is input into a hierarchical detection head for fine-grained classification to output an initial detection node set containing cognitive uncertainty; a dynamic space topology reasoning engine is constructed to map the node set into a graph structure and construct edges based on relative spatial relationships, cognitive uncertainty is used to modulate the weight of the graph attention network to aggregate context semantic features, and an optimized detection result is obtained through decoding and visualization. The application is mainly used in home security scenes and can improve the detection accuracy and robustness in complex environments.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method, apparatus and medium for classifying and detecting gas equipment based on spatiotemporal feature fusion and dynamic graph attention reasoning. Background Technology

[0002] With social development and increased safety awareness among residents, safety inspections of in-home gas appliances have become a crucial step in ensuring public safety and the security of family property. Traditional gas appliance safety inspections rely heavily on manual visual inspections, which have inherent limitations such as low efficiency, susceptibility to subjective judgment, and difficulty in guaranteeing accuracy in complex environments.

[0003] In terms of automated detection technology, existing solutions typically adopt the following technical approach (as our company has applied for patent CN120766031A): First, a single raw image of the site is captured by the camera of a mobile device. Then, a visual feature extraction network containing a backbone network (such as EfficientNet-B3) and an attention module (such as CBAM) is used to extract features from the image to obtain a focused visual feature map of the device. Subsequently, the feature map is input into a hierarchical detection head component containing a first-level detection module and a second-level detection module. The first-level detection module is responsible for identifying the major categories of the device, and the second-level detection module performs secondary identification on the categories that need to be further subdivided, outputting an initial detection result list. Finally, a spatial relationship rule set for gas equipment is constructed. Based on manually preset rules (such as "the gas stove is located below the gas pipeline"), the initial detection results are subjected to context awareness and confidence correction to obtain optimized detection results and generate a visualized image.

[0004] The aforementioned scheme alleviates the problem of the wide variety and high similarity of appearance of gas equipment to some extent through a hierarchical detection strategy. However, its spatial relationship reasoning relies entirely on manually preset hard-coded rules, and the confidence correction range is determined by fixed empirical parameters, lacking the ability to adapt to different installation layouts. In addition, this scheme only performs detection based on a single frame of static image. During the dynamic scanning process of mobile devices, when key equipment loses features in a single frame due to reflection or occlusion, it cannot be compensated for using effective information from other frames.

[0005] Therefore, there is an urgent need for a gas equipment classification and detection method, device, and medium based on spatiotemporal feature fusion and dynamic graph attention reasoning to solve the problems existing in the current technology. Summary of the Invention

[0006] This invention provides a gas equipment classification and detection method, device, and medium based on spatiotemporal feature fusion and dynamic graph attention reasoning. It addresses the three technical bottlenecks in existing gas equipment classification and detection methods, which are unable to simultaneously overcome the lack of temporal features in single-frame detection, poor confidence calibration of fine-grained classification, and insufficient generalization ability of static rule sets. These bottlenecks lead to poor robustness and limited accuracy in complex in-home security inspection scenarios.

[0007] The core technology of this invention is to upgrade single-frame image detection to multi-frame temporal input of video stream to extract spatiotemporal fusion features. In the second-level fine-grained classification of hierarchical detection, an uncertainty perception mechanism is introduced to output cognitive uncertainty. This cognitive uncertainty is then used as a modulation signal to introduce into a dynamic spatial topology inference engine based on graph attention network to adaptively modulate the attention weights. This allows high-uncertainty detection results to obtain enhanced context compensation from neighboring nodes, thereby achieving temporal compensation and adaptive spatial context correction of the detection results.

[0008] In a first aspect, the present invention provides a gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning, the method comprising the following steps: S1: Acquire the on-site video stream collected during the security check scanning process and extract continuous multi-frame time series images from it; S2: Extract spatiotemporal visual features from multiple time-series images, and output a spatiotemporal device focusing visual feature map by fusing spatial feature extraction with inter-frame temporal features. S3: Input the spatiotemporal device focused visual feature map into the hierarchical detection head component for target detection and fine-grained classification, and introduce an uncertainty perception mechanism in the fine-grained classification process to output an initial detection result node set containing device type label, bounding box information, initial confidence and cognitive uncertainty; S4: Construct a dynamic spatial topology inference engine based on graph attention network. Map the initial detection result node set to a graph structure. Dynamically construct the edges of the graph structure based on the relative spatial relationships between nodes. Aggregate the contextual semantic features of adjacent nodes through graph attention mechanism to update node features. In particular, the attention weights of the graph attention network are modulated based on cognitive uncertainty so that nodes with higher cognitive uncertainty receive increased contextual compensation. Then, the updated node features are decoded to obtain refined confidence and classification labels, resulting in an optimized detection result list. S5: Based on the optimized detection result list and keyframe images selected from the live video stream, generate a visualization image with bounding boxes and device type labels.

[0009] Furthermore, spatiotemporal visual features are extracted from multiple time-series images, including: Spatial features of single-frame images are extracted using a backbone network and an attention module; The spatial features of consecutive frames are input into a spatiotemporal long short-term memory (LSTM) convolutional network. The spatiotemporal LSM convolutional network performs temporal fusion of the spatial features of consecutive frames through gating mechanisms of input gate, forget gate, and output gate. The forget gate adaptively adjusts the retention ratio of historical frame information according to the feature quality of the current frame, and the input gate controls the writing ratio of new information in the current frame. The hidden state output of the last frame of the time series is taken as the spatiotemporal device focusing visual feature map, so as to use the memory state of historical frames to temporally compensate for the feature loss in the current frame caused by occlusion or reflection.

[0010] Furthermore, the hierarchical detection head component includes a first-level detection module and a second-level detection module. The first-level detection module is used to detect the major categories of equipment. When the detected major category of equipment does not require further subdivision, the major category label is used as the final equipment type label, and the corresponding cognitive uncertainty is set to a preset low value. When the detected major category of equipment needs further subdivision, the corresponding equipment region features are extracted from the spatiotemporal equipment focusing visual feature map and input into the second-level detection module for fine-grained classification. The second-level detection module outputs the corresponding cognitive uncertainty while outputting the fine-grained classification result.

[0011] Furthermore, the uncertainty perception mechanism is implemented through a dynamic loss constraint function, which includes an adaptive loss weighting term and a regularization term. The adaptive loss weighting term is configured to reduce the weight of the classification loss of the samples according to the predicted cognitive uncertainty, and the regularization term is configured to impose a lower bound constraint on the cognitive uncertainty to prevent overestimation.

[0012] Furthermore, the relative spatial relationships participate in the calculation of the graph attention mechanism in the form of spatial distance encoding. The spatial distance encoding is a multi-dimensional vector, and the feature components contained in the multi-dimensional vector include at least: the horizontal relative offset component and the vertical relative offset component of the bounding box information in the normalized image coordinate system, the Euclidean distance component after logarithmic processing, and the intersection-union ratio component between the bounding boxes.

[0013] Furthermore, the edges of the graph structure are dynamically constructed based on the relative spatial relationships between nodes, including: calculating the normalized Euclidean distance between the center points of the bounding boxes of each node; constructing edges between corresponding nodes when the normalized Euclidean distance is lower than a preset distance threshold; and forcibly retaining a preset number of nearest neighbor edges for each node to ensure the connectivity of the graph structure and form a sparse connected graph.

[0014] Furthermore, the attention weights of the graph attention network are modulated based on cognitive uncertainty, including: When calculating the normalized attention weights between nodes, a temperature scaling modulation based on cognitive uncertainty is introduced. The uncertainty temperature coefficient of a node is defined to be positively correlated with cognitive uncertainty. The attention coefficient is scaled using the temperature coefficient, so that the attention weight distribution of nodes with high cognitive uncertainty tends to be flattened, thereby absorbing contextual information from a wider neighborhood. The updated node features are decoded to obtain refined confidence scores and classification labels, including: The updated node features are mapped to a refined category probability distribution through a classification decoding network, and the updated node features are concatenated with the original initial confidence and cognitive uncertainty information and then mapped to a refined confidence through a confidence decoding network.

[0015] Secondly, the present invention provides a gas equipment classification and detection device based on spatiotemporal feature fusion and dynamic graph attention reasoning, comprising: The video stream acquisition module is configured to acquire the on-site video stream collected during the security check scanning process and extract continuous multi-frame time series images from it; The spatiotemporal visual feature extraction module is configured to extract spatiotemporal visual features from multiple time-series images. By fusing spatial feature extraction with inter-frame temporal features, it outputs a spatiotemporal device focusing visual feature map. The uncertainty-aware hierarchical detection module is configured to input the spatiotemporal device focused visual feature map into the hierarchical detection head component for target detection and fine-grained classification, and introduce an uncertainty-aware mechanism in the fine-grained classification process to output an initial detection result node set containing device type label, bounding box information, initial confidence and cognitive uncertainty; The dynamic spatial topology reasoning module is configured to build a dynamic spatial topology reasoning engine based on a graph attention network. It maps the initial detection result node set into a graph structure, dynamically constructs the edges of the graph structure based on the relative spatial relationships between nodes, and aggregates the contextual semantic features of adjacent nodes through a graph attention mechanism to update the node features. Specifically, it applies temperature scaling modulation to the attention weights of the graph attention network based on cognitive uncertainty, so that nodes with higher cognitive uncertainty receive increased contextual compensation. Subsequently, it decodes the updated node features to obtain refined confidence and classification labels, resulting in an optimized detection result list. The keyframe visualization module is configured to generate visualization images with bounding boxes and device type labels based on an optimized list of detection results and keyframe images selected from the live video stream.

[0016] Thirdly, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to execute the above-described gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning.

[0017] Fourthly, the present invention provides a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning as described above.

[0018] The main contributions and innovations of this invention are as follows: 1. This invention introduces a spatiotemporal long short-term memory convolutional network to perform gated temporal fusion of spatial features in consecutive frames. It utilizes the hidden states and cell states of historical frames to temporally compensate for damaged features in the current frame caused by reflections, occlusions, or motion blur. Even if the gas equipment is briefly obscured in a frame, the output of the ConvLSTM can still contain valid equipment information from historical frames. This overcomes the high false negative rate of single-frame detection in complex environments in existing solutions, achieving a qualitative leap from "instantaneous single-frame detection" to "temporally stable detection."

[0019] 2. This invention introduces an uncertainty-aware loss function into the second-level fine-grained classifier, enabling the classifier to actively assess the reliability of its own predictions while outputting the prediction results, and outputting cognitive uncertainty. When the model encounters device models with highly similar appearances, it automatically outputs a large cognitive uncertainty to indicate that the detection result needs further correction. This overcomes the inherent defects of existing deep neural networks, such as overconfidence in classification probabilities and difficulty in correcting misclassifications by downstream rules, achieving a leap from "blind confidence" to "self-awareness" in confidence rating.

[0020] 3. This invention replaces the manual static rule set in existing technologies with a dynamic spatial topology inference engine based on graph attention networks. It abstracts the detection results into nodes in a graph structure and adaptively learns the spatial topology relationships between devices through an attention mechanism. Even when faced with extremely rare non-standard pipeline layouts such as unauthorized connections and non-standard modifications, it can infer the hidden connection relationships between devices through graph convolutional logic. This overcomes the technical bias that manual rules cannot cover all installation forms and realizes a paradigm shift from "rigid rule-driven" to "data-driven adaptive inference".

[0021] 4. This invention uses the cognitive uncertainty output by the uncertainty perception module as the modulation signal for the attention weights of the graph attention network. By introducing a modulation term positively correlated with cognitive uncertainty into the attention normalization exponent through learnable modulation coefficients, high-uncertainty nodes receive stronger contextual compensation during the information aggregation stage, achieving an adaptive feedback loop of "uncertainty equals correction." This mechanism allows limited computational resources to be precisely focused on the detection results that most need correction, avoiding the "one-size-fits-all" flaw of applying the same correction factor to all low-confidence results in existing schemes. This significantly improves both correction efficiency and accuracy.

[0022] Details of one or more embodiments of the present invention are set forth in the following drawings and description, so that other features, objects and advantages of the invention will be more readily understood. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of a gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning according to an embodiment of the present invention; Figure 2 This is a system architecture and data flow diagram according to an embodiment of the present invention; Figure 3 This is a diagram of the spatiotemporal visual feature extraction network structure according to an embodiment of the present invention; Figure 4 This is a structural diagram of an uncertainty-aware hierarchical detection head assembly according to an embodiment of the present invention; Figure 5 This is a structural diagram of a dynamic spatial topology inference engine according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the uncertainty modulation mechanism according to an embodiment of the present invention; Figure 7 This is a schematic diagram of a sparse graph construction strategy according to an embodiment of the present invention; Figure 8 This is a flowchart of a two-stage training strategy according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0025] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0026] Example 1 This embodiment provides a complete implementation of a gas appliance classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning. This method is applied to in-home security inspection scenarios and is executed by a mobile device pre-installed with a dedicated security inspection application (such as a smartphone or tablet equipped with a high-resolution camera), or by a mobile device capturing video streams and transmitting them to a remote processing server for execution. The overall process of the method is as follows: Figure 1 As shown, the system architecture and data flow are as follows: Figure 2 As shown in the accompanying drawings, the specific implementation of this method will be described in detail below: I. Video Stream Acquisition (Step S1) like Figure 1 As shown, in this embodiment, step S1 is first executed: acquiring the on-site video stream continuously collected by the mobile device camera during the security check scanning process, and extracting continuous multi-frame time series images from it.

[0027] In the actual operation of in-home security checks, security personnel enter users' homes carrying mobile devices pre-installed with a dedicated security check application. The mobile device is a smartphone or tablet equipped with a high-resolution camera. After the security personnel launch the dedicated security check application, the application activates the camera module and enters video capture mode. Unlike traditional photo-taking modes, this embodiment uses a continuous video capture mode, recording the on-site video stream uninterrupted throughout the entire security check process.

[0028] Security personnel point their mobile device's camera at the area of ​​gas appliances to be inspected, scanning the appliances in areas such as kitchens and balconies at an appropriate movement speed. During the scan, the mobile device's autofocus, auto exposure, and auto white balance functions automatically adjust according to scene information to ensure the visual quality of the video frames. The video stream is continuously acquired at a preset frame rate and resolution. In a specific example, the frame rate is set to 15 frames per second or 30 frames per second, and the resolution is set to 1920×1080 pixels. The video stream is stored in the mobile device's local storage in an uncompressed format or transmitted in real time to a cloud processing terminal via a wireless network.

[0029] When extracting consecutive multi-frame time-series images from the acquired video stream, a fixed-interval sampling strategy is adopted: one frame is extracted from the video stream every preset number of frames to form a time-series image set containing T frames. The choice of the frame number T needs to balance temporal coverage and computational efficiency—if T is too small, the temporal information is insufficient; if T is too large, the computational overhead increases and may introduce too many redundant frames. In one specific instance, one frame is extracted every 5 frames, T=8. In another specific instance, one frame is extracted every 10 frames, T=16. The extracted time-series images serve as input for subsequent steps. There is a temporal correlation between consecutive frames in the sequence, and the position and appearance of the same device in adjacent frames show a continuous changing pattern.

[0030] II. Spatiotemporal visual feature extraction (step S2) like Figure 1 and Figure 3 As shown, this embodiment then proceeds to step S2: extracting spatiotemporal visual features from multiple time-series images, and outputting a spatiotemporal device focusing visual feature map by fusing spatial feature extraction with inter-frame temporal features.

[0031] 2.1 Image Preprocessing Before inputting the spatiotemporal visual feature extraction network, a uniform preprocessing operation is performed on each frame of time-series image extracted in step S1.

[0032] First, size normalization is performed: Since different mobile devices or shooting conditions may result in varying original image resolutions, each frame is scaled to a preset standard size using bilinear interpolation. In a specific example, the preset standard size is 300×300 pixels.

[0033] Then, pixel value standardization is performed: the RGB pixel values ​​of each frame are normalized from the range [0,255] to the range [0,1], and further standardized using the mean and standard deviation of the ImageNet dataset. For the RGB three channels, the mean is set to [0.485, 0.456, 0.406], and the standard deviation is set to [0.229, 0.224, 0.225]. This standardization operation makes the distribution of the input data consistent with the expected distribution of the pre-trained model, which helps the model converge faster and improves its generalization ability.

[0034] After preprocessing, a standardized set of time series images is obtained. They are then sequentially input into the spatiotemporal visual feature extraction network.

[0035] 2.2 EfficientNet-B3 Backbone Network like Figure 3As shown, the spatiotemporal visual feature extraction network includes a backbone network, an attention module, and a temporal fusion module. The backbone network uses the EfficientNet-B3 network to extract spatial features from each frame of the image. EfficientNet-B3 is an efficient convolutional neural network architecture composed of a series of moving inverted bottleneck (MBConv) modules stacked together. Each MBConv module contains depthwise separable convolutions, a compression and activation (SE) attention mechanism, and residual connections. EfficientNet-B3 significantly reduces the number of model parameters and computational cost while maintaining high accuracy through a composite scaling method—that is, systematically scaling the depth, width, and resolution of the network uniformly. Each frame of the preprocessed image... After being input into EfficientNet-B3, the data is processed through multiple stages of the MBConv module, outputting the spatial feature map of the frame at a specific layer of the network. This spatial feature map contains multi-scale structural information and high-level semantic information of the gas equipment in the image.

[0036] 2.3 Convolutional Block Attention Module (CBAM) like Figure 3 As shown, Convolutional Block Attention (CBAM) modules are embedded after certain layers of the EfficientNet-B3 backbone network. CBAM is a lightweight, general-purpose attention module used to enhance the feature representation capabilities of convolutional neural networks.

[0037] The convolutional block attention module consists of two sequential sub-modules: the channel attention module and the spatial attention module.

[0038] The channel attention module performs global average pooling and global max pooling on the input feature map, feeds the two pooling results into a shared multilayer perceptron, sums the outputs of the multilayer perceptron, and generates channel attention weights through a sigmoid activation function. These weights are then multiplied channel-by-channel with the original feature map to emphasize important feature channels related to the gas equipment.

[0039] The spatial attention module receives the feature map after channel-weighted attention processing, performs global average pooling and global max pooling on it along the channel dimension, concatenates the two results, and then generates spatial attention weights through a standard convolutional layer (7×7 kernel size) and a sigmoid activation function. These weights are multiplied pixel-by-pixel with the channel-weighted feature map to emphasize the important spatial location of the gas equipment.

[0040] Through a dual mechanism of channel attention and spatial attention, the network can adaptively focus on key feature channels and spatial regions in the image that are relevant to the gas equipment, effectively suppressing interference from irrelevant background information. Each frame of the image... After processing by EfficientNet-B3 and the convolutional block attention module, the spatial feature map of the frame is output. The weight parameters of EfficientNet-B3 and the convolutional block attention module are first initialized by pre-training on the large general image dataset ImageNet, and then fine-tuned on a specially constructed gas equipment image dataset to better adapt the network to the specific feature distribution of the gas equipment recognition task.

[0041] 2.4 Temporal Fusion of Convolutional Long Short-Term Memory Network (ConvLSTM) like Figure 3 As shown, the temporal fusion module employs a Convolutional Long Short-Term Memory (ConvLSTM) network to perform temporal fusion of spatial features from consecutive frames. It utilizes the memory state of historical frames to temporally compensate for feature loss in the current frame due to occlusion or reflection. Unlike standard LSTM, which uses fully connected operations, ConvLSTM replaces the gating operations of LSTM with convolutional operations, enabling it to directly process two-dimensional spatial feature maps and perform temporal information fusion while preserving the spatial topology.

[0042] Spatial feature maps of consecutive frames The data are sequentially fed into the convolutional long short-term memory network. At each time step t, the convolutional long short-term memory network receives the spatial features of the current frame. and the hidden state of the previous moment. and cell state The gating mechanism, consisting of an input gate, a forget gate, and an output gate, determines how to fuse temporal information and outputs the spatiotemporal fusion feature map of the current moment. Its evolution formula is:

[0043] Specifically, the internal computation process of a convolutional long short-term memory network is as follows: Input gate Control the spatial features of the current frame How much new information is written to the cell state:

[0044] Forgotten Gate Controlling the cell state at the previous moment How much information is retained:

[0045] Candidate cell state Based on the current input and the previous hidden state Generated upon activation, it serves as new candidate information for writing cell states:

[0046] Cell state update It combines historical information preserved by the forget gate with new information written by the input gate:

[0047] Output gate How much information from the cell state is output as the hidden state at the current moment:

[0048]

[0049] in This represents the convolution operation. This represents element-wise multiplication. This represents the Sigmoid activation function. and For learnable parameters, subscript The gate control unit to which the identifier parameters belong.

[0050] The core technical advantage of Convolutional Long Short-Term Memory (LSTM) networks lies in temporal feature compensation. When the spatial features of a gas device are damaged due to strong reflection or occlusion in a frame, the LSTM network can utilize the memory state (cell state) of historical frames. This involves temporal compensation for the damaged features of the current frame. Specifically, the forget gate... Through learning, the quality of features in the current frame can be identified—when the current frame... When the feature response of a certain region in a cell is abnormally weak (indicating that the region is occluded or has reflections), the forget gate increases the retention ratio of the corresponding position, ensuring that the clear feature information of that region in the historical frame is preserved in the cell state. The middle part has been retained. Meanwhile, the input gate... This reduces the proportion of new information written to the corresponding location, preventing damaged features from contaminating the cell state. Through this gating mechanism, even if the device features of the current frame are severely lacking, the output of the convolutional long short-term memory network... It can still contain valid device information from historical frames.

[0051] Take the hidden state output of the convolutional long short-term memory network in the last frame (frame T) of the sequence. The final output of step S2 is the spatiotemporal device-focused visual feature map. The hidden state of the last frame is chosen because it accumulates information from the entire time series, containing the most complete temporal fusion result. This spatiotemporal device-focused visual feature map serves as the input for step S3.

[0052] III. Hierarchical Uncertainty Perception Detection (Step S3) like Figure 1 and Figure 4 As shown, this embodiment then proceeds to step S3: inputting the spatiotemporal device focusing visual feature map into the hierarchical detection head component for target detection and fine-grained classification, and introducing an uncertainty perception mechanism during the fine-grained classification process to output an initial detection result node set containing device type labels, bounding box information, initial confidence, and cognitive uncertainty.

[0053] 3.1 First-level detection module The hierarchical detection head component includes a first-level detection module and a second-level detection module. The first-level detection module performs device category detection on the spatiotemporal device focusing visual feature map, outputting the bounding boxes, labels, and confidence scores of the device categories. In this embodiment, the first-level detection module adopts a YOLO detection head structure, which contains a series of convolutional operations, ultimately mapping the feature map to a multidimensional tensor through the output convolutional layer. The tensor is divided into a grid in the spatial dimension, and each grid cell predicts several bounding boxes. Each bounding box contains the center point coordinates, width, height, and object confidence score, while simultaneously predicting a probability distribution of a set of device category labels. The YOLO detection head performs multi-scale predictions on feature maps of different scales in the backbone network to handle devices of different sizes. After detection, low-confidence prediction boxes are filtered according to a preset confidence threshold (e.g., 0.7), and non-maximum suppression (intersection over union threshold, e.g., 0.5) is applied to eliminate overlapping prediction boxes, finally obtaining a list of device category detection results.

[0054] 3.2 Hierarchical Classification Strategy For each detection result in the list of major equipment categories, a judgment is made based on a pre-configured equipment classification hierarchy mapping table. When the detected major equipment category does not require further subdivision (e.g., gas meter, gas pipeline, valve, etc.), the major equipment category label is directly used as the final equipment type label, and the corresponding cognitive uncertainty is set to a preset low value (e.g., 0.01). When the detected major equipment category requires further subdivision (e.g., stoves need to be subdivided into built-in and countertop types, water heaters need to be subdivided into forced draft and balanced flue types), the corresponding equipment region features are extracted from the spatiotemporal equipment focusing visual feature map and input into the second-level detection module for fine-grained classification.

[0055] 3.3 Second-level detection module and uncertainty perception mechanism The second-level detection module is an uncertainty-aware fine-grained classifier, whose architecture is a small convolutional neural network or multilayer perceptron, specifically designed to distinguish subcategories within a specific broad category. Let the total number of subcategories to be distinguished be... (For example, if the category of "cooktop" includes two subcategories: "built-in cooktop" and "tabletop cooktop," then C=2). This classifier outputs the cognitive uncertainty along with the classification result. Specifically, the last fully connected layer of the classifier outputs a variance logit scalar value in addition to the C category logit values. This scalar value is processed by the Softplus activation function to obtain the sample-level cognitive uncertainty. :

[0056] in Let logit be the variance of the i-th sample. The Softplus function represents the cognitive uncertainty after activation. Ensure that cognitive uncertainty is always positive.

[0057] During training, the uncertainty awareness mechanism is implemented through a dynamic loss constraint function, which includes an adaptive loss weighting term and a regularization term:

[0058] in The standard classification loss (e.g., cross-entropy loss) is used for the i-th sample. The cognitive uncertainty of the network prediction. Adaptive loss weighting term. The configuration is set to reduce the weight of the classification loss of a sample according to the predicted cognitive uncertainty—when the cognitive uncertainty is large, the weight of the classification loss of that sample in the total loss is automatically reduced, avoiding overfitting of the model to noisy samples that are difficult to classify; regularization term The configuration imposes a lower bound constraint on cognitive uncertainty to prevent overestimation, forcing the network to output smaller uncertainties when it can reliably classify the data. Through the training constraints of the aforementioned dynamic loss function, the classifier proactively outputs larger cognitive uncertainties when faced with device models that are visually similar, indicating that the detection result requires contextual correction by the subsequent GAT inference engine.

[0059] After the first-level detection and the second-level uncertainty perception fine-grained classification, an initial detection result node set is constructed. Each node in this node set corresponds to a detected gas device and includes the device type label, bounding box coordinates, initial confidence level, and perceived uncertainty.

[0060] IV. Dynamic Space Topological Reasoning (Step S4) like Figure 1 , Figure 5 , Figure 6 and Figure 7As shown, this embodiment then proceeds to step S4: constructing a dynamic spatial topology inference engine based on a graph attention network, mapping the initial detection result node set to a graph structure, dynamically constructing edges of the graph structure based on the relative spatial relationships between nodes, and aggregating the contextual semantic features of adjacent nodes through a graph attention mechanism to update node features; wherein, the attention weights of the graph attention network are modulated based on cognitive uncertainty so that nodes with higher cognitive uncertainty receive increased contextual compensation, and then the updated node features are decoded to obtain refined confidence and classification labels, resulting in an optimized detection result list.

[0061] 4.1 Graph Structure Construction Map the initial detection result node set as a graph structure Each detected gas device corresponds to a node in the graph. Node feature vectors The edges in the graph are obtained by global average pooling of the ROI features corresponding to the device. The edges are dynamically constructed based on the relative spatial relationships between nodes: the normalized Euclidean distance between the center points of the bounding boxes of all node pairs is calculated.

[0062] in and Here, W and H are the coordinates of the center points of the bounding boxes of nodes i and j, respectively, and the width and height of the image are W and H, respectively. When the normalized Euclidean distance is lower than a preset distance threshold... (For example When an edge is constructed between corresponding nodes, it is used to construct an edge. Simultaneously, to ensure the connectivity of the graph structure, a preset number of nearest neighbor edges (e.g., k=3) are forcibly retained for each node, forming a sparsely connected graph. This sparse graph construction strategy ensures that graph connections exist only between physically adjacent devices, avoiding erroneous attention transfer between unrelated, distant devices.

[0063] 4.2 Spatial Distance Coding For each edge in the graph Calculate its spatial distance encoding Spatial distance is encoded as a multidimensional vector. The multidimensional vector contains at least the following feature components: the horizontal relative offset component and the vertical relative offset component of the bounding box information in the normalized image coordinate system; the logarithmically converted Euclidean distance component; and the intersection-union ratio (IUU) component between bounding boxes.

[0064] in , , The Euclidean distance between the center points of the two bounding boxes. For example, take a smoothing constant (e.g., take...) ), This represents the intersection-union ratio (IUGR) of the two bounding boxes. This encoding describes the physical spatial topology between devices from four dimensions: horizontal relative position, vertical relative position, overall distance scale, and spatial overlap.

[0065] 4.3 Attention Calculation and Uncertainty Modulation in Multi-Head Graphs In this embodiment, K parallel attention heads (e.g., K=4 or K=8) are used for graph attention computation. For the k-th attention head, the unnormalized attention coefficients of node pair (i,j) are first calculated:

[0066] in and For node feature vectors, For learnable weight matrix, For learnable attention parameter vectors, This represents a splicing operation. This is a spatial distance encoding. The attention coefficient is simultaneously constrained by both the semantic feature similarity of nodes and the physical spatial topological relationship.

[0067] In calculating the normalized attention weights between nodes, this invention adaptively modulates the attention weights based on cognitive uncertainty. The core idea is to use the cognitive uncertainty output in step S3 as a control signal for the graph attention network. By changing the distribution characteristics during the attention weight normalization process, nodes with higher cognitive uncertainty can absorb contextual information from a wider neighborhood during the information aggregation stage, thus achieving stronger contextual compensation. Specifically, this modulation mechanism is implemented as follows: a modulation parameter positively correlated with the node's cognitive uncertainty is defined, and this parameter is used to adjust the normalization calculation process of the attention weights. When a node's cognitive uncertainty is high, its attention weight distribution tends to flatten (i.e., a relatively uniform distribution of attention weights to more neighboring nodes), thereby expanding the absorption range of contextual information. When a node's cognitive uncertainty is low, the attention weight distribution maintains the same concentrated characteristics as a standard graph attention network, and the node absorbs neighbor information only at a normal proportion.

[0068] The general principle of this modulation mechanism is as follows: by introducing a modulating factor positively correlated with cognitive uncertainty, the entropy of the normalized probability distribution of attention increases with the increase of uncertainty, thereby achieving an adaptive feedback loop of "uncertainty is correction" at the information aggregation level. In specific implementation, this modulation parameter can be introduced in ways that are not limited to, such as temperature scaling, exponential term bias, distribution sharpening / smoothing, etc., which can change the entropy of the normalized distribution.

[0069] In this embodiment, as Figure 6As shown, a temperature scaling modulation mechanism is preferably used to achieve the above adaptive modulation. The uncertainty temperature coefficient of node i is defined as:

[0070] in Let i be the cognitive uncertainty. These are learnable uncertainty modulation coefficients. When cognitive uncertainty is high, When cognitive uncertainty is low (close to zero), By scaling the exponential term of the attention coefficient using this temperature coefficient, the modulated normalized attention weights are obtained:

[0071] in Let be the set of neighboring nodes of node i. The mathematical principle behind this temperature scaling modulation is that the temperature parameter in the Softmax function controls the "steepness" of the output probability distribution.

[0072] when When the input of Softmax is compressed, the output distribution tends to be flattened (i.e., the attention weights of each neighbor node are more uniform), allowing high uncertainty nodes to absorb contextual information from a wider neighborhood. when At this time, the attention weight distribution is consistent with the standard GAT, and the node absorbs neighbor information only in a normal proportion.

[0073] The essential difference between this modulation mechanism and the simple additive modulation in the prior art (which has no practical effect on softmax normalization) is that temperature scaling regulates the breadth of information absorption by changing the entropy value of the softmax distribution, rather than adding to the exponential terms that can be canceled out by normalization.

[0074] The output of each attention head is projected through its respective output matrix. After mapping, summation and aggregation yield the final context-aware updated features:

[0075] Where K is the number of attention heads. This is the output projection matrix of the k-th head. It is an exponential linear unit activation function. In practical implementations, multiple layers (e.g., 2-3 layers) of GAT can be stacked to expand the receptive field, allowing information to propagate further in the graph.

[0076] 4.4 GAT Output Decoding The updated node features are decoded to obtain refined confidence scores and classification labels. Let the updated node features be... The dimension is (Right now (Determined by the output dimension of GAT). The classification decoding network will update the node features. After passing through two layers of MLP, the refined class probability distribution is output via Softmax:

[0077] in For classification and decoding of multilayer perceptrons, its structure is as follows: The network consists of two fully connected layers, where C represents the total number of fine-grained device categories. The confidence decoding network updates the node features. Compared with the original initial confidence level and cognitive uncertainty After concatenating the three pieces of information, the confidence level is refined through a two-layer MLP and Sigmoid activation:

[0078] in For confidence-based decoding, the structure of the multilayer perceptron is as follows: A two-layer fully connected network, The activation function is Sigmoid. The refined confidence score integrates three aspects of information: context-optimized semantic features, original detection confidence score, and cognitive uncertainty assessment. After GAT inference and decoding, the classification label (the category corresponding to the maximum refined category probability) and confidence score (refined confidence score) of each node are updated to obtain a list of optimized detection results.

[0079] V. Keyframe Selection and Visualization (Step S5) like Figure 1 and Figure 8 As shown, the final step S5 of this embodiment is to generate a visualization image with bounding boxes and device type labels based on the optimized detection result list and the key frame images selected from the live video stream.

[0080] 5.1 Keyframe Selection First, keyframes are selected from the video stream acquired in step S1 for visualization. For each frame in the video stream, the average initial confidence score of all initial detection result nodes is calculated. and the coverage of detected device categories ,in , Let be the number of device categories detected in frame t. The total number of device categories expected to appear in the scene (preset based on prior knowledge of the in-home security check scenario). The candidate frame with the largest product of average initial confidence and coverage parameter is selected as the keyframe image.

[0081] This selection criterion ensures that the selected frames have both high detection reliability and cover as many types of gas appliances as possible.

[0082] 5.2 Visualization Generation Subsequently, on the original keyframe image, the list of optimized detection results is traversed, and an image processing library (such as OpenCV) is used to draw bounding boxes for each detection result (with preset colors and line thicknesses, such as green borders and 2-pixel line widths), and device type labels and refined confidence scores are added. A visualization image with bounding boxes and device type labels is generated, saved as a JPEG or PNG file, and displayed on a mobile device screen for security personnel or uploaded to a cloud server for archiving. Preferably, the specific steps are as follows: First, draw the bounding boxes: extract the coordinates of the bounding boxes from the detection results (top left and bottom right corner coordinates), and draw rectangular boxes on the keyframe images using OpenCV or Pillow image processing libraries. Preset the color (e.g., green) and line thickness (e.g., 2 pixels) of the bounding boxes to ensure clear visibility under different backgrounds.

[0083] Then, label the results: extract the final device type label and refine the confidence level from the detection results, and place the text information in the upper left corner or above the bounding box. Preset the font type, font size (e.g., 16pt), and text color (e.g., white), and add a semi-transparent background color block below the text to improve contrast and readability.

[0084] Repeat the above drawing operation for all test results to finally generate a visual image containing the bounding boxes and type labels of all gas equipment. Save it as a JPEG or PNG file and display it on a mobile device screen for safety inspectors or upload it to a cloud server for archiving.

[0085] In this embodiment, as Figure 8 As shown, the training strategy is as follows: 1. First stage: Spatiotemporal feature extraction and detection training On a video dataset labeled with gas equipment types and bounding boxes, an end-to-end spatiotemporal visual feature extraction network (EfficientNet-B3, CBAM, and ConvLSTM) and a hierarchical detection head component were trained. Optimization objectives included detection loss (bounding box regression loss and classification loss) and uncertainty-aware loss. EfficientNet-B3 uses ImageNet pre-trained weights for initialization, and the ConvLSTM and detector head are initialized using Xavier. It employs the Adam optimizer with an initial learning rate of [value missing]. A cosine annealing learning rate scheduling strategy was employed. The number of training epochs was 100-200, and the batch size was 8-16 video sequences. Data augmentation included random horizontal flipping, random color jittering, and random frame sampling.

[0086] The training dataset contains numerous video clips depicting in-home security inspection scenarios. Each video clip is annotated with the bounding box and category label of the gas equipment in each frame. The video clips should cover various scene variations: different lighting conditions (natural light, indoor lighting, mixed light sources), different equipment combinations (kitchen stove + water heater + gas meter, balcony wall-mounted boiler + pipes, etc.), different degrees of occlusion (no occlusion, partial occlusion, severe occlusion), and different scanning paths. For equipment categories that require further subdivision (e.g., stoves need to be subdivided into embedded and countertop types), the training data also includes subdivision type labels. For uncertainty-aware training, the annotations should include difficult samples (e.g., devices with highly blurred appearances or severe occlusion) to enable the model to learn to distinguish between certain and uncertain predictions.

[0087] Preferably, the joint loss function in the first stage for:

[0088] in The standard YOLO detection loss (including classification loss, bounding box regression loss, and confidence loss) is used. For the aforementioned perceived loss due to uncertainty, This is the balance coefficient. In a specific example, .

[0089] 2. Second Phase: Training the Graph Reasoning Engine The parameters of the spatiotemporal visual feature extraction network and hierarchical detection head components trained in the first stage are frozen. Using the initial detection result node set (including cognitive uncertainty) output by these components on the training set as input, a dynamic spatial topology inference engine (GAT layer and decoding network) based on a graph attention network is trained on a subset of data labeled with spatial relationships of gas equipment. The optimization objective is the graph inference loss, including the cross-entropy loss between the refined class probability and the true label, and the mean squared error loss between the refined confidence and the true confidence.

[0090] Specifically, the model trained in the first stage is run on the training set video to collect the initial detection result node set (containing node features) for each frame. Equipment labels Bounding box Initial confidence level and cognitive uncertainty For each node set, spatial distance encoding is calculated based on the bounding box coordinates. The sparse adjacency mask matrix M is used. Simultaneously, for each initial detection result node, its correctly classified label and correct confidence score, verified by manual review, are labeled as the training target for the graph attention network inference engine.

[0091] Loss function in the second stage Loss for graph reasoning:

[0092] in For refining category probabilities Cross-entropy loss between the actual label and the real label To refine confidence level The mean squared error loss between the true confidence level and the actual confidence level. This is the balance coefficient. In a specific example, Learnable uncertainty modulation coefficients Training also takes place during this phase. Through backpropagation, Automatically learn the optimal modulation strength—if uncertainty modulation helps improve classification accuracy. It will increase; conversely, it will decrease.

[0093] 3. End-to-end joint fine-tuning After the second phase of training is completed, further end-to-end joint fine-tuning can be performed: all parameters are unfrozen, and overall fine-tuning is carried out using the joint loss function. To detect loss and perceive loss due to uncertainty The weighted sum of graph reasoning losses, for example:

[0094] in These are the weighting coefficients for the graph inference loss. In a specific instance, Joint fine-tuning enables globally coordinated optimization of parameters across the entire pipeline. Gradient pass-through allows the graph attention network to make inverse adjustments to the detection head—for example, if the graph attention network detects certain features that are detrimental to contextual inference, the gradient signal is fed back to the detection head, prompting it to adjust its feature representation. Joint fine-tuning uses a relatively small learning rate; in a specific instance, the learning rate is set to... Train 20 to 50 times.

[0095] Example 2 This embodiment provides a gas equipment classification and detection device based on spatiotemporal feature fusion and dynamic graph attention reasoning. This device corresponds one-to-one with the method in Embodiment 1 and is used to implement the method in Embodiment 1. See also Figure 2 The device includes: The video stream acquisition module is configured to acquire the on-site video stream captured during security check scanning and extract continuous multi-frame time-series images from it. This module runs on the mobile device, activates the camera module and continuously records the video stream, extracting time-series images according to a fixed frame interval sampling strategy.

[0096] The spatiotemporal visual feature extraction module is configured to extract spatiotemporal visual features from multi-frame time-series images. It outputs a spatiotemporal device-focused visual feature map by fusing spatial feature extraction with inter-frame temporal features. This module includes: a backbone network using the EfficientNet-B3 network for multi-scale spatial feature extraction; an attention module using a Convolutional Block Attention (CBAM) module for dual attention enhancement in both channel and spatial dimensions; and a temporal fusion module using a Convolutional Long Short-Term Memory (ConvLSTM) network for gated temporal fusion and compensation of spatial features from consecutive frames.

[0097] The uncertainty-aware hierarchical detection module is configured to input spatiotemporal device focused visual feature maps into a hierarchical detection head component for target detection and fine-grained classification. An uncertainty-aware mechanism is introduced during the fine-grained classification process to output an initial detection result node set containing device type labels, bounding box information, initial confidence levels, and cognitive uncertainty. This module includes: a first-level detection module using a YOLO detection head structure for broad device category detection; and a second-level detection module employing an uncertainty-aware fine-grained classifier, outputting both category probabilities and cognitive uncertainty.

[0098] The dynamic spatial topology reasoning module is configured to build a dynamic spatial topology reasoning engine based on a graph attention network. It maps the initial detection result node set into a graph structure, dynamically constructs the edges of the graph structure based on the relative spatial relationships between nodes, and aggregates the contextual semantic features of adjacent nodes through a graph attention mechanism to update the node features. Specifically, the attention weights of the graph attention network are modulated based on cognitive uncertainty so that nodes with higher cognitive uncertainty receive increased contextual compensation. Subsequently, the updated node features are decoded to obtain refined confidence scores and classification labels, resulting in an optimized list of detection results.

[0099] The keyframe visualization module is configured to generate visualization images with bounding boxes and device type labels based on an optimized list of detection results and keyframe images selected from the live video stream.

[0100] The specific implementation methods of the above modules are consistent with the descriptions of steps S1 to S5 in the first method embodiment, and will not be repeated here.

[0101] Example 3 This embodiment also provides an electronic device, see reference. Figure 9It includes a memory 402 and a processor 401, wherein the memory 402 stores a computer program and the processor 401 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0102] Specifically, the processor 401 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0103] The memory 402 may include a mass storage device for data or instructions. For example, and not limitingly, the memory 402 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 402 may include removable or non-removable (or fixed) media. Where appropriate, the memory 402 may be internal or external to a data processing device. In a particular embodiment, the memory 402 is non-volatile memory. In a particular embodiment, the memory 402 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random Access Memory (FPMDRAM), Extended Data Out Dynamic Random Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0104] The memory 402 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 401.

[0105] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement any of the gas equipment classification and detection methods based on spatiotemporal feature fusion and dynamic graph attention reasoning in the above embodiments.

[0106] Optionally, the electronic device may further include a transmission device 403 and an input / output device 404, wherein the transmission device 403 is connected to the processor 401 and the input / output device 404 is connected to the processor 401.

[0107] The transmission device 403 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 403 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0108] Input / output device 404 is used for inputting or outputting information. It can be a speaker, microphone, monitor, or keyboard.

[0109] Example 4 This embodiment also provides a readable storage medium storing a computer program, which includes program code for controlling a process to execute the process, the process including the gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning according to Embodiment 1.

[0110] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0111] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0112] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer-executable components configured to perform embodiments when the program is run. One or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted that any block in the logical flow of the figures may represent a program step, or interconnected logical circuitry, blocks and functions, or a combination of program steps and logical circuitry, blocks and functions. The software may be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0113] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] The above embodiments are merely illustrative of several implementations of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the appended claims.

Claims

1. A gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning, characterized in that, Includes the following steps: S1: Acquire the on-site video stream collected during the security check scanning process and extract continuous multi-frame time series images from it; S2: Extract spatiotemporal visual features from the multi-frame time-series images, and output a spatiotemporal device focusing visual feature map by fusing spatial feature extraction with inter-frame temporal features. S3: Input the spatiotemporal device focused visual feature map into the hierarchical detection head component for target detection and fine-grained classification, and introduce an uncertainty perception mechanism in the fine-grained classification process to output an initial detection result node set containing device type label, bounding box information, initial confidence and cognitive uncertainty; S4: Construct a dynamic spatial topology reasoning engine based on a graph attention network. Map the initial detection result node set to a graph structure. Dynamically construct the edges of the graph structure based on the relative spatial relationships between nodes. Aggregate the contextual semantic features of adjacent nodes through a graph attention mechanism to update node features. The attention weights of the graph attention network are modulated based on the cognitive uncertainty so that nodes with higher cognitive uncertainty receive increased contextual compensation. The updated node features are then decoded to obtain refined confidence and classification labels, resulting in an optimized detection result list. S5: Based on the optimized detection result list and the keyframe images selected from the live video stream, generate a visualization image with bounding boxes and device type labels.

2. The gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning as described in claim 1, characterized in that, Spatiotemporal visual feature extraction of the multi-frame time-series images includes: Spatial features of single-frame images are extracted using a backbone network and an attention module; The spatial features of consecutive frames are input into a spatiotemporal long short-term memory convolutional network. The spatiotemporal long short-term memory convolutional network performs temporal fusion of the spatial features of consecutive frames through a gating mechanism of input gate, forget gate and output gate. The forget gate adaptively adjusts the retention ratio of historical frame information according to the feature quality of the current frame. The input gate controls the writing ratio of new information in the current frame. The hidden state output of the last frame of the time series is taken as the focusing visual feature map of the spatiotemporal device, so as to use the memory state of historical frames to temporally compensate for the feature loss in the current frame caused by occlusion or reflection.

3. The gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning as described in claim 1, characterized in that, The hierarchical detection head assembly includes a first-level detection module and a second-level detection module. The first-level detection module is used to detect the major categories of equipment. When the detected major category of equipment does not need further subdivision, the major category label is used as the final equipment type label, and the corresponding cognitive uncertainty is set to a preset low value. When the detected major category of equipment needs further subdivision, the corresponding equipment region features are extracted from the spatiotemporal equipment focusing visual feature map and input into the second-level detection module for fine-grained classification. The second-level detection module outputs the corresponding cognitive uncertainty while outputting the fine-grained classification result.

4. The gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning as described in claim 3, characterized in that, The uncertainty perception mechanism is implemented through a dynamic loss constraint function, which includes an adaptive loss weighting term and a regularization term. The adaptive loss weighting term is configured to reduce the weight of the classification loss of the sample according to the predicted cognitive uncertainty, and the regularization term is configured to impose a lower bound constraint on the cognitive uncertainty to prevent overestimation.

5. The gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning as described in claim 1, characterized in that, The relative spatial relationship is used in the graph attention mechanism calculation in the form of spatial distance encoding. The spatial distance encoding is a multi-dimensional vector. The feature components contained in the multi-dimensional vector include at least: the horizontal relative offset component, the vertical relative offset component, the logarithmic Euclidean distance component, and the intersection-union ratio component between the bounding boxes in the normalized image coordinate system.

6. The gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning as described in claim 1, characterized in that, The method of dynamically constructing edges of the graph structure based on the relative spatial relationship between nodes includes: calculating the normalized Euclidean distance between the center points of the bounding boxes of each node; constructing edges between corresponding nodes when the normalized Euclidean distance is lower than a preset distance threshold; and forcibly retaining a preset number of nearest neighbor edges for each node to ensure the connectivity of the graph structure and form a sparse connected graph.

7. The gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning as described in claim 1, characterized in that, Modulating the attention weights of the graph attention network based on the cognitive uncertainty includes: When calculating the normalized attention weights between nodes, a temperature scaling modulation based on the cognitive uncertainty is introduced. The uncertainty temperature coefficient of a node is defined to be positively correlated with the cognitive uncertainty. The attention coefficient is scaled using the temperature coefficient, so that the attention weight distribution of nodes with high cognitive uncertainty tends to be flattened, thereby absorbing contextual information from a wider neighborhood. The updated node features are decoded to obtain refined confidence scores and classification labels, including: The updated node features are mapped to a refined category probability distribution through a classification decoding network, and the updated node features are concatenated with the original initial confidence and cognitive uncertainty information and then mapped to a refined confidence through a confidence decoding network.

8. A gas equipment classification and detection device based on spatiotemporal feature fusion and dynamic graph attention reasoning, characterized in that, include: The video stream acquisition module is configured to acquire the on-site video stream collected during the security check scanning process and extract continuous multi-frame time series images from it; The spatiotemporal visual feature extraction module is configured to extract spatiotemporal visual features from the multi-frame time-series images, and output a spatiotemporal device focusing visual feature map by fusing spatial feature extraction with inter-frame temporal features. The uncertainty-aware hierarchical detection module is configured to input the spatiotemporal device focused visual feature map into the hierarchical detection head component for target detection and fine-grained classification, and introduce an uncertainty-aware mechanism in the fine-grained classification process to output an initial detection result node set containing device type label, bounding box information, initial confidence and cognitive uncertainty; The dynamic spatial topology reasoning module is configured to construct a dynamic spatial topology reasoning engine based on a graph attention network. It maps the initial detection result node set into a graph structure, dynamically constructs the edges of the graph structure based on the relative spatial relationships between nodes, and aggregates the contextual semantic features of adjacent nodes through a graph attention mechanism to update node features. Specifically, it applies temperature scaling modulation to the attention weights of the graph attention network based on the cognitive uncertainty, so that nodes with higher cognitive uncertainty receive increased contextual compensation. Subsequently, it decodes the updated node features to obtain refined confidence and classification labels, resulting in an optimized detection result list. The keyframe visualization module is configured to generate a visualization image with bounding boxes and device type labels based on the optimized detection result list and keyframe images selected from the live video stream.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to execute the gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, the computer program including program code for controlling a process to execute the process, the process including the gas equipment classification and detection method based on spatiotemporal feature fusion and dynamic graph attention reasoning according to any one of claims 1 to 7.

Images