Method and device for identifying hidden danger of power transmission line channel, medium and equipment

Abstract

The invention discloses an electric power transmission line channel hidden danger identification method and device, a medium and equipment. The method comprises the following steps: performing feature extraction on acquired multi-modal data of a target scene in a power transmission line to obtain multi-modal data features; performing cross-modal fusion on the multi-modal data features to generate multi-modal fusion features; performing hidden danger detection on the multi-modal fusion features through a pre-trained fusion type detection model, and outputting a structured hidden danger set; performing risk level reasoning on the structured hidden danger set according to the safety supervision knowledge base to generate a reasoning result; and generating a global operation strategy of the target scene according to the reasoning result in combination with the actual safety supervision and operation scheduling scene of the power transmission line.

Description

Technical Field

[0001] This invention relates to the field of power transmission line hazard identification technology, and more specifically, to a method, apparatus, medium and equipment for identifying hazards in power transmission line corridors. Background Technology

[0002] With the continuous expansion of power systems and the increasing complexity of transmission line corridors, safety hazards within transmission channels are becoming more diverse and dynamic. Traditional hazard identification in power transmission line corridors mainly relies on manual inspections, fixed-point monitoring, and AI algorithms based on single-vision recognition to detect hazards such as personnel violations, equipment malfunctions, and defects in the working environment. While these methods have improved the safety management level of transmission lines to some extent, a series of problems still need to be addressed. First, transmission channels, as open and long-distance spatial scenarios, exhibit significant spatial topological complexity. Existing hazard identification systems typically only focus on detecting local targets in images (such as personnel, equipment, and warning lines), lacking modeling of the complete spatial structure of "transmission line-tower-working environment," making it difficult to identify hazards dependent on spatial relationships, such as "risks of cross-tower operations" and "safe distances for crane intrusions." Secondly, existing systems lack sufficient understanding of business interpretation and cannot integrate textual information such as work permits and safety regulations. This means that while the model can detect anomalies, it cannot determine whether these anomalies truly violate operational procedures or pose safety risks. Safety inspectors still need to rely on human experience, increasing cognitive burden and the probability of misjudgment. More seriously, current technologies mostly use single-frame images or short video clips as input, lacking the ability to model the dynamic changes of hazards over time and with environmental changes. This makes it difficult to promptly detect and warn of potential risks that "currently show no significant anomalies but will evolve into major hazards over time or with environmental changes." Furthermore, safety hazards in transmission lines are often closely related to a large amount of tacit knowledge, such as unstructured business knowledge like safety regulations, historical accident cases, and accident investigation reports. This knowledge is not fully utilized by existing systems, resulting in identification results lacking business interpretability and decision support value.

[0003] In summary, existing methods for identifying potential hazards in power transmission channels generally suffer from four major technical shortcomings: lack of spatial modeling, insufficient understanding of business operations, lack of dynamic trends, and failure to utilize tacit knowledge. These shortcomings make it difficult to meet the actual needs of intelligent safety monitoring of power transmission lines, which requires comprehensiveness, accuracy, dynamism, and interpretability. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method, apparatus, medium, and equipment for identifying potential hazards in power transmission line corridors.

[0005] According to one aspect of the present invention, a method for identifying potential hazards in power transmission line corridors is provided, comprising:

[0006] Feature extraction is performed on the multimodal data of the target scene collected in the power transmission line to obtain multimodal data features;

[0007] Cross-modal fusion of multimodal data features generates multimodal fused features;

[0008] A pre-trained fusion detection model is used to detect hidden dangers based on multimodal fusion features, and a structured set of hidden dangers is output.

[0009] Based on the safety supervision knowledge base, risk level inference is performed on the structured set of hidden dangers, and inference results are generated;

[0010] Based on the reasoning results, and combined with the actual safety monitoring and operation scheduling scenarios of power transmission lines, a global operation strategy for the target scenario is generated.

[0011] Optionally, the multimodal data includes: image / video data, work permits and safety supervision documents, transmission line spatial topology data, and environmental dynamic data. Among them, the image / video data includes image / video data of each key node of the transmission line corridor; the work permits and safety supervision documents include work task information, risk control measures, and safety supervision clauses; the transmission line spatial topology data includes the spatial coordinates and topological relationships of power towers, tower crossing areas, and work areas; and the environmental dynamic data includes real-time wind speed, air temperature, temperature, and air pressure.

[0012] Optionally, feature extraction is performed on the multimodal data of the target scene collected in the power transmission line to obtain multimodal data features, including:

[0013] A convolutional neural network incorporating a self-attention mechanism is used to extract features from image / video data to obtain visual features;

[0014] A pre-trained language model incorporating a self-attention mechanism is used to extract features from work orders and safety supervision texts to obtain text features;

[0015] A graph neural network is used to construct the spatial topology structure of the transmission line based on the spatial topology data, and the spatial topology features are determined based on the spatial topology structure.

[0016] Normalize the environmental dynamic data to obtain the environmental dynamic characteristics;

[0017] Visual features, text features, spatial topological features, and environmental dynamic features are standardized to obtain multimodal data features.

[0018] Optionally, the expression for standardization is:

[0019]

[0020] In the formula, X represents visual feature F I Text features F T Spatial topological features S G and environmental dynamic characteristics F E μ X σ is the mean of this type of feature; X F is the standard deviation of this type of feature; X These are the standardized features.

[0021] Optionally, cross-modal fusion is performed on the multimodal data features to generate multimodal fused features, including:

[0022] A dual-stream cross-attention mechanism is used to fuse visual features and textual features to obtain the first fused feature;

[0023] The first fusion feature and the spatial topological feature are fused using a graph neural network to obtain the second fusion feature;

[0024] By introducing an environment-based weighted graph neural network with adjacent edges, the second fusion feature and the environmental dynamic feature are fused to obtain the multimodal fusion feature.

[0025] Optionally, the expression for the environment-based weighting term is:

[0026]

[0027] In the formula, W represents the state of node i in the l-th layer of the graph neural network. l Let α be a learnable weight matrix. ij It is the attention weight between nodes i and j; the environmental perception factor f(F) E ) = r w ·wind+r t ·temp; where wind and temp represent the wind speed and temperature at the current node, respectively, r w and r t γ is a learnable weight parameter that controls the overall influence strength of this environmental item.

[0028] Optionally, the graph neural network, which introduces an environment-based weighting term for adjacent edges, incorporates a regularization term during training. The expression for the regular expression is:

[0029]

[0030] In the formula, Let σ represent the expected edge weights under ideal environment-driven conditions, where σ is the Sigmoid function, λ is the regularization coefficient, and wind... ij Let temp be the wind speed between node i and node j. ijLet be the temperature between node i and node j.

[0031] Optionally, a pre-trained fusion detection model is used to detect hidden dangers in the multimodal fusion features, outputting a structured set of hidden dangers, including:

[0032] A lightweight multilayer perceptron recognition head using a fusion detection model linearly maps multimodal fusion features to output semantic vectors of candidate potential hazards;

[0033] A multi-head feature decoupling mechanism of the fusion detection model is used to extract spatial location information, text rule information, and environmental state information from the multimodal fusion features;

[0034] Spatial location information, textual rule information, and environmental state information are bound to the semantic vector of each candidate hazard to output a structured hazard set.

[0035] Optionally, the semantic vector of the candidate hidden danger is calculated as follows:

[0036]

[0037] In the formula, W h Let b be the weight matrix. h For bias terms, Let F be the feature vector of the i-th hazard candidate region. S′ The probability of the existence of this hidden danger is obtained after applying an activation function to the multimodal fusion feature.

[0038] Optionally, the structured hazard set includes the hazard type, the spatial location of the hazard, its corresponding work permit / safety supervision description, environmental condition information when the hazard occurs, and the detection confidence level.

[0039] Optionally, risk level inference is performed on the structured hazard set based on the safety supervision knowledge base to generate inference results, including:

[0040] Linear projection is performed on each structured hazard in the set of structured hazards, its spatial location, and the corresponding work order / safety supervision description to construct a joint code for each structured hazard;

[0041] Based on joint coding, the knowledge retrieval results obtained from the safety supervision knowledge base and the dynamic environmental information of each structured hazard are input into the language generation model, and the risk level of each structured hazard and the business explanation in natural language form are output.

[0042] Based on the hazard type, knowledge retrieval results, risk level, and business explanation for each structured hazard, inference results are generated.

[0043] Optionally, based on the reasoning results and combined with the actual safety monitoring and operation scheduling scenarios of power transmission lines, a global operation strategy for the target scenario is generated, including:

[0044] Based on the risk level and business interpretation of each structured hazard in the structured hazard set, generate hazard handling recommendations for each structured hazard;

[0045] Based on the risk level of all structured hazards in the structured hazard set, a global operation control strategy is generated;

[0046] Based on the hazard management recommendations for each structured hazard and the overall operation control strategy, a global operation strategy for the target scenario is generated.

[0047] According to another aspect of the present invention, a device for identifying potential hazards in power transmission line corridors is provided, comprising:

[0048] The extraction module is used to extract features from the multimodal data of the target scene in the collected power transmission line to obtain multimodal data features;

[0049] The fusion module is used to perform cross-modal fusion of multimodal data features to generate multimodal fused features;

[0050] The detection module is used to detect potential hazards by using a pre-trained fusion detection model to analyze multimodal fusion features and output a structured set of potential hazards.

[0051] The reasoning module is used to perform risk level reasoning on the structured set of hidden dangers based on the safety supervision knowledge base and generate reasoning results;

[0052] The generation module is used to generate a global operation strategy for the target scenario based on the reasoning results and the actual safety monitoring and operation scheduling scenarios of power transmission lines.

[0053] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the methods described in any of the above aspects of the present invention.

[0054] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method described in any of the preceding aspects of the present invention.

[0055] Therefore, this invention proposes a method for identifying hidden dangers in power transmission line corridors that integrates spatial structure, operational information, and the evolution trend of potential hazards. By introducing an innovative multimodal data fusion mechanism, this invention connects multiple information sources, including on-site image information, work permits and safety supervision documents, and transmission line spatial topology, solving the problem of insufficient perception of spatial relationships and operational interpretation in traditional methods. Furthermore, the system constructs a dynamic prediction mechanism for the evolution of hazards over time and in the environment, overcoming the shortcomings of existing methods in modeling hazard trend evolution and enabling early identification of potential hazards with spatiotemporal evolution characteristics. In addition, this invention designs a hazard knowledge reasoning and interpretation mechanism oriented towards safety supervision operations, fully utilizing tacit knowledge such as safety supervision regulations, accident cases, and risk assessment standards in the power transmission industry to support intelligent interpretation and compliance judgment of hazards. Through these innovations, this invention achieves a spatial-operational-temporal triple fusion of hazard identification, breaking through the limitations of existing technologies and significantly improving the accuracy, real-time performance, and intelligence level of hazard identification in power transmission line corridors, meeting the actual needs of the power industry for "comprehensive perception, dynamic early warning, and explainable decision-making" in intelligent safety supervision systems. Attached Figure Description

[0056] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures:

[0057] Figure 1 This is a flowchart illustrating a method for identifying potential hazards in power transmission line corridors provided by an exemplary embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of the structure of a device for identifying potential hazards in power transmission line channels provided in an exemplary embodiment of the present invention;

[0059] Figure 3 This is the structure of an electronic device provided in an exemplary embodiment of the present invention. Detailed Implementation

[0060] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0061] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0062] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of the present invention are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0063] It should also be understood that in the embodiments of the present invention, "multiple" can refer to two or more, and "at least one" can refer to one, two or more.

[0064] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more unless explicitly defined or given contrary instructions in the context.

[0065] Furthermore, the term "and / or" in this invention is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this invention generally indicates that the preceding and following related objects have an "or" relationship.

[0066] It should also be understood that the description of the various embodiments in this invention emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0067] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0068] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0069] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0070] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0071] The embodiments of this invention can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Well-known examples of terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0072] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0073] Exemplary methods

[0074] Figure 1 This is a flowchart illustrating a method for identifying potential hazards in power transmission line corridors according to an exemplary embodiment of the present invention. This embodiment can be applied to electronic devices, such as… Figure 1 As shown, the method 100 for identifying potential hazards in power transmission line corridors includes the following steps:

[0075] Step 101: Extract features from the multimodal data of the target scene in the collected power transmission line to obtain multimodal data features;

[0076] Step 102: Perform cross-modal fusion on the multimodal data features to generate multimodal fused features;

[0077] Step 103: Detect potential hazards by using a pre-trained fusion detection model to analyze the multimodal fusion features and output a structured set of potential hazards;

[0078] Step 104: Based on the safety supervision knowledge base, perform risk level reasoning on the structured hazard set and generate the reasoning results;

[0079] Step 105: Based on the reasoning results and combined with the actual safety monitoring and operation scheduling scenarios of power transmission lines, generate a global operation strategy for the target scenario.

[0080] Specifically, this invention addresses the technical problems existing in the prior art by proposing a method and system for identifying hidden dangers in power transmission line corridors that integrates spatial structure, operational information, and the evolution trend of potential hazards. This invention introduces an innovative multimodal data fusion mechanism, connecting multiple information sources such as "on-site image information, work permits and safety supervision texts, and transmission line spatial topology," thus solving the problem of insufficient perception of spatial relationships and operational interpretation in traditional methods. Furthermore, the system constructs a dynamic prediction mechanism for the evolution of hazards over time and in the environment, overcoming the shortcomings of existing methods in modeling the trend evolution of hazards, and enabling early identification of potential hazards with spatiotemporal evolution characteristics. In addition, this invention designs a hazard knowledge reasoning and interpretation mechanism oriented towards safety supervision operations, fully utilizing tacit knowledge such as safety supervision regulations, accident cases, and risk assessment standards in the power transmission industry to support intelligent interpretation and compliance judgment of hazards, improving the system's operational applicability and decision support capabilities. Through the above innovations, this invention achieves spatial-operational-temporal triple integration of hazard identification, breaking through the limitations of existing technologies. It significantly improves the accuracy, real-time performance, and intelligence level of hazard identification in power transmission line corridors, meeting the power industry's practical needs for intelligent safety monitoring systems that offer "comprehensive perception, dynamic early warning, and explainable decision-making." The specific implementation process is as follows:

[0081] Step 101: Collect multimodal data of the target scene, including on-site images / videos, work permits / safety supervision, GIS (Geographic Information System) topology and environmental data, and perform feature extraction to obtain corresponding visual features, text features, spatial topology features and environmental dynamic features.

[0082] Specifically, this step focuses on multimodal data acquisition and feature extraction in the identification of potential hazards in power transmission line corridors. The task objective is to complete the raw feature processing of "images / videos, work permits / safety supervision texts, transmission corridor spatial topology, and environmental dynamic data" for power safety supervision scenarios, and output standardized multimodal features to provide input for subsequent multimodal fusion modeling.

[0083] Specifically, four types of data are collected:

[0084] The on-site image / video data I comes from monitoring cameras and drone inspections at key nodes along the power transmission line corridor;

[0085] The work order and safety supervision document T contain work task information, risk control measures, and safety supervision clauses;

[0086] The spatial topology data G of the transmission line includes the spatial coordinates and topological relationships of the towers, tower crossing areas, and work areas;

[0087] Environmental dynamic data E includes real-time external environmental data related to operational risks, such as wind speed, temperature, humidity, and air pressure.

[0088] Furthermore, to ensure the independence of each modality's features and facilitate flexible combination in subsequent fusion modeling stages, this step designs four feature extraction mechanisms, applied to visual data, text data, spatial topology data, and environmental dynamic data, respectively. The specific design is as follows:

[0089] Visual Feature Extraction (Image / Video I): Visual feature extraction employs a Convolutional Neural Network (CNN) model to extract features from images. By using convolutional kernels of different sizes at different layers of the network, the model can capture visual information of key areas (such as tower bases, cross-towers, and boom areas) in the power transmission line operation site. In particular, a region attention mechanism enhances the focus on these high-risk areas. This mechanism incorporates self-attention into a multi-layer convolutional network, allowing the model to automatically adjust attention weights based on the importance of the region, thereby extracting more accurate region features. The extracted features are represented as multi-scale visual features F. I And maintain spatial location information.

[0090] Text Feature Extraction (Work Permit / Safety Supervision T): Text feature extraction employs a pre-trained language model (such as BERT, RoBERTa, etc.) to perform semantic parsing of work permits and safety supervision texts. The model processes a lexicon of terms in the power safety supervision field, focusing on identifying key information related to risk control (such as "no warning zone" and "no safety belt worn"), and weights the key information in the text through a self-attention mechanism. Finally, the extracted text features are represented as text semantic features F. T This is used for subsequent cross-modal learning.

[0091] Spatial Topology Feature Extraction (GIS Topology G): Spatial topology feature extraction uses graph neural networks (GNNs) to model the spatial topology of transmission lines. The input spatial data (tower locations, work areas, conductors, etc.) is first processed using node embedding methods in the GNN to generate node feature representations containing spatial coordinates and physical attributes. Edge relationships between nodes represent physical connections, and edge weights can be dynamically adjusted based on actual environmental factors (such as wind speed and work area type). Finally, the generated spatial topology feature representation is S. G It contains spatial relationship information.

[0092] Environmental dynamic feature extraction (environmental data E): Environmental dynamic feature extraction is based on real-time sensor data and combined with power operation standards to extract environmental features (such as wind speed, temperature, humidity, etc.) that affect safe operation. This data is then normalized to extract environmental risk indicators. These environmental dynamic features F E Used for subsequent multimodal fusion and risk assessment, reflecting the dynamic changes at the work site.

[0093] Furthermore, after feature extraction is complete, all extracted features (visual features F) I Text features F T Spatial topological features S G and environmental dynamic characteristics F E The features will be standardized to ensure they are on the same scale. The standardization formula is as follows:

[0094]

[0095] Where: X represents visual feature F I Text features F T Spatial topological features S G and environmental dynamic characteristics F E μ X σ is the mean of this type of feature; X F is the standard deviation of this type of feature; X These are the standardized features.

[0096] The final output feature F I ,F T ,S G ,F E The four standardized features will serve as inputs for step 102, entering the multimodal fusion modeling stage. While maintaining the independence of each modality, they will be prepared for subsequent cross-modal learning and integration.

[0097] Step 102: Perform cross-modal fusion modeling on the multimodal data to generate fused features that integrate visual features, text features, spatial topological features, and environmental dynamic features.

[0098] Specifically, this step follows the multimodal features output in step 101, including visual features F. I (Image / Video), Text Features F T (Work permit and safety supervision document), spatial topological features S G (Node Topology) and Environmental Dynamics F E (Wind speed, temperature, humidity, etc.). The task of this step is to construct a unified multimodal fusion feature F. S′ This is to support the subsequent hazard detection and hazard business reasoning modules in their global understanding of "image-text-space-environment", especially suitable for typical scenarios such as "work permit regulations-environmental changes-complex spatial structures" in power transmission line channels.

[0099] Unlike existing methods that only perform shallow fusion of images and text or images and space, this step innovatively introduces a visual-text cross-modal attention mechanism, a graph neural network spatial propagation structure, and a dynamic environment-based edge weight modulation mechanism to construct a deep embedding model with multimodal correlation and dynamic perception capabilities.

[0100] First, in terms of visual and text modal fusion, a two-stream cross-attention mechanism is adopted: two channels respectively input visual features F. I With text features F T Explicit information exchange is achieved through a cross-attention module. This mechanism, based on a multi-head attention structure, allows for efficient capture of the correspondence between hazard descriptions in sentences and actual operational behaviors in images and text. For example, if the text states "no fence has been set up," the system will emphasize the fenced area in the image. The fused output is the visual-text fusion feature F. IT It already possesses image perception capabilities for business semantics.

[0101] Secondly, F IT Spatial topological characteristics S of power transmission channels G This is combined with spatial propagation via a graph neural network (GNN). During this process, F... IT Each node in the graph (e.g., power tower, tower crossing, crane) is assigned an initial embedding vector. Subsequently, an adjacency-based A algorithm is performed between the nodes. ij Through propagation operations, the model can update node representations layer by layer, achieving multi-hop diffusion of spatial semantics. This stage outputs a representation F that fuses the spatial structure. ITG This enables the model to understand the spatial structure of the work site.

[0102] Furthermore, to incorporate dynamic environmental information, particularly the moderating effect of variables such as wind speed and temperature on hazard propagation, an edge weight modulation mechanism was designed during the GNN propagation process. This involves introducing environment-based weighting terms into the adjacent edges of the graph neural network. The specific calculation is as follows:

[0103]

[0104] in, W represents the state of node i in the l-th layer of the graph neural network. l Let α be a learnable weight matrix. ij This is the attention weight between nodes i and j. This attention value considers not only the structural distance between nodes but also the following environmental awareness factors:

[0105] f(F E ) = r w ·wind+r t ·temp (3)

[0106] Where wind and temp represent the wind speed and temperature at the current node, respectively, r w and r t As a learnable weight parameter, γ controls the overall influence strength of this environmental item. Through this design, changes in environmental conditions will dynamically affect the spread of hazard information in a specific area, making the model more inclined to increase the weight of risk information in areas with high wind speeds and abnormal temperatures.

[0107] To further enhance the model's focus on high-risk areas, a regularization term was introduced during training. This regularization term is used to constrain the model to have a higher responsiveness to changes in edge weights caused by "dynamic environmental changes". The regularization term is defined as follows:

[0108]

[0109] in, Let σ represent the expected edge weights under ideal environmental conditions, where σ is the sigmoid function and λ is the regularization coefficient. This design allows the model to assign higher weights to risk transmission between important nodes under dynamic conditions such as high wind speeds, thereby improving the ability to identify environmentally sensitive hazards.

[0110] Finally, the output multimodal fusion feature representation F S′ This approach aims to create a unified high-dimensional representation that integrates visual semantics, textual rules, spatial structure, and dynamic environments to support downstream hazard detection tasks. This fusion feature offers the following advantages:

[0111] In the image dimension, it retains key visual risk information; in the text dimension, it embeds work permits and safety supervision clauses; in the spatial dimension, it reflects the physical connection and propagation logic between power transmission structures; and in the environmental dimension, it has the ability to adaptively adjust to dynamic changes such as wind speed and temperature.

[0112] This step constructs a multimodal fusion feature F suitable for power transmission channel scenarios through a fusion mechanism of depth map structure modeling and environment-driven approaches. S′ This enables context alignment and risk propagation awareness among multi-source information, laying a high-quality semantic foundation for the subsequent hazard detection in step 103.

[0113] Step 103: Detect potential hazards using a pre-trained fusion detection model based on the fusion features, and output a set of structured hazards; the structured hazard data includes hazard type, hazard spatial location, corresponding work order / safety supervision description, environmental state when the hazard occurs, and detection confidence level.

[0114] Specifically, this step follows the multimodal fusion feature F output in step 102. S′This multimodal fusion feature integrates information from four modalities: image, text, spatial structure, and environmental dynamics, possessing complete context-aware capabilities. The core task of this step is to identify potential hazards in power transmission line corridors based on this fusion feature, and to output the hazard information in a structured manner, forming a standard hazard dataset H′, providing a unified input for subsequent business interpretation, reasoning, and the generation of handling suggestions.

[0115] Furthermore, unlike traditional object detection which only outputs bounding boxes and category labels, this step designs a fusion detection model targeting the multi-factor features of power hazards (type, spatial location, work permit terms, environmental conditions, etc.). This ensures that the model can not only identify the type of hazard but also analyze the spatial and environmental context in which it occurred. To achieve this goal, the model structure consists of two functional channels: one is a hazard target identification channel, used to determine the specific type of hazard in the image; the other is a context attribute extraction channel, used to extract the multi-modal fusion features F S′ Extract the spatial location of the potential hazard, related business explanations, and environmental conditions.

[0116] Furthermore, in its implementation, the model first employs a lightweight multilayer perceptron (MLP) as the hazard identification head, targeting F... S′ After linear mapping, the semantic vector F of the i-th candidate hidden danger region is output. hi The calculation is as follows:

[0117]

[0118] Among them, W h Let b be the weight matrix. h For bias terms, The feature vector of the i-th hazard candidate region is used to obtain the probability of the hazard's existence after passing through an activation function.

[0119] At the same time, the system also from F S′ Spatial location information is extracted from S in step 101 respectively. G Provided), text rule information (by F) T (represented by) and environmental state (by F) E The structured hazard set H′ is represented as follows: H′ is bound to each detected hazard type and used to form a structured output item. Unlike general object detection frameworks in image processing, this structure emphasizes "hazard-bound output in a multimodal context," enabling the model output to not only have recognition capabilities but also understand the business implications and environmental causes behind the hazards. The structured hazard set H′ output takes the following form:

[0120]

[0121] Among them, h iIndicates the type of hazard identified; loc i This indicates the spatial location of potential hazards in the transmission topology diagram (e.g., cross-tower areas, tower bases, etc.), represented by F. S′ Spatial embedding module decoding; task i For the work permit content or safety supervision clauses associated with this hazard, the text feature F is used to integrate the text. T Obtained by mapping; env i This represents the environmental state information at the time of hazard identification, extracted from the F feature. E part; The confidence level for identifying this potential hazard instance.

[0122] Furthermore, to enhance the sensitivity of the detection model to multi-factor hazards, this step further employs a multi-head feature decoupling mechanism for F. S′ By performing channel-specific encoding, the model is guided to focus on image morphological features, textual rule elements, and spatial-environmental context in different subspaces during training, thereby achieving more robust hazard identification.

[0123] In summary, this step involves fusing the four-modal features F. S′ The detection and decoding of the system outputs a structurally standardized dataset H′ of potential hazards, which for the first time realizes a joint detection mechanism of "hazard-space-business-environment" for power transmission channels, providing key data support for the intelligent closed loop of the overall patent solution.

[0124] Step 104: Combine the safety supervision knowledge base to infer the risk level of the structured hazard set RAG (retrieval-enhanced generation) and generate inference results; wherein, the inference results include hazard type, knowledge retrieval results, current hazard risk level, and business explanation in natural language form.

[0125] Specifically, this step uses the structured hazard data output in step 103. The input is [characteristic], and the goal is to infer the risk level L of the identified hazards by combining rule knowledge from the field of safety supervision with operational context information. i It outputs a business-specific explanation with semantic interpretation capabilities. i This forms a set of inference results C for generating subsequent business handling suggestions. Unlike traditional rule matching or static classification methods, this step adopts a retrieval-augmented generation (RAG) mechanism, introducing structured hazards as queries. Through vectorized retrieval in the safety supervision knowledge base, and then inputting them together with dynamic environmental information into the generation model, semantic hazard level inference with scene understanding capabilities is achieved.

[0126] First, for each potential hazard hi Combined with its spatial location loc i and related work ticket content tasks i Construct a joint encoding representation q i This serves as the query vector for vectorized retrieval. This representation is obtained through linear projection:

[0127] q i =W q ·Concat(h i ,loc i ,task i (7)

[0128] Where Concat(·) represents the feature concatenation operation, W q This is a trainable weight matrix. The query vector q... i The input is fed into the embedded index of the safety supervision knowledge base K for vector retrieval, returning a set of the most relevant knowledge retrieval results r. i Its content includes the safety regulations that the potential hazard may violate, historical accident cases, and standard handling procedures. This knowledge serves as semantic support, providing a basis for risk interpretation in subsequent generation tasks.

[0129] Furthermore, in the reasoning generation stage, the knowledge retrieval results r obtained from the retrieval are combined... i and dynamic environment information env i Together, they serve as context input to the language generation model G. gen In the middle, the risk level L of the current hidden danger is generated. i and business explanation in natural language form i Environment data (env) i Instead of being treated as an external variable in post-processing, it guides the generator to focus on environmental conditions within the model context as a "scene cue." For example, "Current wind speed is 15 m / s, operation type is high-altitude cross-tower operation," this cue will significantly influence the model's assessment of the risk level. Finally, the structure of the system output C is as follows:

[0130]

[0131] Where: h i Indicates the type of hazard; r i The results are for knowledge retrieval, including safety supervision regulations and case support; L i The current risk level of potential hazards (e.g., high, medium, low); explain i This is a semantically interpretable text that combines knowledge and context to assist in on-site decision-making and oversight.

[0132] The innovations of this step are mainly reflected in the following three aspects: First, when constructing the retrieval vector, the hazard information considers both "spatial location" and "business terms," ​​enabling the RAG model to perform deeper semantic-level retrieval and matching; Second, dynamic environmental variables are used as input components in the language generation stage for the first time, no longer processed in isolation, thereby improving the model's response capability under special operating conditions (such as strong winds and high temperatures); Third, the generated explain... i The output combines multi-source information including "potential hazards, environment, and business interpretation," giving the system good business interpretability and providing context-complete input support for downstream decision-making modules.

[0133] This mechanism is particularly crucial in the scenario of identifying hidden dangers in power transmission channels. It can effectively cope with the characteristics of the environment in which hidden dangers occur, such as strong rule dependence and complex spatial structure. It realizes a closed-loop transition from "detection results" to "compliance interpretation + dynamic risk judgment". It is an important bridge link connecting the perception module and the intelligent decision-making module in this invention.

[0134] Step 105: Based on the reasoning results, combined with the actual safety supervision and operation scheduling scenarios of power transmission lines, generate targeted safety handling suggestions and operation plans, complete the decision output, and directly serve on-site safety supervision personnel and dispatching command.

[0135] Specifically, this step follows the reasoning result output in step 104. This includes the types of potential hazards, related safety supervision rules, risk levels under dynamic environments, and deduced business explanations. The goal is to generate safety handling suggestions and operational plans for actual safety supervision and operation scheduling scenarios of power transmission lines, thereby deriving the final decision output D and providing intelligent and urgent risk control measures for on-site safety supervisors and dispatching personnel.

[0136] Among them, the decision-making process for identifying potential hazards in power transmission lines requires consideration of the hazard risk level L. i Spatial location node loc i Dynamic environment characteristics (env) i Safety supervision clauses i By comprehensively analyzing these factors, a reasonable safety response plan can be generated. For example, when there is a "not wearing a safety belt" hazard in the tower crossing work area, if the ambient wind speed exceeds 12m / s, the system should automatically generate an emergency measure of "immediately stop tower crossing work and evacuate," rather than a low-intensity measure such as "strengthen patrols."

[0137] Therefore, this step designs a decision generation mechanism M based on the reasoning results. decideThe core task of this mechanism is to drive the generation of specific decision-making strategies based on the results of hazard reasoning. Decision generation includes two levels:

[0138] Generation of handling suggestions for individual hazards: For each hazard h i Combined with its risk level L i Explain the business i Generate specific suggestions for handling potential hazards. i Examples include "re-wearing safety helmets" and "suspending operations."

[0139] Generation of operational control strategies based on overall safety situation: comprehensively considering all potential hazards. i and its risk level L i Assess the overall safety situation at the work site and output a comprehensive work control strategy. global Such as "Operation continues", "Operation suspended" or "Complete evacuation".

[0140] Furthermore, the specific generation process is as follows: First, M decide According to each hidden danger h i Its associated risk level L i Explain the business i Query the built-in decision rule base to generate hazard handling suggestions. i This rule base contains business terms, environmental factors, historical cases, etc., related to potential hazards. For example, when L... i When the risk level is high, the system will generate robust emergency measures, such as:

[0141] If loc i For cross-tower regions and env i The wind speed is displayed as greater than 12 m / s, according to the safety monitoring system. i If wearing a safety belt is required for working at heights, then d i The order is "Immediately cease operations and initiate evacuation procedures";

[0142] If loc i For the base region and L i If the risk level is medium, the system will generate a standard response plan of "re-wear safety equipment and continue work".

[0143] Next, M decide It will be based on the risk level L of all potential hazards. i and environmental factors env i The system performs a global hazard risk assessment. When the number of high-risk hazards reaches a predetermined threshold, or when a single fatal hazard exists (such as not wearing a safety belt in a tower crossing work area), and the wind speed is higher than a certain critical value (e.g., 15 m / s), the system will automatically generate a global strategy D. globalFor example, "suspend operations" or "activate emergency plan". Conversely, when the risk of a hazard is low, the system generates measures such as "continue operations and strengthen inspections".

[0144] Ultimately, M decide The output D contains two parts:

[0145]

[0146] Where: D global This is a global operational strategy, indicating whether to continue operations or activate an emergency response; d i For each hidden danger h i The corresponding specific handling suggestions are targeted and practical.

[0147] Therefore, this invention proposes a method for identifying hidden dangers in power transmission line corridors that integrates spatial structure, operational information, and the evolution trend of potential hazards. By introducing an innovative multimodal data fusion mechanism, it connects multiple information sources, including on-site image information, work permits and safety supervision documents, and transmission line spatial topology, thus solving the problem of insufficient perception of spatial relationships and operational interpretation in traditional methods. Furthermore, the system constructs a dynamic prediction mechanism for the evolution of hazards over time and in the environment, overcoming the shortcoming of existing methods in lacking modeling of hazard trend evolution, and can identify potential hazards with spatiotemporal evolution characteristics in advance. In addition, this invention designs a hazard knowledge reasoning and interpretation mechanism oriented towards safety supervision operations, making full use of tacit knowledge such as safety supervision systems, accident cases, and risk assessment standards in the power transmission industry to support intelligent interpretation and compliance judgment of hazards, thereby improving the system's operational applicability and decision support capabilities. Through the above innovations, this invention achieves spatial-business-temporal triple integration of hazard identification, breaking through the limitations of existing technologies. It can significantly improve the accuracy, real-time performance, and intelligence level of hazard identification in power transmission line channels, meeting the actual needs of the power industry for "comprehensive perception, dynamic early warning, and explainable decision-making" in intelligent safety monitoring systems.

[0148] Exemplary device

[0149] Figure 2 This is a schematic diagram of the structure of a device for identifying potential hazards in power transmission line corridors provided in an exemplary embodiment of the present invention. Figure 2 As shown, the device 200 includes:

[0150] The extraction module 210 is used to extract features from the multimodal data of the target scene in the collected power transmission line and obtain multimodal data features;

[0151] The fusion module 220 is used to perform cross-modal fusion of multimodal data features to generate multimodal fused features;

[0152] The detection module 230 is used to detect hidden dangers by multimodal fusion features through a pre-trained fusion detection model and output a structured set of hidden dangers.

[0153] The reasoning module 240 is used to perform risk level reasoning on the structured hazard set based on the safety supervision knowledge base and generate reasoning results;

[0154] The generation module 250 is used to generate a global operation strategy for the target scenario based on the reasoning results and in combination with the actual safety monitoring and operation scheduling scenarios of power transmission lines.

[0155] Optionally, the multimodal data includes: image / video data, work permits and safety supervision documents, transmission line spatial topology data, and environmental dynamic data. Among them, the image / video data includes image / video data of each key node of the transmission line corridor; the work permits and safety supervision documents include work task information, risk control measures, and safety supervision clauses; the transmission line spatial topology data includes the spatial coordinates and topological relationships of power towers, tower crossing areas, and work areas; and the environmental dynamic data includes real-time wind speed, air temperature, temperature, and air pressure.

[0156] Optionally, the extraction module 210 includes:

[0157] The first extraction submodule is used to extract features from image / video data using a convolutional neural network with a self-attention mechanism to obtain visual features;

[0158] The second extraction submodule is used to extract features from work orders and safety supervision texts using a pre-trained language model incorporating a self-attention mechanism, thereby obtaining text features.

[0159] A submodule is constructed to use a graph neural network to construct the spatial topology of the transmission line based on the spatial topology data, and to determine the spatial topology features based on the spatial topology.

[0160] The acquisition submodule is used to normalize environmental dynamic data and acquire environmental dynamic characteristics.

[0161] The standardization submodule is used to standardize visual features, text features, spatial topological features, and environmental dynamic features respectively to obtain multimodal data features.

[0162] Optionally, the expression for standardization is:

[0163]

[0164] In the formula, X represents visual feature F I Text features F T Spatial topological features S G and environmental dynamic characteristics F E μX σ is the mean of this type of feature; X F is the standard deviation of this type of feature; X These are the standardized features.

[0165] Optionally, the fusion module 220 includes:

[0166] The first fusion submodule is used to fuse visual features and text features using a dual-stream cross-attention mechanism to obtain the first fused feature;

[0167] The second fusion submodule is used to fuse the first fusion feature and the spatial topological feature through a graph neural network to obtain the second fusion feature;

[0168] The third fusion submodule is used to fuse the second fusion feature and the environmental dynamic feature by introducing a graph neural network with environment-based weighting terms through adjacent edges, so as to obtain multimodal fusion features.

[0169] Optionally, the expression for the environment-based weighting term is:

[0170]

[0171] In the formula, W represents the state of node i in the l-th layer of the graph neural network. l Let α be a learnable weight matrix. ij It is the attention weight between nodes i and j; the environmental perception factor f(F) E ) = r w ·wind+r t ·temp; where wind and temp represent the wind speed and temperature at the current node, respectively, r w and r t γ is a learnable weight parameter that controls the overall influence strength of this environmental item.

[0172] Optionally, the graph neural network, which introduces an environment-based weighting term for adjacent edges, incorporates a regularization term during training. The expression for the regular expression is:

[0173]

[0174] In the formula, Let σ represent the expected edge weights under ideal environment-driven conditions, where σ is the Sigmoid function, λ is the regularization coefficient, and wind... ij Let temp be the wind speed between node i and node j. ij Let be the temperature between node i and node j.

[0175] Optionally, the detection module 230 includes:

[0176] The mapping submodule is used to perform linear mapping on multimodal fusion features using a lightweight multilayer perceptron recognition head with a fusion detection model, and output the semantic vector of candidate hidden dangers.

[0177] The third extraction submodule is used to extract spatial location information, text rule information and environmental state information from multimodal fusion features using the multi-head feature decoupling mechanism of the fusion detection model;

[0178] The binding submodule is used to bind spatial location information, text rule information, and environmental state information with the semantic vector of each candidate hazard, and output a structured hazard set.

[0179] Optionally, the semantic vector of the candidate hidden danger is calculated as follows:

[0180]

[0181] In the formula, W h Let b be the weight matrix. h For bias terms, Let F be the feature vector of the i-th hazard candidate region. S′ The probability of the existence of this hidden danger is obtained after applying an activation function to the multimodal fusion feature.

[0182] Optionally, the structured hazard set includes the hazard type, the spatial location of the hazard, its corresponding work permit / safety supervision description, environmental condition information when the hazard occurs, and the detection confidence level.

[0183] Optionally, the inference module 240 includes:

[0184] The projection submodule is used to perform linear projection on each structured hazard in the set of structured hazards, its spatial location, and the corresponding work order / safety supervision description, and to construct a joint code for each structured hazard.

[0185] The output submodule is used to input the knowledge retrieval results obtained from the safety supervision knowledge base and the dynamic environmental information of each structured hazard into the language generation model based on joint coding, and output the risk level of each structured hazard and the business explanation in natural language form.

[0186] The first generation submodule is used to generate reasoning results based on the hazard type, knowledge retrieval results, risk level, and business explanation of each structured hazard.

[0187] Optionally, the generation module 250 includes:

[0188] The second generation submodule is used to generate hazard handling suggestions for each structured hazard based on the risk level and business interpretation of each structured hazard in the set of structured hazards.

[0189] The third generation submodule is used to generate a global operation control strategy based on the risk level of all structured hazards in the structured hazard set.

[0190] The fourth generation submodule is used to generate a global operation strategy for the target scenario based on the hazard handling suggestions for each structured hazard and the global operation control strategy.

[0191] Exemplary electronic devices

[0192] Figure 3 This is the structure of an electronic device provided in an exemplary embodiment of the present invention. For example... Figure 3 As shown, the electronic device 30 includes one or more processors 31 and memory 32.

[0193] The processor 31 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0194] The memory 32 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 31 may execute the program instructions to implement the methods of the software programs of the various embodiments of the present invention described above, and / or other desired functions. In one example, the electronic device may also include an input device 33 and an output device 34, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0195] In addition, the input device 33 may also include, for example, a keyboard, a mouse, etc.

[0196] The output device 34 can output various information to the outside. The output device 34 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0197] Of course, for the sake of simplicity, Figure 3Only some of the components of this electronic device relevant to the present invention are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0198] Exemplary computer program products and computer-readable storage media

[0199] In addition to the methods and apparatus described above, embodiments of the present invention may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of the present invention described in the "Exemplary Methods" section above.

[0200] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0201] Furthermore, embodiments of the present invention may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of the present invention described in the "Exemplary Methods" section above.

[0202] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0203] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the aforementioned specific details.

[0204] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0205] The block diagrams of devices, systems, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, systems, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0206] The methods and systems of the present invention may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of the present invention are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the present invention may also be implemented as a program recorded on a recording medium, the program comprising machine-readable instructions for implementing the methods according to the present invention. Thus, the present invention also covers recording media storing programs for performing the methods according to the present invention.

[0207] It should also be noted that in the systems, apparatus, and methods of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered equivalents of the present invention. The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0208] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for identifying potential hazards in power transmission line corridors, characterized in that, include: Feature extraction is performed on the multimodal data of the target scene collected in the power transmission line to obtain multimodal data features; Cross-modal fusion is performed on the multimodal data features to generate multimodal fused features; The multimodal fusion features are detected by a pre-trained fusion detection model, and a structured set of potential hazards is output. Based on the safety supervision knowledge base, risk level inference is performed on the structured hazard set to generate inference results; Based on the reasoning results, and combined with the actual safety monitoring and operation scheduling scenarios of power transmission lines, a global operation strategy for the target scenario is generated.

2. The method according to claim 1, characterized in that, The multimodal data includes: image / video data, work permits and safety supervision documents, transmission line spatial topology data, and environmental dynamic data. The image / video data includes image / video data of key nodes in the transmission line corridor; the work permits and safety supervision documents include work task information, risk control measures, and safety supervision clauses; the transmission line spatial topology data includes the spatial coordinates and topological relationships of power towers, tower crossing areas, and work areas; and the environmental dynamic data includes real-time wind speed, air temperature, temperature, and air pressure.

3. The method according to claim 2, characterized in that, Feature extraction is performed on the multimodal data of the target scene collected in the power transmission line to obtain multimodal data features, including: A convolutional neural network incorporating a self-attention mechanism is used to extract features from the image / video data to obtain visual features; A pre-trained language model incorporating a self-attention mechanism is used to extract features from the work order and safety supervision text to obtain text features; A graph neural network is used to construct the spatial topology structure of the transmission line based on the spatial topology data, and spatial topology features are determined based on the spatial topology structure. The environmental dynamic data is normalized to obtain environmental dynamic characteristics; The visual features, text features, spatial topology features, and environmental dynamic features are standardized respectively to obtain the multimodal data features.

4. The method according to claim 3, characterized in that, The expression for the standardization process is: In the formula, X represents visual feature F I Text features F T Spatial topological features S G and environmental dynamic characteristics F E μ X σ is the mean of this type of feature; X F is the standard deviation of this type of feature; X These are the standardized features.

5. The method according to claim 3, characterized in that, Cross-modal fusion of the multimodal data features to generate multimodal fused features includes: The visual features and the text features are fused using a dual-stream cross-attention mechanism to obtain a first fused feature; The first fusion feature and the spatial topology feature are fused using a graph neural network to obtain the second fusion feature; The second fusion feature and the environmental dynamic feature are fused by introducing an environment-based weighted graph neural network with adjacent edges to obtain the multimodal fusion feature.

6. The method according to claim 5, characterized in that, The expression for the environment-based weighting term is: In the formula, W represents the state of node i in the l-th layer of the graph neural network. l Let α be a learnable weight matrix. ij It is the attention weight between nodes i and j; Environmental perception factor f(F) E ) = r w ·wind+r t ·temp; where wind and temp represent the wind speed and temperature at the current node, respectively, r w and r t γ is a learnable weight parameter that controls the overall influence strength of this environmental item.

7. The method according to claim 6, characterized in that, The adjacent edges are introduced into the graph neural network with environment-based weighting, and a regularization term is introduced during training. The expression for the regular expression is: In the formula, Let σ represent the expected edge weights under ideal environment-driven conditions, where σ is the Sigmoid function, λ is the regularization coefficient, and wind... ij Let temp be the wind speed between node i and node j. ij Let be the temperature between node i and node j.

8. The method according to claim 1, characterized in that, The multimodal fusion features are used to detect potential hazards using a pre-trained fusion detection model, outputting a structured hazard set, including: The lightweight multilayer perceptron recognition head of the fusion detection model is used to linearly map the multimodal fusion features and output the semantic vector of the candidate hidden dangers. The multi-head feature decoupling mechanism of the fusion detection model is used to extract spatial location information, text rule information, and environmental state information from the multimodal fusion features; The spatial location information, the text rule information, and the environmental state information are bound to the semantic vector of each candidate hazard to output a structured hazard set.

9. The method according to claim 8, characterized in that, The semantic vector of the candidate potential hazard is calculated as follows: In the formula, W h Let b be the weight matrix. h For bias terms, Let F be the feature vector of the i-th hazard candidate region. S′ The probability of the existence of this hidden danger is obtained after applying an activation function to the multimodal fusion feature.

10. The method according to claim 1, characterized in that, The structured hazard set includes hazard type, hazard spatial location, corresponding work order / safety supervision description, environmental condition information when the hazard occurs, and detection confidence level.

11. The method according to claim 10, characterized in that, Based on the safety supervision knowledge base, risk level inference is performed on the structured hazard set to generate inference results, including: Linear projection is performed on each structured hazard in the structured hazard set, its spatial location, and the corresponding work order / safety supervision description to construct a joint code for each structured hazard; Based on the joint coding, the knowledge retrieval results obtained from the safety supervision knowledge base and the dynamic environmental information of each structured hazard are input into the language generation model, and the risk level of each structured hazard and the business explanation in natural language form are output. The reasoning result is generated based on the hazard type of each structured hazard, the knowledge retrieval result, the risk level of each structured hazard, and the business explanation.

12. The method according to claim 11, characterized in that, Based on the reasoning results, and combined with the actual safety monitoring and operation scheduling scenarios of power transmission lines, a global operation strategy for the target scenario is generated, including: Based on the risk level of each structured hazard in the structured hazard set and the business interpretation, generate hazard handling suggestions for each structured hazard; A global operation control strategy is generated based on the risk level of all structured hazards in the structured hazard set. Based on the hazard handling recommendations for each structured hazard and the global operation control strategy, a global operation strategy for the target scenario is generated.

13. A device for identifying potential hazards in power transmission line corridors, characterized in that, include: The extraction module is used to extract features from the multimodal data of the target scene in the collected power transmission line to obtain multimodal data features; The fusion module is used to perform cross-modal fusion of the multimodal data features to generate multimodal fused features; The detection module is used to detect potential hazards in the multimodal fusion features using a pre-trained fusion detection model and output a structured set of potential hazards. The reasoning module is used to perform risk level reasoning on the structured hazard set based on the safety supervision knowledge base and generate reasoning results; The generation module is used to generate a global operation strategy for the target scenario based on the reasoning results and in combination with the actual safety monitoring and operation scheduling scenarios of power transmission lines.

14. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for performing the method described in any one of claims 1-12.

15. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-12.

Images