A substation adaptive inspection method based on reinforcement learning and unmanned aerial vehicle

Abstract

The application relates to a power substation adaptive inspection method based on reinforcement learning and a UAV, and belongs to the technical field of power inspection. In view of the problems of an inspection dead angle, a safety hidden danger, low efficiency of manual calibration of a flight route and weak scene knowledge reuse in the prior art, first, a UAV radar / SLAM is used to collect power station multi-source data, heterogeneous point clouds and visible light data coordinates are unified, and a high-precision 3D power station model is constructed based on super-resolution reconstruction; a double-flow cross-attention module PROMPT-CAFFM is used to fuse radar and visible light coding features, and a device voxel semantic positioning model is generated; finally, an initial flight route is generated based on the model to analyze a task prompt word, and a reinforcement learning model is combined to optimize a waypoint trajectory in real time; the reinforcement learning model constructs a multi-dimensional reward function with a device safety distance, imaging quality and flight efficiency, dynamically corrects a flight trajectory, and specifically comprises the following steps: 1) a flight route file containing a waypoint coordinate and a holder angle is output according to semantic analysis of a prompt word; and 2) a flight route is dynamically adjusted based on real-time sensing data.

Description

Technical Field

[0001] This invention relates to the fields of substation safety supervision and industrial image feature mining, and provides an adaptive substation inspection method based on reinforcement learning and UAVs. Background Technology

[0002] In the field of substation safety supervision, traditional safety supervision relies on daily patrols by work teams and fixed cameras. Limited by the equipment density of the substation, patrols have many blind spots, and the patrol plans are not flexible enough, posing significant safety hazards. To improve patrol efficiency and operational safety, related professional research is increasing, with drone-based solutions becoming increasingly popular.

[0003] Traditional flight methods primarily rely on skilled pilots for manual training. However, the high cost of human training, the inability to verify training routes, and unclear safety responsibilities hinder widespread adoption. In adaptive flight for drones, solutions led by DJI utilize advanced radars like the Zenmuse L2 to pre-scan airfields and then manually calibrate flight paths, solving the problem of unverifiable routes and providing a basic safety verification environment. However, manual route calibration suffers from slow speed, uncontrollable accuracy, and difficulty in handling unexpected events. This is particularly true when construction within airfields obstructs flight paths, making it difficult for drones to adapt. While obstacle avoidance is normally sufficient to prevent collisions, the dense equipment at airfields necessitates disabling obstacle avoidance in high-density, strong magnetic environments, hindering normal patrols and requiring human supervision.

[0004] Regarding flight path generation, initially, professionals taught the equipment to set shooting points, shooting distances, and gimbal parameters. This evolved into a system where specialized software calculated and marked shooting points based on radar signals, algorithms automatically planned flight paths, and professionals adjusted waypoint data. The new approach ensured accuracy and controllable safety, but the long development cycle and significant manpower investment hindered its widespread adoption. Furthermore, the pre-defined nature of waypoint configuration made the approach inflexible and unable to quickly execute temporary flight paths.

[0005] In terms of technology solution promotion, existing technologies only focus on improving the performance of individual substations or equipment, neglecting common research on substations. This means that we need to implement the solution throughout the entire process for different substations, which is resource-intensive, and even slight changes in the environment can have a significant impact on the solution.

[0006] In summary, this patent addresses the challenges of verifying inspection routes for substation maintenance drones, the significant risks associated with flight route execution, the poor adaptability of route configuration methods, the inability to quickly configure routes for temporary tasks, and the lack of shared features and scenario knowledge in the technical solutions' algorithms. It proposes a novel adaptive substation inspection technology that combines reinforcement learning and drone equipment, aiming to solve the problems of route verification, online route correction, and flexible route configuration. Summary of the Invention

[0007] The purpose of this invention is to solve the problems of insufficient inspection accuracy, delayed emergency response and high resource consumption caused by high safety risks in densely populated equipment areas, low efficiency and poor flexibility of traditional manual route marking, and weak ability to integrate multi-source heterogeneous data and reuse scenario knowledge in substation inspections.

[0008] To achieve the above objectives, the present invention employs the following technical means:

[0009] This invention provides an adaptive substation inspection method based on reinforcement learning and unmanned aerial vehicles (UAVs), comprising:

[0010] Step 1: Collect multi-source scanning data of the site using radar and SLAM equipment carried by the UAV, perform coordinate system unification processing on the heterogeneous radar point cloud and visible light data, and build a high-precision 3D site model based on the super-resolution reconstruction model.

[0011] Step 2: The 3D site model is encoded with radar signal and visible light signal respectively. Multimodal features are fused through the PROMPT-CAFFM dual-stream cross-attention module with cue words to generate a device voxel semantic localization model.

[0012] Step 3: Based on the voxel semantic localization model, the task prompt words are parsed to generate an initial flight path, and the waypoint flight trajectory is optimized in real time by combining the reinforcement learning model; the reinforcement learning model constructs a multi-dimensional reward function based on the device safety distance, imaging quality and flight efficiency, and controls the local flight path between waypoints through the action space;

[0013] Flight route planning: Based on the task prompt, the semantic positioning information of the device is parsed, and the flight route file containing waypoint coordinates and gimbal parameters is output through a reinforcement learning model;

[0014] Flight route execution: Based on the aforementioned flight route document, and combined with real-time radar and visible light data, the flight trajectory is dynamically corrected through a reinforcement learning model.

[0015] In the above scheme, step 1 includes the following steps:

[0016] Step 1.1 Use a drone to scan the site from multiple angles and distances along a safe flight path to obtain drone radar scan data and visible light scan data;

[0017] Step 1.2 Use SLAM equipment to perform high-precision scanning of the densely equipped area at the bottom of the site to obtain SLAM radar scanning data and visible light scanning data;

[0018] Step 1.3: Fuse UAV radar scan data and SLAM radar data, perform synchronous calculation of fused features, and obtain a high-precision 3D model of the site.

[0019] In the above scheme, step 1.3 includes the following steps:

[0020] Step 1.3.1: Using joint calibration technology, obtain a coordinate system representation of the heterogeneous multimodal data from the UAV radar and visible light scanning data, SLAM radar data, and visible light scanning data;

[0021] For visible light data, a deep learning model is used to reconstruct dense depth information from two-dimensional images, and unified station depth information is obtained based on the station's spatial coordinate system. The construction method is as follows:

[0022] Re1(X)=I A (X+O)

[0023] Where X = (x1, x2, x3) represents the relative position coordinates, x1, x2, x3 correspond to longitude, latitude and altitude dimensions respectively, O is the coordinates of the base point of the station, A represents the radar point cloud set, that is, the union of the UAV radar point cloud and the SLAM point cloud, and Re1 is the indicator function of whether the relative position of the midpoint X of the station with respect to the base point O belongs to the point cloud set A.

[0024] For visible light data, a deep learning model is used to reconstruct dense depth information from the visible light, and its coordinate representation is consistent with that of radar signals.

[0025]

[0026] D is a deep learning model that performs dense estimation of depth information. The k-th partial image sequence, k∈1,...,n;

[0027] Dense depth reconstruction of the visible light sequence is performed using a deep learning model D to generate a set of visible light point clouds B.

[0028] In the visible light point cloud set B, its depth information is represented in the station's spatial coordinate system as follows:

[0029] Re2(X)=I B (X+O)

[0030] Re2 is the indicator function for whether the relative position of the midpoint X of the station with respect to the base point O belongs to the visible light point cloud set B;

[0031] Step 1.3.2 Based on Re1(X) and Re2(X), point cloud registration and feature reconstruction are performed using the super-resolution reconstruction model to generate a 3D site model with fused pose signals.

[0032] In the above scheme, step 2 includes the following steps:

[0033] Step 2.1 For the high-precision 3D site model, perform radar signal encoding and visible light signal encoding using a multi-head self-attention mechanism (MHSA):

[0034] Step 2.2 Construct the PROMPT-CAFFM dual-stream cross-attention module with cue words;

[0035] Step 2.3 Generate a voxel semantic localization model based on the output features of PROMPT-CAFFM and the decoder:

[0036] Using a Transformer decoder, with CAFFM output features as the key / value pair, and fusing task prompts and CAFFM features as the query, the output device voxel semantic localization model is generated.

[0037] In the above scheme, step 2.1 includes the following steps:

[0038] Step 2.1.1 Extract features from the visible light visual representation in the 3D site model using a multi-head self-attention mechanism to obtain visible light self-attention encoding;

[0039] Step 2.1.2 For the radar positioning information in the 3D site model, perform multi-head self-attention encoding based on point cloud attributes: Define the radar point attribute set:

[0040] X=(RGB,XYZ,Intensity,NOR,…)

[0041] RGB represents the color space value of this point, XYZ represents the corrected spatial coordinates, Intensity represents the radar signal strength value, and NOR represents the number of echoes.

[0042] The associated attributes are assigned to different attention heads for encoding, and processed through n1 layers of stacked MHSA modules:

[0043]

[0044] W(i) is the spatial window identifier, indicating that only points located within window W(i) are processed using MHSA. That is, different windows undergo different MHSA processing. Let d be the query, key, and value vector of the h-th attention head. kW is the scaling factor for the key vector dimension. OR The weights are linearly transformed for multi-head outputs.

[0045] In the above scheme, step 2.2 includes the following steps:

[0046] Step 2.2.1 First Cross-Attention Flow:

[0047] The radar coding features are linearly transformed into key K. r Value V r The visible light encoded features are linearly transformed into query Q. v The text features Q are obtained by embedding the task prompt word. p Query vector fusion: Q vp =Q v +Q p ;

[0048] Calculate cross attention:

[0049]

[0050] Concat(·) represents the concatenation operation. It is the query vector Query of the h-th attention head. This represents the result of a linear transformation of radar features as keys. This represents the transpose of the key vector, d k It is the dimension of the key vector. The result of a linear transformation of the value vector Value, W RO Linear transformation weights representing radar characteristics;

[0051] Step 2.2.2 Second Cross-Attention Flow:

[0052] Linear transformation of visible light encoded features into key K v Value V v The radar-coded features are linearly transformed into query Q. r Query vector fusion: Q rp =Q r +Q p Calculate cross-attention:

[0053]

[0054] Among them W VO The weights are the linear transformation weights for visible light characteristics. The query vector of the h-th attention head is generated by the enhanced query encoding obtained by fusing radar features and prompt words. The result is the linear transformation of visible light characteristics as bonds. The result of a linear transformation of a value vector is given by the vector, where W is a value representing the visible light characteristics. (i) It is a space window identifier, W VO The linear transformation weights for visible light features;

[0055] Step 2.2.3 Stack steps 2.2.1 and 2.2.2 to form a basic module, and repeat this module to build the PROMPT-CAFFM structure;

[0056] In the above scheme, step 3 includes the following steps:

[0057] Step 3.1 Generate adaptive flight path based on device voxel localization model: Obtain the spatial location of shooting point by parsing the semantic information of the device through task prompt words;

[0058] Step 3.1.1 Parse the device semantic information using the task prompt word to obtain the spatial location of the shooting point;

[0059] Step 3.1.2 Calculate the safe flight area and determine the optimal waypoint in the high-precision 3D airfield model;

[0060] Step 3.1.3 Generate gimbal and camera shooting parameters based on UAV parameters and spatial scale information;

[0061] Step 3.1.4 Determine the waypoint flight order using a route planning algorithm and format the record as a route file:

[0062]

[0063] Where: L si,B To supplement waypoints i The distance to the target waypoint B, ax si ,ax B waypoint s i And the coordinates of B on the x, y, z axes;

[0064] Step 3.2 Construct a reinforcement learning model to optimize flight routes:

[0065] Step 3.2.1 Define the state space S, action space A1 (flight direction) and A2 (flight distance), and set the reward function:

[0066] To waypoint s i and s i+1 By scaling the flight distance, waypoint approach bonuses are obtained:

[0067]

[0068] W a Approach reward scaling factor;

[0069] To waypoint s i and s i+1 For each flight segment, the warning reward is obtained by calculating the shortest distance from the device voxel to the segment.

[0070]

[0071] Among them l max This is the maximum relative distance cutoff value, Dist is the function for finding the shortest distance, Clip is the distance cutoff function, and W... w It is the warning reward scaling factor, l safe This refers to the safe operating distance for drones and equipment. Indicate waypoint s i To waypoint S i+1 The trajectory of the line segment, Point device Represents the set of points of obstacles and electrical equipment in 3D coordinate space;

[0072] To waypoint s i and s i+1 For each state, calculate the target imaging distortion difference to obtain the imaging similarity reward:

[0073]

[0074] waypoint s i Target imaging distortion features, W s T is the scaling factor for similarity rewards. B For the imaging features or distortion features of the target waypoint B, Simi represents the imaging structure similarity function, which measures the degree of image feature matching.

[0075] Step 3.2.2 Constructing the reinforcement learning objective function:

[0076]

[0077] Where π(a|s) is the probability distribution of choosing action a in state s, and ρ(s) is the state distribution function, defined as:

[0078]

[0079] Where γ is the discount factor, γ t This represents the present value of the reward at step t, P(s). t =s|π) represents the probability that the system is in state s at time step t under the influence of policy π.

[0080] The action value function is defined as:

[0081]

[0082] Where r(s) t ,a t The immediate reward after performing the action at time t is:

[0083]

[0084] Reward i (a t |s t ) represents one of the reward functions explicitly given above, debuff(a t |s t ) indicates that in state s t Take multi-dimensional actions a t In addition to the three reward functions mentioned above, some negative reward settings also need to be set.

[0085] Step 3.2.3 Real-time flight control: The reinforcement learning model dynamically adjusts the flight trajectory based on real-time environmental feedback.

[0086] Because the present invention employs the above-mentioned technical means, it has the following beneficial effects:

[0087] 1. This invention solves the problem of insufficient model accuracy caused by a single radar data source by integrating multi-source radar data and calculation technology (refer to step 1.3), and achieves the effect of constructing a high-precision 3D site model.

[0088] Technical problem analysis: Traditional inspection relies on single radar data (such as only UAV radar), which is prone to losing details due to limited field of view or environmental interference (such as the sparsity of point clouds in densely populated areas of underlying equipment), resulting in low resolution of 3D models, which affects subsequent route planning and flight safety.

[0089] Step 1.3 innovatively integrates UAV radar and SLAM radar data, utilizing joint calibration techniques to unify heterogeneous coordinate systems (e.g., aligning radar and visible light point clouds using indicator functions Re1(X) and Re2(X)). This achieves complementarity among multi-source data: UAV radar covers mid-to-high-level equipment, while SLAM radar supplements lower-level details. Point cloud registration is further optimized using super-resolution reconstruction models (e.g., PointTransformer), eliminating blind spots from single data sources and generating a richly detailed 3D model. This model provides a reliable spatial reference for adaptive inspection, avoiding the need for manual repairs and improving overall inspection efficiency.

[0090] 2. This invention solves the problem of semantic alignment difficulties of heterogeneous data (radar and visible light) by using the multimodal feature fusion module CAFFM (refer to step 2.2), and achieves the effect of accurate device voxel semantic localization.

[0091] Technical problem analysis: Radar data (including location attributes) and visible light data (including visual semantics) are difficult to fuse directly due to modal differences, resulting in inaccurate equipment positioning (such as incorrect equipment type identification or position offset), which affects the accuracy of route generation.

[0092] Step 2.2 Design a dual cross-attention mechanism (CAFFM module), using radar encoding as the key / value and visible light encoding as the query (or vice versa), and introduce a task prompt to modulate the query vector (such as the Q in step 2.2.1). vρ (Generated). This module computes multi-head cross-attention (e.g., MHCA). out The formula enables cross-modal interaction at the feature level: radar data provides precise positional constraints, while visible light data supplements device semantics (e.g., learning device name associations through pre-trained models). Stacking multiple layers of CAFFM (step 2.2.3) enhances feature extraction, ultimately decoding to generate a voxel semantic localization model. This ensures high-precision alignment of device location and type, laying the foundation for adaptive route planning.

[0093] 3. This invention solves the problems of low efficiency and poor flexibility of manual route calibration by using a route generation model based on reinforcement learning (refer to step 3.1), and achieves the effect of quickly and adaptively generating inspection routes.

[0094] Technical problem analysis: Traditional routes rely on manual teaching or pre-installed software, which cannot respond to temporary tasks (such as emergency inspections), and the generation process is time-consuming (requiring professionals to adjust waypoints), resulting in delays in inspection response.

[0095] Step 3.1 uses a voxel semantic localization model as the core, combined with task prompts (such as outputting the shooting point based on the prompt words in Step 3.1) to parse the device location, and optimizes the waypoint sequence through a path planning algorithm. The reinforcement learning model (Step 3.2) integrates a reward and penalty mechanism (such as waypoint approach reward and warning penalty). warn The system dynamically calculates optimal waypoints, gimbal parameters, and flight sequence within a 3D airfield model. This solution achieves "description-as-generation": the user inputs task prompts, and the system automatically outputs a complete flight path file without manual intervention. This not only shortens the flight path configuration cycle but also supports rapid deployment of ad-hoc tasks, improving inspection flexibility.

[0096] 4. Through its overall technical architecture (referencing steps 1-3), this invention solves the problem of knowledge sharing across different substation scenarios, achieving a highly scalable solution.

[0097] Technical problem analysis: The existing solution requires a customized whole process for each station (such as data collection to route generation), which leads to repeated consumption of resources. Furthermore, subtle changes in the environment (such as equipment displacement) require remodeling, which limits large-scale application.

[0098] Step 1, high-precision 3D modeling (multi-source data fusion), and Step 2, voxel semantic localization (CAFFM module), generate a general site representation. Step 3, a reinforcement learning model (such as a flight route model), takes local states as input, does not rely on global site data, and can be pre-trained using historical data and transferred to new scenarios (Step 3.2.1). This forms a "model once, reuse many times" framework: new sites only need to perform the scan in Step 1, and subsequently, the shared model can be directly called to generate flight routes. This reduces deployment costs and improves the robustness of the solution to site differences. Attached Figure Description

[0099] Figure 1 This is a schematic diagram of the adaptive inspection process for unmanned aerial vehicles (UAVs).

[0100] Figure 2 This is a schematic diagram of the PROMPT-CAFFM structure. Detailed Implementation

[0101] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.

[0102] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.

[0103] This invention provides an adaptive substation inspection method based on reinforcement learning and unmanned aerial vehicles (UAVs), comprising the following steps:

[0104] Step 1: Use drones and high-precision SLAM equipment to scan and solve the site to obtain a high-resolution 3D site model;

[0105] Step 2: Perform multi-source signal fusion modeling on the high-precision 3D site model to obtain the equipment voxel semantic localization model;

[0106] Step 3: Combine the device voxel semantic localization model with a reinforcement learning model to obtain the adaptive route planning algorithm and the route execution model:

[0107] Flight route planning: Based on the equipment voxel localization model, the system is provided with equipment identification and orientation information. According to the customer's task requirements, a reinforcement learning model is used to output the flight route.

[0108] Flight route execution: The reinforcement learning model uses flight route files (which record 3D information, waypoints, camera gimbal parameters, etc.) as a reference and real-time radar and visible light as correction criteria to achieve automated flight route execution.

[0109] In the above scheme, step 1 specifically includes:

[0110] Step 1.1: Use a drone to scan the airfield at different angles and distances along a safe flight path to obtain drone radar and visible light scan data;

[0111] Step 1.2: Perform high-precision SLAM scanning on the densely equipped area of ​​the site object to obtain SLAM radar and visible light scanning data;

[0112] Step 1.3: Fuse UAV radar scan data and SLAM radar data, and simultaneously calculate the fused features to obtain a high-precision 3D model of the airfield. (Innovation: Using different forms of radar data for calculation solves the problem of low accuracy in single-radar calculations.)

[0113] Step 1.3.1: Using joint calibration techniques, obtain a coordinate system representation of the heterogeneous multimodal data from the UAV radar and visible light scan data, SLAM radar data, and visible light scan data; (coordinate system alignment is performed using heterogeneous data).

[0114] For the aforementioned UAV radar, SLAM radar, and visible light imaging of the UAV and SLAM, based on base station signals, a unified representation is performed in a relative spatial coordinate system. The origin of this spatial coordinate system is a stable point (not prone to displacement) selected at the site. Visible light is two-dimensional data without depth information; therefore, a 3D reconstruction model is used to obtain a simulated depth signal, which is then used to obtain unified site depth information based on the site's spatial coordinate system. The construction method is as follows:

[0115] Re1(X)=I A (X+O)

[0116] Where X represents a point in the spatial coordinate system (relative position coordinates), with three dimensions x1, x2, and x3 representing longitude, latitude, and altitude, respectively. 0 is the coordinate of the base station. The right side of the formula indicates whether the current coordinate X belongs to set A in the latitude and longitude coordinate system. A represents the radar point cloud set, i.e., the union of the UAV radar point cloud and the SLAM point cloud. The Re1 function is the indicator function for whether the relative position of the midpoint X of the base station relative to the base point 0 belongs to the point cloud set A. For visible light sequences, we use deep learning models, such as Dense Prediction Transformers (DPT), to reconstruct dense depth information from the visible light. Its coordinate representation uses the same relative position coordinates with respect to the base point 0 as the radar signal.

[0117]

[0118] D is a deep learning model that performs dense estimation of depth information. After reconstructing the depth information of visible light, we can construct the point cloud of the station equipment in the relative coordinate system. This represents the first image in the local shooting sequence. Using a deep learning model, we can obtain a visualized image simulation device point cloud in a relative positional spatial coordinate system; this set is named B. Then, in the visible light point cloud B, its depth information is represented in the site's spatial coordinate system as follows:

[0119] Re2(X)=I B (X+O)

[0120] Based on the aforementioned radar data and visible light images, we obtain a unified multimodal data representation using the Re1 and Re2 indicative functions.

[0121] Step 1.3.2: Represent the coordinate system and use a super-resolution reconstruction model to obtain a 3D model of the site that fuses pose signals. Using multi-source radar and multi-source visible light signals, a high-precision 3D model is constructed using a deep learning model based on super-resolution reconstruction.

[0122] For point cloud Re1 and Re2 data, a point cloud registration and feature reconstruction model like Point Transformer is used to obtain more accurate point representations. (This involves fine-tuning an existing registration and reconstruction model for a specific vertical domain, and then fusing it with specific point cloud data.)

[0123] In the above scheme, step 2 specifically includes:

[0124] Step 2.1: For the high-precision 3D model representation of the site, different transformer visual encoding models are used to obtain independent visual encoding information: radar encoding information and visible light encoding information. (Innovation: Multi-source data encoding provides a basis for fusing positioning, equipment attitude, and equipment type, facilitating subsequent feature mining);

[0125] Step 2.1.1: For the visible light visual representation in the high-precision 3D model of the site, use a multi-head self-attention mechanism to learn its inherent features and obtain the self-attention encoding of visible light; (normal operation)

[0126] Step 2.1.2: Represent the radar positioning information in the high-precision 3D model of the station, use a multi-head attention mechanism to learn the position information contained therein, and obtain the self-attention position code of the radar signal;

[0127] For the attributes RGB, corrected spatial coordinates, intensity values, echo counts, etc. of points in the Re1 point cloud, autocorrelation coding feature extraction is performed on the information using a multi-head self-attention (MHSA) encoder.

[0128] For the point attribute representation in Re1, the MHSA technique is used to obtain the autocorrelation coding features of the raw data. The input is X, which has m attributes, including RGB values, spatial coordinates, intensity values, echo counts, etc. When the first MHSA processes the input, it will be divided into h heads. Each head has a different feature dimension. For example, the RGB head has three feature dimensions, while the echo count has only one feature dimension. In order to aggregate the sparse attributes, we group the attributes with high correlation into one head in the multi-head design to increase the size of its feature dimension.

[0129] X = (Month GB, XYZ, Intensity, NOR, ...)

[0130] Here, RGB represents the color space value of the point, XYZ represents the corrected spatial coordinates, Intensity represents the radar signal strength value, and NOR represents the echo count. In the point representation above, we divide RGB into one header, XYZ into another, and Intensity and NOR into yet another. In reality, there are many more radar attributes. We assign highly correlated attributes to each header in this manner, and after MHSA processing, we obtain the feature associations within each attribute class.

[0131]

[0132] Here, W(i) is the spatial window identifier, indicating that only points located within window W(i) are processed using MHSA, meaning different windows undergo different MHSA processing.OR This refers to the weights used in the autoencoder stage for linear processing of multi-head results. The radar autoencoder stage stacks n1 MHSA layers for feature extraction. After the first MHSA layer, the first head outputs hi feature dimensions. For the first n1 / 3 layers, the number of output channels from each head in the previous layer is used as the number of groups in the next layer. That is, in these layers, modeling is only performed within these attributes. Subsequent layers use a standard transformer module design, with each head using the same number of channels to enhance cross-attribute modeling. Although the channel allocation method for each head differs in the pipeline, the calculation formula is consistent with the MHSAout calculation method described above.

[0133] Step 2.2: For the visible light self-attention encoding and self-attention position encoding, a custom cross-attention module is used to obtain the feature fusion module (CAFFM); (Innovation: A CAFFM structure was developed to guide the model to learn two heterogeneous data, fuse multimodal data features, and provide data dependencies for decoding and mining voxel category and position information)

[0134] Step 2.2.1: Use radar-coded features as KEY and VALUE, and visible light-coded features as QUERY to generate cross-attention features and update radar features;

[0135] Based on the aforementioned radar MHSAout data, KEY(K) is obtained through linear transformation. r ), VALUE (V r );

[0136] Based on the aforementioned visible light MHSAout data, QUERY(Q) is obtained through linear transformation. v );

[0137] For the task description prompt, a pre-trained word embedding model is used to obtain the text features Q. p ;

[0138] Regarding Q v and Q p By using addition to fuse the two encodings, the enhanced query encoding Q is obtained. vp ;

[0139] For K r V r Q v Substituting these values ​​into the MHSA calculation formula yields the multi-head cross-attention calculation model, whose calculation formula can be simplified as follows:

[0140]

[0141] Among them, W R0It is the linear transformation weight of radar features in the cross-attention phase with respect to the multi-head output results.

[0142] Step 2.2.2: Use visible light coded features as KEY and VALUE, and radar coded features as...

[0143] QUERY generates cross-attention features and updates visible light features;

[0144] For the above visible light MHSAout data, KEY(K) was obtained through linear transformation. v ), VALUE (V v );

[0145] Based on the aforementioned radar MHSAout data, QUERY(Q) is obtained through linear transformation. r );

[0146] Regarding Q r and Q p By using addition to fuse the two encodings, an enhanced query encoding is obtained.

[0147] For K r V r Q v Substituting these values ​​into the MHSA calculation formula yields the multi-head cross-attention calculation model, whose calculation formula can be simplified as follows:

[0148]

[0149] Step 2.2.3: Stack the above two steps to form a basic module, and then repeat this basic module to form a CAFFM module;

[0150] Step 2.3: Construct a decoder for multi-source fusion features to generate a voxel semantic localization module. The decoder uses a transformer structure as the basic module. The KEY and VALUE are obtained by linear transformation of the features generated by the CAFFM module. The features extracted by prompt and CAFFM are fused as the QUERY. The module finally outputs a non-linear perceptual voxel localization model.

[0151] Similar to the encoder stage, the prompt is a dynamic task description text that needs to be expanded using a large language model and encoded using word embeddings of the language model to obtain the prompt encoding.

[0152] In the above scheme, step 3 specifically includes:

[0153] Step 3.1: For the voxel localization model, based on the task prompt, obtain the adaptively generated flight path; (using a multimodal visual large model to generate 3D flight path and waypoint information)

[0154] For the voxel localization model, descriptive terms for the desired flight path are given, and after word embedding processing, prompt codes are obtained. Radar and visible light signals with unified representations are provided, and the voxel localization model provides their spatial locations and key attribute information. Using these devices and their attribute information, waypoint and flight path information is constructed.

[0155] For the task description, the voxel localization model is used to obtain the position (shooting point) of the target to be photographed in the relative spatial coordinate system;

[0156] For the aforementioned shooting points, the safe flight area is calculated in a high-precision 3D field station to obtain the optimal waypoint;

[0157] For the array of waypoints and shooting points, the optimal gimbal and camera shooting parameters are calculated using UAV parameters and spatial scale information (shooting-related parameters need to be corrected in real time in actual applications).

[0158] For the waypoints, the optimal waypoint flight order is obtained using a path planning algorithm;

[0159] The flight sequence, gimbal and camera shooting parameters for the waypoints are recorded in an orderly and formatted manner to obtain the flight route file.

[0160] Step 3.2: For the high-precision 3D site model and voxel localization model, configure the reinforcement learning model reward and penalty terms to obtain a reinforcement learning model that can be used for route generation and flight execution; (A waypoint-based reinforcement learning scheme is proposed to solve the model transfer and adaptation problem for different sites)

[0161] Step 3.2.1: Using historical 3D information and device voxel sets, set up flight motion direction space and flight motion distance space. Set up reward and penalty functions in terms of device-to-line distance, waiting-to-fly distance, and waypoint shooting similarity. Weight these reward scores and obtain a reinforcement learning model by setting a score maximization function. This model is used for the calculation model of waypoint-to-line flight.

[0162] For the flight direction, the motion direction space A1 is obtained by using the six directions of "up, down, forward, backward, left and right" set in the relative position space as the motion direction movement.

[0163] For the flight distance, the maximum flight distance is the flight distance per unit time at the maximum safe flight speed, and the unit flight distance is the flight distance per unit time at the minimum flight speed. The maximum flight distance is discretized from the unit flight distance to obtain the flight distance space A2.

[0164] For the route from waypoint A to waypoint B, a series of supplementary waypoints are planned to obtain the execution route trajectory AirLine.

[0165] For the supplementary waypoint, the current supplementary waypoint record is s. i The next supplementary waypoint record is s i+1 S is the state space of all possible supplementary waypoints.

[0166] For supplementary waypoints s i Given waypoint B, calculate the minimum flight distance in the state space to obtain the distance to be flown L. si，B .

[0167]

[0168] To waypoint s i and s i+1 By scaling the flight distance, a waypoint approach reward approach is obtained.

[0169]

[0170] Where is W a Scaling factor for supplementary waypoint approach.

[0171] To waypoint s i and s i+1 For each flight segment, a warning reward is given by calculating the shortest distance from the device voxel to the segment.

[0172]

[0173] Among them l max This is the maximum relative distance cutoff value, Dist is the function for finding the shortest distance, Clip is the distance cutoff function, and W... w It is the warning reward scaling factor, l safe It refers to the safe operating distance for drones and equipment.

[0174] To waypoint s i and s i+1 For each state, the difference in target imaging distortion is calculated to obtain the imaging similarity reward.

[0175]

[0176] The objective equation of the reinforcement learning model is:

[0177]

[0178] Where π(a|s) is the probability distribution of choosing action a in state s, and ρ(s) is the state distribution function:

[0179]

[0180] The probability function P represents the state s at time t under the distribution π. t The probability, γ is the discount factor, which balances the weights of the states at each time step. Q π (s, a) is the action value function, representing the expected cumulative reward following policy π after performing action a in state s. The cumulative reward function is obtained by recursively calculating the expectation of the three reward functions mentioned above based on the state sequence (time points). The relationship between the expected cumulative reward function and the three reward functions of the aforementioned state transitions can be summarized by the following formula:

[0181]

[0182] Where r(s) t a t The immediate reward after performing the action at time t is:

[0183]

[0184] The debuff refers to some penalties imposed during the state transition, including penalties for moving away from the target point and penalties for wandering in the area.

[0185] Step 3.2.2: Issue a specific flight mission. The UAV scans the airfield in real time and feeds back the current environment to the reinforcement learning model. Based on the current environment voxel information, flight path information, and sensor environment information, the reinforcement learning model provides guidance for the next action.

Claims

1. A substation adaptive inspection method based on reinforcement learning and unmanned aerial vehicles (UAVs), characterized in that, include: Step 1: Collect multi-source scanning data of the site using radar and SLAM equipment carried by the UAV, perform coordinate system unification processing on the heterogeneous radar point cloud and visible light data, and build a high-precision 3D site model based on the super-resolution reconstruction model. Step 2: The 3D site model is encoded with radar signal and visible light signal respectively. Multimodal features are fused through the PROMPT-CAFFM dual-stream cross-attention module with cue words to generate a device voxel semantic localization model. Step 3: Based on the voxel semantic localization model, the task prompts are parsed to generate an initial flight path, and the waypoint flight trajectory is optimized in real time using a reinforcement learning model; the reinforcement learning model constructs a multi-dimensional reward function based on equipment safety distance, imaging quality, and flight efficiency, and controls local flight paths between waypoints through the action space; Flight route planning: Based on the task prompt, the semantic positioning information of the device is parsed, and the flight route file containing waypoint coordinates and gimbal parameters is output through a reinforcement learning model; Flight route execution: Based on the aforementioned flight route document, and combined with real-time radar and visible light data, the flight trajectory is dynamically corrected through a reinforcement learning model.

2. The method according to claim 1, characterized in that, Step 1 includes the following steps: Step 1.1 Use a drone to scan the site from multiple angles and distances along a safe flight path to obtain drone radar scan data and visible light scan data; Step 1.2 Use SLAM equipment to perform high-precision scanning of the densely equipped area at the bottom of the site to obtain SLAM radar scanning data and visible light scanning data; Step 1.3 Integrate UAV radar scan data and SLAM radar data, perform synchronous calculation of fused features, and obtain a high-precision 3D model of the site.

3. The method according to claim 2, characterized in that, Step 1.3 includes the following steps: Step 1.3.1: Using joint calibration technology, obtain a coordinate system representation of the heterogeneous multimodal data from the UAV radar and visible light scanning data, SLAM radar data, and visible light scanning data; For radar data, its coordinate system representation is constructed as follows: in Represents relative position coordinates, These correspond to longitude, latitude, and altitude, respectively. These are the coordinates of the station's base point. This represents the set of radar point clouds, specifically the union of the UAV radar point cloud and the SLAM point cloud. Midpoint of the station Relative to the base point Does the relative position belong to the point cloud set? Indicator functions; For visible light data, a deep learning model is used to reconstruct dense depth information from the visible light, and its coordinate representation is consistent with that of radar signals. It is a deep learning model that performs dense estimation of depth information. : No. Zhang's partial image sequence, ; Dense depth reconstruction of the visible light sequence is performed using a deep learning model D to generate a set of visible light point clouds B. In the visible light point cloud set B, its depth information is represented in the station's spatial coordinate system as follows: Midpoint of the station Relative to the base point Whether the relative position belongs to the indicator function of the visible light point cloud set B; Step 1.3.2 Based on the above and A super-resolution reconstruction model is used for point cloud registration and feature reconstruction to generate a 3D station model with fused pose signals.

4. The method according to claim 2, characterized in that, Step 2 includes the following steps: Step 2.1 For the high-precision 3D site model, use the multi-head self-attention mechanism (MHSA) to encode radar signals and visible light signals respectively; Step 2.2 Construct the PROMPT-CAFFM dual-stream cross-attention module with cue words; Step 2.3 Generate a voxel semantic localization model based on PROMPT-CAFFM output features and decoder: Using a Transformer decoder, with CAFFM output features as the key / value pair, and fusing task prompts and CAFFM features as the query, the output device voxel semantic localization model is generated.

5. The method according to claim 4, characterized in that, Step 2.1 includes the following steps: Step 2.1.1 Extract features from the visible light visual representation in the 3D site model using a multi-head self-attention mechanism to obtain visible light self-attention encoding; Step 2.1.2 For the radar positioning information in the 3D site model, perform multi-head self-attention encoding based on point cloud attributes: Define the radar point attribute set: RGB represents the color space value of this point, XYZ represents the corrected spatial coordinates, Intensity represents the radar signal strength value, and NOR represents the number of echoes. Assigning related attributes to different attention heads for encoding, through... Processing of stacked MHSA modules: It is a spatial window identifier, indicating that it is located in a window. Only points within a given window are processed using MHSA; that is, different windows undergo different MHSA processing. , , For the first A query, key, and value vector for each attention head. The scaling factor for the key vector dimension. The weights are linearly transformed for multi-head outputs.

6. The method according to claim 4, characterized in that, Step 2.2 includes the following steps: Step 2.2.1 First Cross-Attention Flow: The radar-coded features are linearly transformed into keys. ,value The visible light encoded features are linearly transformed into a query. Text features are obtained by embedding the task prompt word. , merge query vectors: ; Calculate cross attention: This indicates a splicing operation. It is the query vector of the h-th attention head. , Indicates radar characteristics as keys The result after linear transformation This indicates that the key vector is transposed. It is the dimension of the key vector. value vector The result after linear transformation Linear transformation weights representing radar characteristics; Step 2.2.2 Second Cross-Attention Flow: Linear transformation of visible light encoded features into keys ,value The radar-coded features are linearly transformed into query features. , merge query vectors: Calculate cross-attention: in The weights are the linear transformation weights for visible light characteristics. No. The query vector for each attention head is generated by the enhanced query encoding, which is a fusion of radar features and prompt words. The result is the linear transformation of visible light characteristics as bonds. The result of a linear transformation of a value vector is given by the vector, where It uses visible light characteristics as a value representation. It is a space window icon. The linear transformation weights for visible light features; Step 2.2.3 Stack steps 2.2.1 and 2.2.2 to form a basic module, and repeat this module to build the PROMPT-CAFFM structure.

7. The method according to claim 4, characterized in that, Step 3 includes the following steps: Step 3.1 Generate adaptive flight path based on device voxel localization model: Obtain the spatial location of the shooting point by parsing the semantic information of the device through task prompt words; Step 3.1.1 Parse the device semantic information using the task prompt to obtain the spatial location of the shooting point; Step 3.1.2 Calculate the safe flight area and determine the optimal waypoint in the high-precision 3D airfield model; Step 3.1.3 Generate gimbal and camera shooting parameters based on UAV parameters and spatial scale information; Step 3.1.4 Determine the waypoint flight order using the route planning algorithm and format the record as a route file: in: To supplement waypoints to the destination The distance at which it can be flown, waypoints and exist Coordinate values ​​on the axis; Step 3.2 Construct a reinforcement learning model to optimize flight routes: Step 3.2.1 Define the state space S and the action space for the flight direction. Action space and flight distance Set the reward function: waypoint and By scaling the flight distance, waypoint approach bonuses are obtained: Approach reward scaling factor; waypoint and For each flight segment, the warning reward is obtained by calculating the shortest distance from the device voxel to the segment. in It is the maximum relative distance cutoff value. It is a function for finding the shortest distance. It is a distance cutoff function. It is the warning reward scaling factor. This refers to the safe operating distance for drones and equipment. Waypoints to waypoint The trajectory of the line segment Represents the set of points of obstacles and electrical equipment in 3D coordinate space; waypoint and For each state, calculate the target imaging distortion difference to obtain the imaging similarity reward: waypoints Target imaging distortion characteristics, The similarity reward scaling factor, The imaging features or distortion features of the target waypoint B. This represents the imaging structure similarity function, which measures the degree of image feature matching. Step 3.2.2 Constructing the reinforcement learning objective function: in It is in state Select action The probability distribution, It is the state distribution function, defined as: in As a discount factor, Indicates the first The rewards for each step will be discounted to cash. Representation strategy Under the influence of the system, the time step When in a state The probability, The action value function is defined as: in The immediate reward after performing the action at time t: This represents one of the reward functions explicitly given above. Indicates the state Take multi-dimensional actions In addition to the three reward functions mentioned above, a negative reward setting also needs to be configured. Step 3.2.3 Real-time flight control: The reinforcement learning model dynamically adjusts the flight trajectory based on real-time environmental feedback.

Images