Substation adaptive inspection method based on reinforcement learning and unmanned aerial vehicle

Abstract

The invention relates to a substation adaptive inspection method based on reinforcement learning and an unmanned aerial vehicle, and belongs to the technical field of power inspection. Aiming at the problems of inspection dead angles, potential safety hazards, low efficiency of manual calibration of an air route and weak reuse of scene knowledge in the prior art, firstly, station multi-source data is acquired by utilizing an unmanned aerial vehicle radar / SLAM, heterogeneous point cloud and visible light data coordinates are unified, and a high-precision 3D station model is constructed based on super-resolution reconstruction; the radar and visible light coding features are fused through a double-flow cross attention module PROMPT-CAFFM, and an equipment voxel semantic positioning model is generated; finally, task cues are analyzed based on the model to generate an initial route, and a waypoint track is optimized in real time in combination with a reinforcement learning model; the reinforcement learning model constructs a multi-dimensional reward function according to equipment safety distance, imaging quality and flight efficiency, and dynamically corrects a flight path, and specifically comprises the following steps: 1) analyzing semantics according to cue words and outputting an air route file containing waypoint coordinates and a pan-tilt angle; and 2) dynamically adjusting the route based on the real-time sensing data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of substation safety supervision and the field of industrial image feature mining, and provides a substation adaptive inspection method based on reinforcement learning and a UAV. BACKGROUND

[0002] In the field of substation safety supervision, traditional safety supervision relies on daily patrol of teams and fixed cameras, and is limited by the device density of the station. There are many dead angles in the patrol, and the patrol scheme is not flexible enough and has a large safety hazard. In order to improve the patrol efficiency and operation safety, more and more related professional researches are carried out, and the scheme of the UAV is becoming more and more popular.

[0003] The traditional flight scheme mainly relies on excellent pilots for manual demonstration. Due to the high cost of artificial cultivation, the demonstration route cannot be verified, and the safety responsibility is not clear, which makes it difficult to promote. In the adaptive flight of the UAV, the scheme led by DJI relies on an excellent radar such as Zenith L2 to scan the station in advance, and then manually calibrate the route, which solves the problem that the route cannot be verified and provides a basic safety verification environment for flight. However, manual calibration of the route has the problems of slow speed, uncontrollable precision, and difficulty in emergency handling. Especially for the construction in the station that causes the route to be blocked, the UAV is difficult to adapt to the processing. Under normal circumstances, the obstacle avoidance of the UAV can prevent collision, but due to the dense equipment in the station, the flight control needs to close the obstacle avoidance function to normally fly in this high-density strong magnetic environment, which also causes it to be unable to normally patrol and requires someone to be on duty.

[0004] In terms of route generation, the initial mode is to demonstrate by professionals to shoot points, shooting distances, and gimbal parameters. Later, it evolved into a mode in which professional software annotates shooting points according to radar solution signals, and then an algorithm automatically plans a flight route, and finally a professional adjusts the flight point data. The new scheme is guaranteed in terms of accuracy and controllable safety, but the long period and large labor input also hinder the promotion of the scheme. In terms of flight point configuration, because of its preset characteristics, the scheme is not flexible and cannot quickly execute temporary routes.

[0005] In terms of technical scheme promotion, the existing technology only focuses on the performance improvement of a single station or device, ignoring the common research of substations. This makes us need to execute the whole process of the scheme for different substations, which consumes a lot of resources and has a great impact on the scheme due to slight changes in the environment.

[0006] In summary, for the substation operation and maintenance unmanned aerial vehicle line inspection, the route verification is difficult, the route flight right and responsibility risk is large, the route setting mode is poor in adaptability, the route cannot be quickly configured for temporary tasks, the algorithm part of the technical scheme cannot be shared, and the scene knowledge sharing is lacked, and the patent proposes a new type of substation adaptive inspection technology combining reinforcement learning technology and unmanned aerial vehicle equipment, aiming at solving the problems of route verification, online correction of route, and free configuration of route. SUMMARY

[0007] The purpose of the present application is to solve the problems of low inspection accuracy, delayed emergency response and large resource consumption caused by high safety risk in equipment-intensive areas, low efficiency and poor flexibility of traditional manual calibration of routes, and weak fusion and scene knowledge reuse of multi-source heterogeneous data in substation inspection.

[0008] In order to achieve the above purpose, the present application adopts the following technical means:

[0009] The present application provides a substation adaptive inspection method based on reinforcement learning and unmanned aerial vehicle, comprising:

[0010] Step 1: Collecting multi-source scanning data of the station by the radar equipment and SLAM equipment carried by the unmanned aerial vehicle, performing coordinate system unification processing on the heterogeneous radar point cloud and visible light data, and constructing a high-precision 3D station model based on a super-resolution reconstruction model;

[0011] Step 2: Radar signal coding and visible light signal coding are performed on the 3D station model respectively, multi-modal features are fused through a double-flow cross-attention module PROMPT-CAFFM with a prompt, and a device voxel semantic positioning model is generated;

[0012] Step 3: Generating an initial route based on the voxel semantic positioning model to analyze the task prompt, and combining a reinforcement learning model to optimize the flight trajectory of the waypoint in real time; the reinforcement learning model constructs a multi-dimensional reward function with device safety distance, imaging quality and flight efficiency, and controls the local route between waypoints through the action space;

[0013] Route planning: analyzing the device semantic positioning information according to the task prompt prompt, and outputting a route file containing waypoint coordinates and gimbal parameters through the reinforcement learning model;

[0014] Route flight: taking the route file as a reference, combining real-time radar and visible light data, and dynamically correcting the flight trajectory through the reinforcement learning model.

[0015] In the above scheme, step 1 includes the following steps:

[0016] Step 1.1 uses the unmanned aerial vehicle to scan the station at multiple angles and distances on the safe route to obtain unmanned aerial vehicle radar scanning data and visible light scanning data;

[0017] Step 1.2: High-precision scanning of the dense area of the site floor equipment by the SLAM device to obtain SLAM radar scanning data and visible light scanning data;

[0018] Step 1.3: Fusion of the unmanned aerial vehicle radar scanning data and the SLAM radar data to perform synchronous calculation of the fused features and obtain a high-precision site 3D model.

[0019] In the above scheme, step 1.3 includes the following steps:

[0020] Step 1.3.1: Using joint calibration technology on the unmanned aerial vehicle radar and visible light scanning data, SLAM radar data and visible light scanning data to obtain a coordinate system unified representation of the heterogeneous multi-modal data;

[0021] For visible light data, a deep learning model is used to reconstruct the dense depth information of the two-dimensional image, and unified site depth information is obtained according to the site spatial coordinate system, and the construction method is as follows:

[0022] Re1(X)=I A (X+O)

[0023] Where X=(x1,x2,x3) represents the relative position coordinates, x1,x2,x3 correspond to longitude, latitude and height dimensions respectively, O is the coordinate of the site base point, A represents the radar point cloud set, that is, the union of the unmanned aerial vehicle radar point cloud and the SLAM point cloud, and Re1 is an indicator function of whether the relative position of the site point X with respect to the base point O belongs to the point cloud set A;

[0024] For visible light data, a deep learning model is used to reconstruct the dense depth information of the visible light, and the coordinate representation is consistent with the radar signal:

[0025]

[0026] D is a deep learning model for dense estimation of depth information, The k-th local photographed sequence image, k∈1,……,n;

[0027] The visible light sequence is reconstructed by the deep learning model D to generate a visible light point cloud set B;

[0028] In the visible light point cloud set B, the depth information in the site spatial coordinate system is represented as:

[0029] Re2(X)=I B (X+O)

[0030] Re2 is an indicator function of whether the relative position of the site point X with respect to the base point O belongs to the visible light point cloud set B;

[0031] Step 1.3.2, based on Re1(X) and Re2(X), point cloud registration and feature reconstruction are performed using a super-resolution reconstruction model to generate a 3D station model with fused pose signals.

[0032] In the above scheme, step 2 includes the following steps:

[0033] Step 2.1, using multi-head self-attention mechanism (MHSA) for radar signal encoding and visible light signal encoding on the high-precision 3D station model, respectively:

[0034] Step 2.2, constructing a double-flow cross-attention module with prompt words PROMPT-CAFFM;

[0035] Step 2.3, generating a voxel semantic positioning model based on PROMPT-CAFFM output features and a decoder:

[0036] Using a Transformer decoder, taking CAFFM output features as Key / Value, and fusing task prompts and CAFFM features as Query, outputting a device voxel semantic positioning model.

[0037] In the above scheme, step 2.1 includes the following steps:

[0038] Step 2.1.1, using multi-head self-attention mechanism to extract features from the visible light visual representation in the 3D station model, obtaining visible light self-attention encoding;

[0039] Step 2.1.2, based on point cloud attributes, multi-head self-attention encoding is performed on the radar positioning information in the 3D station model: define a set of radar point attributes:

[0040] X = (RGB, XYZ, Intensity, NOR,...)

[0041] RGB is the color space value of the point, XYZ is the corrected spatial coordinate, Intensity is the radar signal strength value, and NOR is the number of echoes;

[0042] Assign the associated attributes to different attention heads for encoding, and process them through n1 layers of stacked MHSA modules:

[0043]

[0044] W(i) is the spatial window identifier, and table n shows that only points within window W(i) are processed using MHSA, i.e., different windows are processed using different MHSA, is the query, key, and value vector of the hth attention head, d kis the key vector dimensionality scaling factor, W OR is the multi-head output linear transformation weight.

[0045] In the above scheme, step 2.2 includes the following steps:

[0046] Step 2.2.1 first cross-attention flow:

[0047] Linearly transforming the radar encoding feature into key K r , value V r , linearly transforming the visible light encoding feature into query Q v , word embedding the task prompt word prompt to obtain the text feature Q p , fusing the query vector: Q vp = Q v + Q p ;

[0048] Computing cross-attention:

[0049]

[0050] Concat(·) represents the concatenation operation, is the query vector of the h-th attention head, represents the result of linearly transforming the radar feature as the key Key, represents transposing the key vector, d k is the dimension of the key vector, is the result of linearly transforming the value vector Value, W RO represents the linear transformation weight of the radar feature;

[0051] Step 2.2.2 second cross-attention flow:

[0052] Linearly transforming the visible light encoding feature into key K v , value V v , linearly transforming the radar encoding feature into query Q r , fusing the query vector: Q rp = Q r + Q p , computing cross-attention:

[0053]

[0054] where W VO is the linear transformation weight of the visible light feature, is the query vector of the h-th attention head, generated by the enhanced query encoding fused by the radar feature and the prompt word, is the result of linearly transforming the visible light feature as the key, is the result of the linear transformation of the value vector, wherein is the representation of the visible light feature as a value, W (i) is the spatial window identifier, W VO is the linear transformation weight of the visible light feature;

[0055] Step 2.2.3 stacks steps 2.2.1 and 2.2.2 to form a basic module, and the module is repeated to build the PROMPT-CAFFM structure;

[0056] In the above scheme, step 3 includes the following steps:

[0057] Step 3.1 generates an adaptive flight route based on the device voxel positioning model: the device semantic information is parsed through the task prompt word to obtain the spatial position of the shooting point;

[0058] Step 3.1.1 parses the device semantic information through the task prompt word prompt to obtain the spatial position of the shooting point;

[0059] Step 3.1.2 calculates the safe flight area in the high-precision 3D station model and determines the optimal navigation point;

[0060] Step 3.1.3 generates gimbal and camera shooting parameters according to the parameters of the unmanned aerial vehicle and the spatial scale information;

[0061] Step 3.1.4 determines the flight order of the navigation points through a path planning algorithm and formats the record into a flight route file:

[0062]

[0063] wherein: L si,B is the distance to be flown from the supplementary navigation point s i to the target navigation point B, ax si , ax B are the coordinate values of the navigation points s i and B on the x, y, and z axes;

[0064] Step 3.2 constructs a reinforcement learning model to realize flight route optimization:

[0065] Step 3.2.1 defines the state space S, the action space A1 (flight direction) and A2 (flight distance), and sets the reward function:

[0066] For the navigation points s i and s i+1 , the approach reward is obtained by scaling the distance to be flown:

[0067]

[0068] W a is the approach reward scaling factor;

[0069] for waypoint s i and s i+1 , the pre-warning reward of the flight segment is obtained by calculating the shortest distance from the device voxel to the line segment

[0070]

[0071] where l max is the maximum relative distance cutoff value, Dist is the function of the shortest distance, Clip is the distance clipping function, W w is the pre-warning reward scaling factor, l safe is the safe operating distance of the UAV and the device, represents the line segment trajectory of the waypoint s i to the waypoint s i+1 , Point device represents the point set of the obstacle and the power device in the 3D coordinate space;

[0072] for the state of the waypoint s i and s i+1 , the target imaging distortion difference is calculated respectively to obtain the imaging similarity reward:

[0073]

[0074] is the target imaging distortion feature of the waypoint s i , W s is the similarity reward scaling factor, T B is the imaging feature or distortion feature of the target waypoint B, and Simi represents the imaging structural similarity function, which measures the matching degree of the image features.

[0075] Step 3.2.2: Construct the reinforcement learning objective function:

[0076]

[0077] where π(a|s) is the probability distribution of selecting action a in state s, and ρ(s) is the state distribution function, defined as:

[0078]

[0079] where γ is the discount factor, γ t represents the discounting of the reward at step t, and P(s t =s|π) represents the probability that the system is in state s at time step t under the action of policy π.

[0080] The action value function is defined as:

[0081]

[0082] where r(s t ,a t ) is the immediate reward after performing action a

[0083]

[0084] Reward i (a t |s t ) represents one of the reward functions explicitly given above, debuff(a t |s t ) represents some negative reward settings in addition to the three reward functions mentioned above, which need to be set when taking a t multi-dimensional action a t in state s

[0085] Step 3.2.3 Real-time flight control: according to the real-time environmental feedback, the reinforcement learning model dynamically adjusts the flight trajectory.

[0086] Because the application adopts the above technical means, it has the following beneficial effects:

[0087] 1. The application solves the problem of insufficient model accuracy caused by a single radar data source by fusing multi-source radar data and solving technology (refer to step 1.3), achieving the effect of constructing a high-precision 3D station model.

[0088] Technical problem analysis: Traditional inspection relies on a single radar data source (such as only unmanned aerial vehicle radar), which is easy to lose details due to perspective limitations or environmental interference (such as point cloud sparseness in areas with dense underlying equipment), resulting in low 3D model resolution, affecting subsequent route planning and flight safety.

[0089] Step 1.3 innovatively fuses unmanned aerial vehicle radar and SLAM radar data, and uses joint calibration technology to unify heterogeneous coordinate systems (such as aligning radar and visible light point clouds through characteristic functions Re1(X) and Re2(X)). This realizes the complementarity of multi-source data: unmanned aerial vehicle radar covers medium and high-level equipment, and SLAM radar supplements bottom-level details. Through super-resolution reconstruction models (such as PointTransformer), point cloud registration is further optimized, eliminating the blind area of a single data source, thereby generating a detailed 3D model. This model provides a reliable spatial reference for adaptive inspection, avoiding the need for manual repair and improving overall inspection efficiency.

[0090] 2. The application solves the problem of semantic alignment of heterogeneous data (radar and visible light) through the multi-modal feature fusion module CAFFM (refer to step 2.2), achieving the effect of accurate device voxel semantic positioning.

[0091] Technical problem analysis: Radar data (including location attributes) and visible light data (including visual semantics) are difficult to directly fuse due to modal differences, resulting in inaccurate device positioning (such as device type identification errors or location offset), affecting the accuracy of route generation.

[0092] Step 2.2 Design a double cross-attention mechanism (CAFFM module) with radar encoding as Key / Value and visible light encoding as Query (or vice versa), and introduce a task prompt (Prompt) to modulate the Query vector (such as Q vρ generation in step 2.2.1). This module calculates the cross-modal interaction at the feature level through multi-head cross-attention (such as MHCA out formula), providing accurate location constraints with radar data and supplementing device semantics with visible light data (such as learning device name associations through pre-trained models). Stacking multiple layers of CAFFM (step 2.2.3) enhances feature extraction, and finally decodes to generate a voxel semantic positioning model. This ensures high-precision alignment of device location and type, laying the foundation for adaptive route planning.

[0093] 3. The present application solves the problem of low efficiency and poor flexibility of manual calibration route by using a route generation model based on reinforcement learning (refer to step 3.1), achieving the effect of quickly and adaptively generating inspection routes.

[0094] Technical problem analysis: Traditional routes rely on manual demonstration or pre-installed software, and cannot respond to temporary tasks (such as emergency inspections), and the generation process is time-consuming (requires professional adjustment of waypoints), resulting in delayed inspection response.

[0095] Step 3.1 takes the voxel semantic positioning model as the core, combines task prompts (Prompt) to analyze device locations (such as outputting shooting points according to prompt words in step 3.1), and optimizes the sequence of waypoints through path planning algorithms. The reinforcement learning model (step 3.2) integrates a reward and punishment mechanism (such as waypoint approach reward Reward warn , and early warning penalty Reward , dynamically calculates the optimal waypoints, gimbal parameters, and flight order in the 3D station model. This scheme realizes "description and generation": the user inputs the task prompt, and the system automatically outputs the complete route file without human intervention. This not only shortens the route configuration period, but also supports the rapid deployment of temporary tasks, improving the flexibility of inspection.

[0096] 4. The present application solves the problem of sharing knowledge in different substation scenarios through the overall technical architecture (refer to steps 1-3), achieving the effect of high promotion of the scheme.

[0097] Technical problem analysis: The existing scheme needs to customize the whole process (such as data collection to route generation) for each station, resulting in repeated consumption of resources, and the need to re-model due to slight changes in the environment (such as equipment displacement), which limits large-scale application.

[0098] Step 1: High-precision 3D modeling (multi-source data fusion) and step 2: voxel semantic positioning (CAFFM module) generate a general station representation; Step 3: reinforcement learning model (such as flight route model) takes local state as input and does not depend on global station data, which can be pre-trained by historical data and migrated to new scenarios (step 3.2.1). This forms a "one modeling, multiple reuse" framework: new stations only need to perform step 1 scanning, and then directly call the shared model to generate routes. This reduces deployment costs and improves the robustness of the scheme to station differences. BRIEF DESCRIPTION OF DRAWINGS

[0099] Figure 1 An adaptive unmanned aerial vehicle inspection process schematic diagram is shown in the figure;

[0100] Figure 2 A PROMPT-CAFFM structure schematic diagram is shown in the figure. DETAILED DESCRIPTION

[0101] The embodiments of the present application will be described in detail below. Although the present application will be described and illustrated by combining some specific embodiments, it should be noted that the present application is not limited to only these embodiments. On the contrary, modifications or equivalent replacements of the present application should be covered in the scope of the claims of the present application.

[0102] In addition, in order to better illustrate the present application, numerous specific details are given in the specific embodiments below. Those skilled in the art will understand that the present application can also be implemented without these specific details.

[0103] The present application provides a substation adaptive inspection method based on reinforcement learning and unmanned aerial vehicle, comprising the following steps:

[0104] Step 1: Scanning the station using an unmanned aerial vehicle and a high-precision SLAM device and solving, obtaining a high-resolution 3D station model;

[0105] Step 2: Multi-source signal fusion modeling of the high-precision 3D station model to obtain a device voxel semantic positioning model;

[0106] Step 3: Combining the device voxel semantic positioning model with a reinforcement learning model to obtain an adaptive route planning algorithm and a flight route model:

[0107] Route planning: based on the device voxel positioning model, provide device identification and position information to the system, use reinforcement learning model to output route according to customer's task requirements;

[0108] Route flying: the reinforcement learning model takes the route file (records 3D information, route waypoints, camera gimbal parameters, etc.) as reference, and uses real-time radar and visible light as correction basis to realize automatic flying of the route;

[0109] In the above scheme, step 1 specifically includes:

[0110] Step 1.1, using the unmanned aerial vehicle to scan the station from different angles and different distances on the safe route, obtaining unmanned aerial vehicle radar and visible light scanning data;

[0111] Step 1.2, high-precision SLAM scanning is performed on the dense area of the bottom equipment of the station object, and SLAM radar and visible light scanning data are obtained;

[0112] Step 1.3, fuse the unmanned aerial vehicle radar scanning data and SLAM radar data, and do synchronous solution of fusion features to obtain high-precision station 3D model.(Innovation: using different forms of radar data solving, solving the problem of low accuracy of single radar calculation)

[0113] Step 1.3.1, using joint calibration technology, the heterogeneous multi-modal data coordinate system is unified to represent the unmanned aerial vehicle radar and visible light scanning data, SLAM radar data and visible light scanning data.(Using heterogeneous data to align the coordinate system)

[0114] The unmanned aerial vehicle radar, SLAM radar, unmanned aerial vehicle and SLAM visible light imaging are unified to the relative spatial coordinate system based on the base station signal, and the origin of the spatial coordinate system is the stable point selected in the station(not easy to displace). Visible light is two-dimensional data without depth information, we use 3D reconstruction model to get depth simulation signal, and then get unified station depth information according to the station spatial coordinate system. Its construction method is:

[0115] Re1(X)=I A (X+O)

[0116] where X represents a point in the spatial coordinate system (relative position coordinates), x1, x2, x3 have three dimensions representing longitude, latitude, and height, respectively, 0 is the coordinate of the base point of the station, and the right side of the formula represents whether the current coordinate X belongs to set A in the longitude and latitude coordinate system, A represents the union of the radar point cloud set, i.e., the union of the unmanned aerial vehicle radar point cloud and the SLAM point cloud, and Re1 is an indicator function that indicates whether the relative position of the point X in the station relative to the base point 0 belongs to the point cloud set A. For the visible light sequence, we use a deep learning model such as Dense Prediction Transformers (DPT) to perform dense depth information reconstruction on the visible light, and the coordinates are represented using relative position coordinates relative to the base point 0, just like the radar signal:

[0117]

[0118] where D is a deep learning model for dense depth information estimation. After depth information reconstruction of the visible light, we can construct a visible light station device point cloud in the relative coordinate system. represents the first image of the local shooting sequence. From the deep learning model, we can obtain a visual image simulation device point cloud in the relative position spatial coordinate system, and this set is named B. Then, in the visible light point cloud B, we obtain the depth information in the station spatial coordinate system as:

[0119] Re2(X)=I B (X+O)

[0120] From the Re1 and Re2 indicator functions, we obtain a unified multi-modal data representation for the above radar data and visible light images.

[0121] Step 1.3.2 represents a coordinate system, and a super-resolution reconstruction model is used to obtain a station 3D model fused with the pose signal. Multi-source radar and multi-source visible light signals are used to construct a high-precision 3D model using a super-resolution reconstruction deep learning model.

[0122] For point cloud Re1, Re2 data, a Point Transformer point cloud registration and feature reconstruction model is used to obtain a more accurate point representation. (Here, the existing registration and reconstruction model is fine-tuned in the vertical field to obtain this model, and the model is used to fuse specific point cloud data)

[0123] In the above scheme, step 2 specifically includes:

[0124] Step 2.1.1, using multi-head self-attention mechanism on the visible light visual representation in the high-precision 3D model of the station, learning its own features, obtaining self-attention coding of visible light; (normal operation)

[0125] Step 2.1.1, using multi-head self-attention mechanism on the visible light visual representation in the high-precision 3D model of the station, learning its own features, obtaining self-attention coding of visible light; (normal operation)

[0126] Step 2.1.2, using multi-head attention mechanism on the radar positioning information representation in the high-precision 3D model of the station, learning its own position information, obtaining self-attention position coding of radar signal;

[0127] The attributes of the points in the point cloud Re1, such as RGB, corrected spatial coordinates, intensity values, and echo times, are encoded and feature extracted by using multi-head self-attention (MHSA) encoder for self-correlation coding.

[0128] For the attribute representation of the points in Re1, MHSA technology is used to obtain self-correlation coding features of the original data, where the input is X, X has m attributes, including RGB values, spatial coordinates, intensity values, and echo times. The first MHSA processing divides the input into h heads, where the feature dimensions of each head are different. For example, the RGB head has three feature dimensions, and the echo time has only one feature dimension. In order to aggregate the attributes with relatively sparse feature representation, we associate these attributes with high correlation into one head to expand the feature dimension size.

[0129] X=(RGB, XYZ, Intensity, NOR,...)

[0130] Where RGB represents the color space value of the point, XYZ represents the corrected spatial coordinates, Intensity represents the radar signal intensity value, and NOR represents the echo time. In the above point representation, RGB is divided into one head, XYZ is divided into one head, Intensity and NOR are divided into one head, and there are many actual radar attributes. We will assign attributes with high correlation to one head in this way, and obtain the feature correlation within each attribute class after MHSA processing,

[0131]

[0132] Where W(i) is the spatial window identifier, indicating that only the points located in the window W(i) are processed using MHSA, i.e. different windows are processed using different MHSA. WOR is the weight of linear processing of multi-head results in the self-encoding stage. The radar self-encoding stage will stack n1 MHSA layers for feature extraction. After the first layer of MHSA processing, the first head will output hi feature dimensions. In the first n1 / 3 layers, the number of output channels of each head of the previous layer is used as the number of groups of the next layer, that is, in these layers, only the attributes inside are modeled. The subsequent layers use the design of the ordinary transformer module, and each head uses the same number of channels to enhance cross-attribute modeling. Although the channel allocation of the head is different on the pipeline, the calculation formula is consistent with the calculation method of MHSAout above.

[0133] Step 2.2, using a custom cross-attention module for visible light self-attention encoding and self-attention position encoding, obtaining a feature fusion module (CAFFM); (Innovation: developed CAFFM structure, guided the model to learn two kinds of heterogeneous data, fused multi-modal data features, and provided decoding and mining data dependence for voxel class and position information)

[0134] Step 2.2.1, using radar encoding features as KEY and VALUE, and visible light encoding features as QUERY, to generate cross-attention features and update radar features;

[0135] For the above radar MHSAout data, KEY (K r ), VALUE (V r ) are obtained through linear transformation;

[0136] For the above visible light MHSAout data, QUERY (Q v ) is obtained through linear transformation;

[0137] For the description of the task text prompt, the pre-trained word embedding model is used to obtain the text feature Q p ;

[0138] For Q v and Q p , use addition to fuse two encodings to obtain enhanced query encoding Q vp ;

[0139] For K r , V r , Q v , the multi-head cross-attention calculation model is obtained by bringing into the MHSA calculation formula, which can be simplified as:

[0140]

[0141] where W R0is a linear transformation weight of radar features about multi-head output results in the cross attention stage.

[0142] Step 2.2.2, using visible light encoding features as KEY, VALUE, radar encoding features as

[0143] QUERY, generating cross attention features, and updating visible light features;

[0144] For the above visible light MHSAout data, KEY(K v ), VALUE(V v ) are obtained by linear transformation respectively;

[0145] For the above radar MHSAout data, QUERY(Q r ) is obtained by linear transformation;

[0146] For Q r and Q p , two kinds of encoding are fused by addition to obtain enhanced query encoding

[0147] For K r , V r , Q v , the multi-head cross attention calculation model is obtained by bringing into the MHSA calculation formula, and the calculation formula can be simplified as:

[0148]

[0149] Step 2.2.3, stack the above two steps to form a basic module, and repeat the basic module to form a CAFFM module;

[0150] Step 2.3, build a decoder of multi-source fusion features to generate a voxel semantic positioning module; the decoder uses a transformer structure as a basic module, KEY and VALUE are obtained by linear transformation of the features generated by the CAFFM module, the features extracted by the prompt and the CAFFM are fused as QUERY, and the module finally outputs a voxel positioning model with nonlinear perception;

[0151] Like the encoder stage, the prompt is a dynamic task description text, which needs to be expanded by a large language model, and the word embedding encoding of the language model is used to obtain the prompt encoding.

[0152] In the above scheme, step 3 specifically comprises:

[0153] Step 3.1, for the voxel positioning model, an adaptive generated flight route is obtained according to the task prompt prompt; (a 3D flight route point information is generated using a multi-modal visual large model)

[0154] For the voxel positioning model, give the description of the desired route, after word embedding processing, get the prompt encoding; give the unified representation of radar, visible light signal, and give its spatial position and key attribute information by the voxel positioning model. Use these devices and their attribute information to construct the waypoint and route information.

[0155] For the task description, use the voxel positioning model to get the position of the target to be photographed in the relative spatial coordinate system (shooting point);

[0156] For the shooting point, calculate the safe flight area in the high-precision 3D station to get the optimal waypoint;

[0157] For the waypoint and shooting point array, calculate the optimal gimbal and camera shooting parameters (shooting related parameters need to be corrected in real time in actual application) through the parameters of the unmanned aerial vehicle and the spatial scale information;

[0158] For the waypoint, use the path planning algorithm to get the optimal waypoint flight order;

[0159] For the waypoint flight order, waypoint gimbal and camera shooting parameters, sequentially format the record to get the route file.

[0160] Step 3.2, for the high-precision 3D station model and voxel positioning model, configure the reward and penalty terms of the reinforcement learning model to get a reinforcement learning model that can be used for route generation and route flight; (propose a reinforcement learning scheme based on waypoints, which solves the model migration and adaptation problem of different stations)

[0161] Step 3.2.1, use historical 3D information and device voxel set to set flight motion direction space and flight motion distance space, set reward and punishment functions in terms of device and route distance, distance to be flown, and waypoint shooting similarity, weight and harmonize these reward scores, set the maximum score function to get the reinforcement learning model, which is used for the calculation model of waypoint and route flight.

[0162] For the flight motion direction, set the "up, down, front, back, left, right" six directions in the relative position space as the motion direction action to get the motion direction space A1.

[0163] For the flight motion distance, set the flight distance per unit time at the maximum safe flight speed as the maximum motion distance, and set the flight distance per unit time at the minimum flight speed as the unit flight distance, discretize the maximum motion distance by the unit flight distance to get the motion distance space A2.

[0164] For the waypoint A to waypoint B, plan a series of supplementary waypoints to get the execution route AirLine.

[0165] For the supplementary waypoint, the current supplementary waypoint record is s i , the next supplementary waypoint record is s i+1 , and all possible supplementary waypoints record the state space S of the supplementary waypoint.

[0166] For the supplementary waypoint s i and the waypoint B, the minimum flight distance in the state space is calculated to obtain the distance to be flown L si，B .

[0167]

[0168] For the waypoints s i and s i+1 , the approach reward of the waypoint is obtained by scaling the distance to be flown.

[0169]

[0170] Where W a is the scaling factor for the approach of the supplementary waypoint.

[0171] For the line segment between the waypoints s i and s i+1 , the warning reward of the flight segment is obtained by calculating the shortest distance from the device voxel to the line segment.

[0172]

[0173] Where l max is the maximum relative distance cutoff value, Dist is the function of the shortest distance, Clip is the distance clipping function, W w is the warning reward scaling factor, and l safe is the operational safety distance of the UAV and the device.

[0174] For the states of the waypoints s i and s i+1 , the target imaging distortion difference is calculated respectively to obtain the imaging similarity reward similarity.

[0175]

[0176] The objective equation of the reinforcement learning model is:

[0177]

[0178] Where π(a|s) is the probability distribution of selecting action a in state s, and ρ(s) is the state distribution function:

[0179]

[0180] The probability function P is the probability of being in state s t at time t under the distribution π, γ is the discount factor that balances the weights of states at different times. Q π (s, a) is the action value function, which represents the expected cumulative reward following the policy π after performing action a in state s. The cumulative reward function is obtained by recursively taking the expectation of the above three reward functions according to the state sequence (time). The relationship between the expected cumulative reward function and the above three reward functions of state transition can be briefly described by the following formula:

[0181]

[0182] Where r(s t , a t ) is the immediate reward after performing action at time t:

[0183]

[0184] Where debuff is some penalty term applied in the state transition process, including action penalty for moving away from the target point, and area wandering penalty.

[0185] Step 3.2.2, issue a specific flight task, the UAV scans the station in real time, feeds back the current environment to the reinforcement learning model, and according to the current environment voxel information, flight path information, sensor environment information, the reinforcement learning model gives the next step action guidance.

Claims

1. A substation adaptive patrol method based on reinforcement learning and unmanned aerial vehicle, characterized in that, The application relates to a method for unmanned aerial vehicle (UAV) flight route planning and execution. The method comprises the following steps: Step 1: Collecting multi-source scanning data of the station by a UAV-mounted radar device and a SLAM device, performing coordinate system unification processing on the heterogeneous radar point cloud and visible light data, and constructing a high-precision 3D station model based on a super-resolution reconstruction model; Step 2: Performing radar signal coding and visible light signal coding on the 3D station model, fusing multi-modal features through a double-flow cross-attention module PROMPT-CAFFM with a prompt, and generating a device voxel semantic positioning model; Step 3: Generating an initial flight route based on the voxel semantic positioning model to analyze the task prompt, and combining a reinforcement learning model to optimize the flight point flight trajectory in real time; the reinforcement learning model constructs a multi-dimensional reward function based on a device safety distance, an imaging quality and flight efficiency, and controls the local flight route between flight points through an action space. Route planning: analyzing device semantic positioning information according to a task prompt prompt, and outputting a flight route file containing flight point coordinates and a pan-tilt parameter through a reinforcement learning model; 2. The method of claim 1, wherein, Flight route execution: taking the flight route file as a reference, combining real-time radar and visible light data, and dynamically correcting a flight trajectory through a reinforcement learning model. Step 1 comprises the following steps: Step 1.1: Scanning the station at multiple angles and distances on a safe flight route by a UAV to obtain UAV radar scanning data and visible light scanning data; Step 1.2: High-precision scanning of a station bottom equipment dense area by a SLAM device to obtain SLAM radar scanning data and visible light scanning data; 3. The method of claim 2, wherein, Step 1.3: Fusing the UAV radar scanning data and the SLAM radar data, performing synchronous calculation of the fused features, and obtaining a high-precision station 3D model. Step 1.3 comprises the following steps: Step 1.3.1: Using joint calibration technology to obtain a coordinate system unified representation of the heterogeneous multi-modal data from the UAV radar and visible light scanning data, the SLAM radar data and the visible light scanning data; Re1(X) = I A (X+O) For visible light data, a deep learning model is used to reconstruct the dense depth information of a two-dimensional image, and unified station depth information is obtained according to a station spatial coordinate system, and the construction mode is as follows: Wherein X=(x1, x2, x3) represents a relative position coordinate, x1, x2 and x3 respectively correspond to longitude, latitude and height dimensions, O is a coordinate of a station base point, A represents a radar point cloud set, that is, a union of UAV radar point clouds and SLAM point clouds, and Re1 is an indicator function of whether the relative position of a station midpoint X to a base point O belongs to the point cloud set A; D is a deep learning model that performs dense estimation of depth information, the first local sequence image, k e 1,..., n; For visible light data, a deep learning model is used to reconstruct the dense depth information of visible light, and the coordinate representation is consistent with that of radar signals: A deep learning model D is used to reconstruct the dense depth of the visible light sequence to generate a visible light point cloud set B; Re2(X) = I B (X+O) In the visible light point cloud set B, the depth information in the station spatial coordinate system is as follows: Re2 is an indicator function of whether the relative position of a station midpoint X to a base point O belongs to the visible light point cloud set B; Step 1.3.2: Based on the Re1(X) and Re2(X), point cloud registration and feature reconstruction are performed by using a super-resolution reconstruction model to generate a 3D station model with fused pose signals.

4. The method of claim 2, wherein, Step 2 includes the following steps: Step 2.1 uses multi-head self-attention mechanism (MHSA) for radar signal encoding and visible light signal encoding on the high-precision 3D station model, respectively: Step 2.2 constructs a double-flow cross-attention module PROMPT-CAFFM with prompt words: Step 2.3 generates a voxel semantic positioning model based on PROMPT-CAFFM output features and a decoder: Use the Transformer decoder, take the CAFFM output features as Key / Value, and fuse the task prompt words and CAFFM features as Query to output the device voxel semantic positioning model.

5. The method of claim 4, wherein, Step 2.1 includes the following steps: Step 2.1.1 uses multi-head self-attention mechanism to extract features from the visible light visual representation in the 3D station model, obtaining visible light self-attention encoding; Step 2.1.2 performs multi-head self-attention encoding on the radar positioning information in the 3D station model based on point cloud attributes: define a set of radar point attributes: X = (RGB, XYZ, Intensity, NOR, ···) RGB is the color space value of the point, XYZ is the corrected spatial coordinate, Intensity is the radar signal strength value, and NOR is the number of echoes; Assign the associated attributes to different attention heads for encoding, and process them through n1 layers of stacked MHSA modules: W(i) is a spatial window identifier, table n shows that only the points in the window W(i) are processed by MHSA, that is, different windows are processed by different MHSA, is the query, key, value vector for the h-th attention head, d k is the key vector dimension scaling factor, W OR is the multi-head output linear transformation weight.

6. The method of claim 4, wherein, Step 2.2 includes the following steps: Step 2.2.1 first cross-attention flow: Linearly transform the radar encoding feature to key K r , value V r Linearly transform the visible light encoding feature to query Q v Word embedding of the task prompt word prompt to get the text feature Q p Fuse the query vector: Q vp = Q v + Q p ; Calculate cross-attention: Concat(·) represents a concatenation operation, is the query vector Query of the h-th attention head, represents the result of linear transformation of the radar feature as the key Key, represents transposing the key vector, d k is the dimension of the key vector, is the result of linear transformation of the value vector Value, W RO represents the linear transformation weight of the radar feature; Step 2.2.2 second cross-attention flow: Linearly transforming visible light encoding features to key K v , value V v Linearly transforming radar encoding features to query Q r , fusing query vectors: Q rp = Q r + Q p , computing cross attention: wherein W VO is a linear transformation weight for visible light features, the query vector of the h-th attention head, generated by the enhanced query encoding after the radar features and the prompt words are fused, is the result after linear transformation of the visible light features as keys, is the result after linear transformation of the value vector, wherein is the representation of the visible light features as values, W (i) is a spatial window identifier, W VO is a linear transformation weight for visible light features; Step 2.2.3 stack steps 2.2.1 and 2.2.2 to form a basic module, and repeat the module to build the PROMPT-CAFFM structure.

7. The method of claim 4, wherein, Step 3 includes the following steps: Step 3.1 generates an adaptive flight path based on the device voxel positioning model: obtain the shooting point spatial position by analyzing the device semantic information through the task prompt words; Step 3.1.1 obtain the shooting point spatial position by analyzing the device semantic information through the task prompt words prompt; Step 3.1.2 calculate the safe flight area in the high-precision 3D station model to determine the optimal waypoint; Step 3.1.3 generate gimbal and camera shooting parameters according to the parameters of the unmanned aerial vehicle and spatial scale information; Step 3.1.4 determine the flight order of the waypoints through the path planning algorithm and format the record as a flight path file: Wherein: L si,B To make up for the waypoint s i The distance to be flown to the target waypoint B, ax si , ax B The coordinate values of the waypoints s i And B on the x, y, z axes; Step 3.2 constructs a reinforcement learning model to optimize the flight path: Step 3.2.1 defines the state space S, action space A1 (flight direction) and A2 (flight distance), and sets the reward function: For waypoint s i and s i+1 , the waypoint approach reward is obtained by scaling the distance to fly: W a Approach reward scaling factor; To waypoint s i And s i+1 The line segment, by calculating the shortest distance of the device voxel to the line segment, get the warning reward of the flight segment where l max is the maximum relative distance cutoff value, Dist is a function of the shortest distance, Clip is a distance clipping function, W w is the pre-warning reward scaling factor, l safe is the operational safety distance of the UAV and the device, denotes the line segment trajectory from waypoint s i to waypoint s i+1 , Point device denotes the set of points of the obstacle and the power device in the 3D coordinate space; To waypoint s i And s i+1 The target imaging distortion difference is calculated according to the state of the waypoint s, and the imaging similarity reward is obtained. For the target imaging distortion feature of waypoint s i W is the target imaging distortion feature of waypoint s s T is the similarity reward scaling factor B T is the target imaging feature or distortion feature of waypoint B, and Simi represents an imaging structural similarity function, which measures the matching degree of image features. Step 3.2.2 constructs a reinforcement learning objective function: where π(a|s) is the probability distribution of selecting action a in state s, and ρ(s) is the state distribution function, defined as: where γ is a discount factor, γ t represents the reward at step t discounted by γ, P(s t = s | π) represents the probability that the system is in state s at time step t under policy π. The action value function is defined as: where r(s t , a t ) is the immediate reward after performing action at time t: Reward i (a t |s t ) represents one of the reward functions explicitly given above, debuff(a t |s t ) represents a negative reward setting in addition to the three reward functions mentioned above, which needs to be set when taking a multi-dimensional action a t in state s t ; Step 3.2.3 real-time flight control: dynamically adjust the flight trajectory based on real-time environmental feedback.

Images