Unmanned aerial vehicle dynamic obstacle avoidance inspection path planning method based on reinforcement learning

Abstract

The present disclosure provides a kind of unmanned aerial vehicle dynamic obstacle avoidance inspection path planning method based on reinforcement learning, including the time sequence consistency analysis of visible light image sequence collected in continuous time window, constructs visual observation consistency description result;Based on visual observation consistency description result and in combination with unmanned aerial vehicle own motion information, the spatial validity of candidate obstacle is determined, and the space effective obstacle region is output by screening out mirror reflection virtual image;Based on space effective obstacle region, the environment state description for reinforcement learning decision is constructed by fusing unmanned aerial vehicle ontology state and inspection task state;Flight control action is generated online using reinforcement learning strategy model, and dynamic obstacle avoidance inspection path is output through rolling planning and safety check, which can accurately identify real obstacles and achieve reliable obstacle avoidance in strong mirror reflection scene, solve the autonomous inspection safety problem caused by virtual-real imaging confusion.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous navigation and path planning technology for unmanned aerial vehicles (UAVs), and in particular to a method for dynamic obstacle avoidance and inspection path planning for UAVs based on reinforcement learning. Background Technology

[0002] In the field of industrial inspection, drones can replace manual labor in performing high-risk, highly repetitive inspection tasks, improving operational efficiency and safety. In these tasks, drones need to perceive their surroundings in real time and dynamically plan their flight paths to avoid suddenly appearing obstacles, such as birds, drones, and floating objects, ensuring the continuity of the inspection process and flight safety. Currently, vision-based obstacle avoidance typically involves using cameras to capture environmental images, employing target detection and tracking algorithms to identify obstacles, and then combining this with path planning algorithms to generate local obstacle avoidance paths.

[0003] However, in inspection scenarios with strong specular reflection characteristics, such as the surfaces of metal pipe corridors, substation busbars, the outer walls of oil and gas storage tanks, and aluminum sound barriers, these highly reflective surfaces are similar to mirrors. When a dynamic obstacle enters the inspection field of view, its visible light imaging simultaneously contains both a real image of the obstacle formed by the direct path and a virtual image formed by specular reflection. These two types of images are highly similar in spatial location, contour orientation, and scale characteristics, and they overlap, leading to a systematic distortion of the obstacle's boundary structure in the image. Since the reflected virtual image does not correspond to its actual physical location, and visual detection algorithms struggle to reliably distinguish between the virtual and real images from a single imaging perspective, a stable and non-linearly correctable offset occurs between the obstacle's geometric position in the image coordinate system and its actual position in the inspection space. This phenomenon fundamentally disrupts the reliable mapping relationship between visual detection results and spatial obstacle avoidance decisions, rendering the detection results unusable as a valid basis for UAV path planning and potentially leading to accidental obstacle avoidance or collision risks.

[0004] Most existing dynamic obstacle avoidance algorithms are not designed to handle interference from specular reflections and virtual images. Directly inputting observations containing virtual images into the path planning module, especially in reinforcement learning systems that rely on environmental state modeling, can lead to distorted decision-making, potentially causing accidental obstacle avoidance or collisions, threatening inspection safety and efficiency. Therefore, there is an urgent need for a UAV path planning method capable of accurately identifying real obstacles and reliably avoiding them in strong specular reflection scenarios, in order to solve the safety problem of autonomous inspection caused by the confusion between real and virtual images. Summary of the Invention

[0005] In view of this, in order to solve the problems brought about by the existing technology, this application provides a method for dynamic obstacle avoidance and inspection path planning of UAVs based on reinforcement learning.

[0006] In a first aspect, this disclosure provides a method for dynamic obstacle avoidance and inspection path planning of unmanned aerial vehicles (UAVs) based on reinforcement learning, the method comprising:

[0007] S1. Perform temporal consistency analysis on the visible light image sequence acquired within a continuous time window to construct a visual observation consistency description result, which is used to characterize the stability of the candidate obstacle imaging structure.

[0008] S2. Based on the visual observation consistency description results and combined with the UAV's own motion information, determine the spatial effectiveness of candidate obstacles, filter out specular reflection virtual images and output the spatially effective obstacle area.

[0009] S3. Based on the effective obstacle area in the space, the UAV's own state and the inspection task state are integrated to construct an environmental state description for reinforcement learning decision-making.

[0010] S4. Based on the environmental state description, generate flight control actions online using a reinforcement learning strategy model, and output a dynamic obstacle avoidance and inspection path through rolling planning and safety verification.

[0011] Optionally, S1 includes:

[0012] Acquire a sequence of visible light images within a continuous time window and perform preprocessing;

[0013] Extract and temporally correlate the contours of candidate obstacles from the preprocessed image sequence, and calculate the basic geometric features of each candidate obstacle in each frame within the time window;

[0014] Based on the aforementioned fundamental geometric features, the position change energy, scale change energy, and contour continuity and fragmentation penalty energy of each candidate obstacle are calculated and synthesized into a visual stability score.

[0015] Based on the visual stability score, a visual observation consistency description result is generated for each candidate obstacle.

[0016] Optionally, S2 includes:

[0017] Based on the visual observation consistency description results, the initial screening score is calculated and candidate obstacles that are initially stable are selected.

[0018] For candidate obstacles that pass the initial screening, verify whether their pixel motion and scale changes conform to the camera motion geometry based on the UAV pose change estimation, and calculate the geometric mapping consistency score.

[0019] By combining the initial screening score and the geometric mapping consistency score, spatial validity is determined, and the spatial validity confidence level is calculated.

[0020] The output includes the spatial validity determination results, which include the region of effective obstacles and their confidence levels.

[0021] Optionally, the spatial validity determination adopts a dual-threshold mechanism with hysteresis, specifically:

[0022] If the geometric mapping consistency score If so, it is marked as a valid occupied area;

[0023] like If so, it is marked as a non-valid spatial region;

[0024] like If so, the previous marked state will be maintained;

[0025] in, To enable the threshold, To close the threshold, and .

[0026] Optionally, S3 includes:

[0027] Extract effective obstacles located within the local perception space of the UAV, and calculate the risk intensity of each effective obstacle relative to the UAV. The risk intensity integrates the spatial effectiveness confidence of the obstacle, the relative distance between the UAV and the obstacle, and the speed of the UAV.

[0028] The drone's own status, inspection mission status, and surrounding threat status based on the risk intensity are integrated to construct a standardized state vector;

[0029] The state vector is encapsulated as an environment state description object, and the incremental control mode action space of the reinforcement learning agent is defined.

[0030] Optionally, S4 includes:

[0031] In each decision cycle, the current environmental state description is input into the reinforcement learning policy network, and flight action commands are generated based on the incremental control mode action space.

[0032] A single-step safety margin check is performed on the flight maneuver command, and the safety margin check is based on the safety buffer zone of the effective obstacle area in the space;

[0033] Perform the actions that pass the verification and generate the predicted path segment in the rolling time domain;

[0034] The trajectory segments executed in each cycle are spliced ​​together to form a complete dynamic obstacle avoidance and inspection path.

[0035] Optionally, the single-step safety margin check is as follows:

[0036] Calculate and verify the single-step safety margin Is it greater than zero, where The calculation formula is:

[0037]

[0038] in, To predict the farthest distance from the UAV to the boundary of the i-th effective obstacle safety buffer zone after performing the action, Where v is the safety radius and v is the current speed of the drone. For decision step size, This is the maximum acceleration.

[0039] Optionally, generating the predicted path segment in the rolling time domain includes:

[0040] Based on the UAV kinematic model and flight action commands that have passed safety margin verification, a predicted state trajectory is generated as a rolling path segment within a preset number of rolling planning steps.

[0041] Calculate the comprehensive cost of the rolling path segment, which is a weighted sum of three factors: risk cost representing obstacle avoidance safety, target distance cost representing task advancement efficiency, and motion change cost representing trajectory smoothness.

[0042] The overall performance of the current rolling path segment is evaluated based on the comprehensive cost, and this performance is used as a real-time basis for flight health monitoring, decision quality assessment, or online strategy adjustment.

[0043] In a second aspect, this disclosure provides an electronic device including a memory and at least one processor, the memory storing a computer program, and the processor executing the computer program to implement the method of the first aspect described above.

[0044] Thirdly, this disclosure provides a computer storage medium storing a computer program that, when executed, implements the method described in the first aspect.

[0045] The beneficial effects of this disclosure are that, compared with the prior art, this disclosure has the following advantages:

[0046] 1) To address the problem of high similarity, overlapping contours, and difficulty in distinguishing real obstacles from reflected virtual images caused by strong specular reflection in visual imaging, this paper analyzes the temporal stability of geometric features such as position, scale, and contour continuity of candidate obstacles across multiple frames. This effectively distinguishes real obstacles with stable spatial projection characteristics from unstable virtual images generated by specular reflection. This solves the fundamental problem of traditional visual detection algorithms failing in such scenarios, providing an accurate and reliable perceptual foundation for subsequent processing.

[0047] 2) To address the issue of the mapping relationship between obstacle image coordinates and their true spatial location being disrupted due to the confusion between virtual and real images, thus preventing the detection results from being directly used for path planning, this paper combines the UAV's own motion information to verify the motion consistency and scale change patterns of candidate obstacles that have passed the initial screening. This accurately identifies valid obstacles that occupy a real three-dimensional space and filters out virtual images that only exist in the image. This reconstructs a reliable mapping relationship from image observation to spatial decision-making.

[0048] 3) To address the issue that existing dynamic obstacle avoidance algorithms fail to consider virtual image interference and may lead to accidental obstacle avoidance or collisions by directly inputting noisy observations into the planning module, a reinforcement learning environment state description that relies solely on valid spatial obstacles was constructed to drive online path planning and decision-making. The entire decision-making loop uses only rigorously verified real obstacle information as environmental constraints, ensuring the purity and reliability of the input information to the planning module. This immunes the interference of specular reflection virtual images at the system level, fundamentally guaranteeing the safety of the generated dynamic obstacle avoidance inspection path while maintaining the execution efficiency of the inspection task. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0050] Figure 1 A flowchart of the UAV dynamic obstacle avoidance and inspection path planning method based on reinforcement learning provided in this disclosure is shown.

[0051] Figure 2 A flowchart of visual observation consistency analysis and initial screening judgment provided in an embodiment of this disclosure is shown;

[0052] Figure 3 A flowchart illustrating the geometric mapping consistency verification and spatial validity determination provided in an embodiment of this disclosure is shown.

[0053] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0054] The present disclosure will be further described below with reference to the accompanying drawings. The following embodiments are only used to illustrate the technical solutions of the present disclosure more clearly, and should not be used to limit the scope of protection of the present disclosure.

[0055] The components of the embodiments of the invention described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0056] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of the invention, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0057] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0058] In inspection scenarios with high specular reflection characteristics, such as metal pipe corridors, substation busbars, oil and gas storage tank exteriors, and aluminum sound barriers, when a dynamic obstacle enters the inspection field of view, its visible light imaging result simultaneously includes a virtual image formed by specular reflection and a real image formed by the direct viewing path. These two types of images are highly similar in spatial location, contour orientation, and scale characteristics, and they overlap, leading to a systematic distortion of the obstacle's boundary structure in the image. Since the reflected virtual image does not correspond to its actual physical location, and the visual inspection process struggles to reliably distinguish between the virtual and real images from a single imaging perspective, a stable and non-linearly correctable offset occurs between the obstacle's geometric position in the image coordinate system and its actual position in the inspection space. This phenomenon fundamentally disrupts the reliable mapping relationship between visual inspection results and spatial obstacle avoidance decisions, rendering the inspection results unusable as a valid basis for UAV path planning and potentially leading to accidental obstacle avoidance or collision risks.

[0059] In view of this, this disclosure provides a reinforcement learning-based method for dynamic obstacle avoidance and inspection path planning of unmanned aerial vehicles (UAVs). The method aims to filter obstacle information that corresponds only to real physical space by performing temporal consistency analysis and spatial validity reconstruction on visual perception results. Based on this, a clean and reliable environmental state description is constructed to drive a reinforcement learning model for online decision-making and path generation, thereby achieving safe and reliable dynamic obstacle avoidance and inspection of UAVs under strong specular reflection interference. The implementation process of this method will be described in detail below with reference to the accompanying drawings and through several embodiments.

[0060] Figure 1 A flowchart of the UAV dynamic obstacle avoidance and inspection path planning method based on reinforcement learning provided in this disclosure is shown below. Figure 1 As shown, the process may include the following steps:

[0061] S1: Perform temporal consistency analysis on the visible light image sequence acquired within a continuous time window to construct a visual observation consistency description result, which is used to characterize the stability of the candidate obstacle imaging structure.

[0062] This paper constructs a visual observation consistency description for specular reflection scenarios. Addressing the issue of strong specular reflection making it difficult to distinguish between real obstacles and virtual images in a single frame, this step analyzes the temporal consistency of the geometric features of obstacle images within a continuous time window and calculates their visual stability score. This constructs an observation consistency description that characterizes the stability of the imaging structure, providing a basis for distinguishing between real obstacles and specularly reflected virtual images. Figure 2 The flowchart of visual observation consistency analysis and initial screening judgment provided in the embodiments of this disclosure is shown, such as Figure 2 As shown, this is achieved through the following sub-steps.

[0063] S1.1: Preprocess the continuous image sequence and construct a time window to provide a basis for time series analysis.

[0064] Acquire a sequence of visible light images captured within a continuous time window, starting from the current moment. The length of this time window is denoted as N, and its value ranges from 3 to 15 frames. In practice, this can be adjusted based on the drone's flight speed and the dynamic range of the scene. Simultaneously, acquire pre-calibrated camera intrinsic parameters.

[0065] For each frame of the original image within the time window, specular highlighting and brightness normalization are performed sequentially. Specular highlighting aims to suppress local pixel saturation caused by strong specular reflections, for example, by compressing the grayscale values ​​of overexposed areas to a resolvable range using adaptive thresholding or inverse tone mapping techniques. Brightness normalization eliminates inter-frame brightness differences caused by variations in lighting, for example, by using histogram equalization or adjustment methods based on a reference white point, ensuring a consistent brightness level throughout the entire image sequence. These two processes work together to suppress localized whitening and edge drift caused by specular reflections within a controllable range.

[0066] Distortion correction is performed on each frame of the image based on camera intrinsic parameters to eliminate image distortion introduced by lens optical characteristics and ensure that the same physical edge has consistent pixel geometry in consecutive frames.

[0067] A unified temporal index is established for each preprocessed image frame, forming a preprocessed image sequence that can be used for subsequent time series analysis. This sequence is the basis for subsequent contour extraction and consistency calculation.

[0068] S1.2: Extract and associate candidate obstacle contours, and calculate their basic geometric features.

[0069] Based on the obtained preprocessed image sequence, the connected boundary contours of all candidate obstacles are extracted in each frame. Contour extraction can be achieved by combining edge detection algorithms with contour finding methods.

[0070] To track the same obstacle over time, an association number needs to be established for each candidate obstacle across the entire time window. The association rules prioritize a combination of spatial proximity and shape similarity criteria. Specifically, for a contour extracted in the current frame, the contour in the next frame that is closest to its centroid and has the most similar shape is searched for and associated with it. This ensures continuous tracking of the same physical object within the time window and prevents the same obstacle from being split into multiple segments in different frames.

[0071] For each candidate obstacle that successfully performs temporal correlation, its underlying geometric feature set is computed in each frame of the time window, including:

[0072] pixel coordinates of the centroid of the contour ;

[0073] The dimensions of the bounding rectangle of the outline are usually expressed in terms of its width and height;

[0074] Pixel area enclosed by the outline ;

[0075] Perimeter of the outline ;

[0076] Number of connected components of the contour It is used to describe the degree of fragmentation of a contour. For example, a complete object may be segmented into multiple disconnected parts in an image due to highlight reflection.

[0077] These fundamental geometric features collectively describe the obstacle's positional changes, scale variations, shape continuity, and degree of fragmentation within the time window, providing a data foundation for subsequent temporal consistency analysis.

[0078] S1.3: Perform temporal consistency analysis, calculate visual stability score based on geometric features, and quantify the temporal stability of the imaging structure.

[0079] Based on the fundamental geometric feature set of each candidate obstacle output by S1.2, three energy indices are calculated for each obstacle within a time window of length N, including position change energy. Scale change energy Contour continuity and fragmentation penalty energy And finally synthesize a unified visual stability score.

[0080] Calculate the energy of position change This is used to quantify the cumulative drift intensity of the obstacle's centroid across consecutive frames. The calculation formula is:

[0081]

[0082] in, , Let be the pixel coordinates of the centroid of the obstacle's outline in frame t; As a scale reference, the median of the length of the diagonal of the inscribed rectangle of the obstacle within the time window is taken.

[0083] The essence of this formula is to average the squares of the centroid displacements between adjacent frames under a unified scale reference. The larger the value, the more unstable the position of the obstacle in the image, and the more likely it is to be a drift caused by a mirror image.

[0084] Calculate the energy of scale change This is used to quantify the nonlinear fluctuations of the contour area within a time window, thereby distinguishing between the smooth scale changes of real objects caused by distance variations and the abrupt boundary expansion caused by specular reflection. Its calculation formula is:

[0085]

[0086] in, Let be the outline area of ​​the t-th frame; Let be the perimeter of the outline in frame t; ln is the natural logarithm. The first term of this formula uses the logarithmic ratio to characterize the proportional relationship of area change, making it more sensitive to abrupt changes; the second term is a normalization term for the perimeter change. The combination of the two can effectively distinguish between smooth changes and abrupt changes.

[0087] Calculate contour continuity and fragmentation penalty energy This is used to quantify whether the obstacle's outline maintains a continuous occupied area over time, and whether the outline is intact, i.e., not fragmented. The calculation formula is:

[0088]

[0089] in, The overlap rate of the obstacle outline filling region between frame t and frame t-1 is denoted by 0, and its value ranges from 0 to 1. It is obtained by calculating the ratio of the intersection to the union of the outline filling regions of the two frames. Let be the number of connected components of the obstacle profile in frame t. The smaller the value, the worse the spatial overlap of objects between adjacent frames; The greater the deviation from 1, the more fragmented the outline, which is common when specular highlights cut off the boundary.

[0090] The above three energy indicators are combined into a unified visual stability score S, and the calculation formula is as follows:

[0091]

[0092] Where exp is the natural exponential function; , , The values ​​are weighting coefficients, ranging from 0.1 to 10, which can be used to emphasize the importance of positional stability, scale stability, or continuity stability depending on the specific inspection scenario. The value of S ranges from 0 to 1. The essence of this formula is to weight and summarize the three types of instability energy and then compress it using an exponential function. The exponential function's properties prevent small fluctuations from being excessively amplified, while systematic drift and fragmentation caused by mirror images (i.e.,...) are mitigated. , , A larger S-value will significantly reduce the S-value. Therefore, the closer the S-value is to 1, the more stable the imaging structure is within the time window, and the more likely it is to be a real obstacle; the lower the S-value, the more likely it is to be a virtual image produced by specular reflection.

[0093] S1.4: Based on the temporal consistency analysis results, encapsulate the visual observation consistency description results and generate preliminary labels according to the stability threshold.

[0094] Based on the calculated energy of the position change of each candidate obstacle Scale change energy Continuous fragmentation energy In addition to the visual stability score S, these data are encapsulated with representative contour summary information of the obstacle within the time window, such as the centroid trajectory, to form a visual observation consistency description of the obstacle.

[0095] At the same time, according to the preset stability threshold Generate an initial stability marker and stability threshold for each obstacle. The value ranges from 0.3 to 0.8, and can be adjusted in engineering practice based on the reflectivity and inspection distance. If If it is positive, it is marked as a stable candidate; otherwise, it is marked as an unstable candidate.

[0096] The visual observation consistency descriptions of all candidate obstacles are compiled to form a visual observation consistency description package for specular reflection scenarios. This package contains key information that distinguishes stable spatial projection imaging structures from unstable specular reflection imaging structures, and provides a direct basis for spatial validity determination in subsequent steps.

[0097] In the technical solution of this disclosure, the problem of distinguishing between real obstacles and specular virtual images on highly reflective surfaces is solved by constructing a visual observation consistency description result under specular reflection scenarios. This step utilizes temporal consistency analysis of the geometric features of obstacle images within a continuous time window. By calculating the energy of position change, scale change, and contour continuity fragmentation, and synthesizing a unified visual stability score, the stability of the imaging structure can be effectively characterized. This enables the system to reliably distinguish between real obstacles with stable spatial projection characteristics and unstable virtual images caused by specular reflection, providing an accurate basis for subsequent processing and reducing virtual image interference from the source of perception.

[0098] S2: Based on the visual observation consistency description results and combined with the UAV's own motion information, determine the spatial validity of candidate obstacles, filter out specular reflection virtual images, and output the spatially valid obstacle area.

[0099] Obstacle spatial validity determination results are generated based on observation consistency. To address the issue that visual stability alone may not completely eliminate some stable virtual images, this step combines the UAV's own motion information to perform geometric mapping verification of pixel motion and scale changes on candidate obstacles that have passed the initial screening. Ultimately, it determines whether they correspond to actual 3D space occupancy, thus outputting accurate spatial validity determination results. This is implemented through the following sub-steps.

[0100] S2.1: Initial screening is performed based on multidimensional consistency indicators, and the initial screening judgment score is calculated to select spatially stable candidates.

[0101] Based on the visual observation consistency description results output by S1, an initial screening judgment is first performed based on multidimensional consistency indicators. For example... Figure 2 As shown, by comprehensively evaluating location, scale, and continuous energy, a preliminary screening score is calculated and compared with a threshold to screen out spatially stable candidate obstacles.

[0102] The initial screening does not rely solely on a single threshold for judgment, but rather employs a joint tolerance mechanism to comprehensively assess the multidimensional stability characteristics of obstacles. This mechanism can avoid being affected by temporary fluctuations in a single dimension, such as location energy. Slight vibrations can cause a slightly higher initial screening score, leading to the mistaken exclusion of truly stable obstacles. Therefore, a comprehensive initial screening score G is calculated, using the following formula:

[0103]

[0104] Wherein, G is the initial screening score, and its value ranges from 0 to 1; , , is the energy weighting coefficient, used to adjust the importance of position, scale and continuity unstable energy in the initial screening, with a value range of 0.2 to 8; b is the gating curvature coefficient, with a value range of 2 to 10, used to enhance the advantage of highly stable targets in the score.

[0105] The core of this formula consists of two parts, the first part Suppressing from exponential function , , The larger these three energies are, the smaller this part's value is. Part Two It is a non-linear sigmoid function. When the visual stability score S is high, for example, greater than 0.5, the value of this part rapidly approaches 1; when S is low, the value of this part rapidly approaches 0. This design allows even small differences in S values ​​to be effectively amplified when the S value is close to the empirical threshold, thus avoiding misjudgments caused by S value fluctuations around the threshold. Multiplying the two parts allows the final G score to simultaneously reflect the requirements of low instability energy and high visual stability score. This design improves the robustness of the initial screening by combining direct energy suppression and comprehensive score gating.

[0106] The calculated G score is compared with the preset initial screening threshold. Comparison, initial screening threshold The value range is from 0.25 to 0.80. If If the initial screening is successful, the candidate obstacle passes the preliminary screening and proceeds to the next step of fine geometric verification. It is then classified as a spatially stable obstacle in the preliminary screening, and its G-value is recorded. Otherwise, it is marked as failing the preliminary screening, as it is believed to be likely caused by unstable factors such as virtual images reflected by mirrors, and will not participate in subsequent spatial mapping analysis.

[0107] S2.2: Verify whether the pixel motion and scale changes of the candidate obstacles conform to the camera motion geometry.

[0108] For obstacles that pass the initial screening, the geometric consistency between their pixel motion and the drone's own motion is further verified. Figure 3 A flowchart illustrating the geometric mapping consistency verification and spatial validity determination process provided in this disclosure is shown, as follows: Figure 3 As shown, by calculating the camera motion consistency residual and the scale mapping consistency residual, the imaging pattern is evaluated to determine whether it conforms to the real spatial projection. The pixel motion of a virtual image generated by specular reflection may be independent of the background motion pattern or mismatched with the motion attitude changes of the UAV camera. Understandably, verifying whether the pixel motion and scale changes of candidate obstacles conform to the camera motion geometry can be achieved as follows: For candidate obstacles that pass the initial screening, combining the UAV pose change summary within the time window with the camera imaging geometry parameters, first calculate the reference displacement that should be generated by the UAV motion on the image plane, and then compare the centroid pixel displacement of the candidate obstacle with the reference displacement to obtain the camera motion consistency residual energy; simultaneously, based on the relative distance surrogate from the UAV to the surface or scene of the inspected object, check the consistency of the inverse trend between the change in the outline area of ​​the candidate obstacle and the change in distance to obtain the scale mapping consistency residual energy; finally, fuse the two types of residuals into a geometric mapping consistency score for subsequent spatial validity determination and confidence calculation.

[0109] To achieve this verification, additional data is required: a summary of the UAV's pose changes within the time window, including position and attitude changes, as well as camera imaging geometry parameters.

[0110] First, calculate the residual energy of camera motion consistency. Given the known pose changes of the UAV, the expected pixel displacement of background objects on the image plane can be estimated, using the reference displacement as a reference. Then, the actual pixel displacement of the candidate obstacle between consecutive frames is... Compare this to the reference displacement. The formula is as follows:

[0111]

[0112] Where N is the number of frames in the time window; Let be the pixel displacement magnitude of the obstacle material core in frame t relative to frame t-1; The median of the expected reference displacement magnitude on the image plane within the same time interval is obtained by converting the UAV pose change summary; the instantaneous reliability of its contour data can be directly obtained from the component reflecting the continuity of the contour between adjacent frames used in step S1.3 when calculating the visual stability score. This is characterized by the overlap rate; For the obstacle within the time window The median of the values.

[0113] molecular The denominator measures the deviation between the actual displacement and the expected displacement. Used for normalization, making the error metric relatively independent of the displacement scale. Multiplier It is a weighted term. When the overlap rate of a certain frame... A lower value may indicate that the frame is severely affected by mirror interference. In this case, the weight of this item is reduced, thereby decreasing the impact of abnormal data from that frame on the overall performance. The influence of this enhances the robustness of the algorithm. The smaller the value, the more consistent the pixel motion of the obstacle is with the camera motion.

[0114] Secondly, calculate the scale mapping consistency residual energy. For a real 3D object, under a pinhole camera model, the change in its area in the image should be inversely proportional to the change in distance from the object to the camera. However, the area change of a virtual image often does not conform to this physical law. The formula is as follows:

[0115]

[0116] in, Let be the outline area of ​​the t-th frame; For frame t, the relative distance surrogate from the UAV to the surface of the inspected object or the scene is obtained from the UAV pose change summary. This is a pixel-independent physical quantity that reflects the relative change in distance, such as the normal distance estimated based on prior information about the geometry of the inspected object, or the relative altitude change obtained directly from the UAV navigation system. ln is the natural logarithm.

[0117] It approximates the rate of change in area. This represents the inverse multiplier of the distance change; the value is positive as the distance decreases. For real objects, in a pinhole camera model, the logarithm of the area change multiplier should exhibit a stable negative correlation with the logarithm of the distance change multiplier. The formula calculates the square of the difference between the two. The larger the value, the less consistent the trend of area change with the trend of distance change, and the more likely it is to be a virtual image.

[0118] Will and Merging into a geometrical mapping consistency score F:

[0119]

[0120] in, , The adjustment coefficient ranges from 0.1 to 2.0. The value of F ranges from 0 to 1. The larger the value of F, the more consistent the imaging motion of the obstacle is with the spatial geometric projection law, and the more likely it is to be a real object.

[0121] S2.3: Combine the initial screening score and the geometric consistency score to determine spatial validity and calculate the confidence level.

[0122] Based on the initial screening score G in step S2.1 and the geometric mapping consistency score F in step S2.2, a final spatial validity determination is made for each obstacle that passes the initial screening, and a continuous spatial validity confidence level is calculated.

[0123] First, a spatial validity determination is performed using binarization. To avoid frequent state transitions caused by score fluctuations within a small range around the threshold, a dual-threshold mechanism with hysteresis is adopted. Two thresholds are set: an enable threshold and an enable threshold. and the closing threshold , threshold Values ​​range from 0.45 to 0.85; threshold is off. The value should range from 0.30 to 0.70, and the following requirements must be met. The threshold and The specific value is calibrated and preset within the range based on the typical specular reflection interference intensity of the actual inspection scenario. The judgment rules are as follows:

[0124] If the current frame calculates If so, the outline area corresponding to the obstacle is marked as the effective occupied area;

[0125] like If so, then mark it as a non-spatial valid region;

[0126] like If it is, then its previous marked state will be maintained.

[0127] This hysteresis mechanism improves the stability of state transitions.

[0128] Secondly, a continuously quantified spatial validity confidence score C is calculated to more finely express the credibility of obstacles in subsequent path planning. The formula is as follows:

[0129]

[0130] Where C represents the spatial validity confidence score, ranging from 0 to 1; F represents the geometric mapping consistency score from sub-step S2.2, which quantifies the degree of consistency between obstacle motion and camera motion; and G represents the initial screening score from sub-step S2.1, which integrates the visual stability score S and various energy indicators. , , This reflects the temporal stability of the obstacle's performance; q and r are shape parameters, ranging from 2 to 12, used to enhance the discrimination of high-scoring targets.

[0131] The formula performs nonlinear compression on F and G separately, and then multiplies the results. This means that to obtain a high confidence level C, both a high geometrical mapping consistency score F and a high initial screening stability score G are required. This design effectively suppresses specular image remnants that might have passed a certain test by chance. For example, an imaging structure might coincidentally move in unison with the background (i.e., a high F), but its visual performance is highly unstable (i.e., a low G). In this case, its confidence level C will be lowered, thus requiring more caution in subsequent decisions.

[0132] The final output consists of two sets of regions: a set of spatially valid obstacle regions, which includes the boundary, confidence level C, and validity label of each region, and a set of non-spatially valid regions.

[0133] S2.4: Encapsulate and output the spatial validity determination results, and identify the valid obstacles and reflection interference areas.

[0134] The output of step S2.3 is organized and packaged to form a standardized obstacle space validity determination result for use in step S3.

[0135] Each region in the set of valid obstacle regions is archived according to its associated obstacle number. For each valid obstacle, the boundary information of its valid occupied area is output, such as the outline polygon, the calculated spatial validity confidence C, and its current validity status label.

[0136] Simultaneously, the set of non-spatial effective regions is output as an independent candidate set of reflection interference regions. In the subsequent environmental state construction, the regions in this set only serve as risk warning information regarding the presence of specular reflection interference in the environment and do not participate in the actual obstacle avoidance distance calculation or spatial occupancy constraints.

[0137] To ensure data interface consistency, the final output judgment results adopt a unified field structure, including, for example, obstacle number, valid area boundary, validity confidence level, and validity status. This encapsulation clearly defines the reliable sources of obstacle information in subsequent path planning steps, namely, information from only the valid obstacle area in space, thereby completely eliminating the direct interference of specular reflection images on the decision-making system at the data flow entry point.

[0138] In the technical solution of this disclosure, obstacle space validity determination results are generated based on observation consistency, and preliminary stable candidate obstacles are subjected to deeper geometric verification. This step analyzes whether the pixel motion and scale changes of candidate obstacles conform to the true three-dimensional spatial projection law by combining the UAV's own motion information, and uses a hysteresis-based dual-threshold mechanism for final determination. This can further filter out those visually temporarily stable but spatially geometrically abnormal mirror images, thereby accurately outputting only the spatially valid obstacle area corresponding to the actual physical location and its confidence level, cutting off the path of virtual image information flowing into the subsequent decision-making process.

[0139] S3: Based on the effective obstacle area in the space, the UAV's own state and the inspection mission state are integrated to construct an environmental state description for reinforcement learning decision-making.

[0140] Understandably, the UAV's own motion information is used to support the geometric mapping verification in the spatial validity determination stage, emphasizing the pose change summary within the time window and its derivation of the image plane reference displacement and distance proxy; the UAV's own state is used to construct the environmental state vector, specifically including three-dimensional spatial position coordinates, heading angle, three-dimensional velocity components, and the remaining percentage of velocity and acceleration relative to the maximum allowable value.

[0141] Constructing a reinforcement learning environment state description based on effective spatial obstacles. To provide reliable and interference-resistant input information to the reinforcement learning decision model, this step fuses and quantifies the determined effective spatial obstacle information, the UAV's own state, and the inspection mission objective to construct a standardized environment state vector, thereby laying a clean decision foundation for subsequent path planning. This is specifically achieved through the following sub-steps.

[0142] S3.1: Extract the threat characteristics of local effective obstacles and calculate their comprehensive risk intensity.

[0143] From the spatial effectiveness assessment results, extract the information of effective obstacles located within the UAV's local perception space. Establish a safety buffer zone for each obstacle, calculate its relative distance and azimuth angle with the UAV, and integrate confidence level, distance, and velocity information to calculate its risk intensity.

[0144] The system acquires the UAV's real-time 3D spatial position coordinates, velocity vector, and the sequence of current and subsequent inspection target points provided by the mission planning module. Based on this information, the system defines a local perception spherical space centered on the UAV's current position. The radius of this space is called the local planning radius, denoted as . Its value typically ranges from 5 meters to 60 meters, and the specific value can be dynamically adjusted based on the drone's current flight speed and the average density of obstacles sensed in historical frames.

[0145] Subsequently, the system retrieves only the obstacle regions marked as spatially valid in step S2 within this local perception space. For each valid obstacle falling within this space, the system first establishes a safety buffer. This buffer characterizes the area occupied by the obstacle in three-dimensional space. Its construction combines the obstacle's two-dimensional image contour, camera geometry, and conservative distance assumptions, and then uniformly expands outward by a safety radius. And thus formed, The value ranges from 0.5 meters to 5 meters. This is intended to provide tolerance for flight control errors, positioning errors, and the geometric uncertainties of the obstacles themselves.

[0146] For each valid obstacle i, calculate the shortest Euclidean distance in three-dimensional space from the current position of the UAV to the boundary of its safe buffer zone, denoted as . Simultaneously, the direction vector from the UAV to the nearest point is calculated, and this vector is projected onto the UAV's horizontal reference plane to obtain the horizontal azimuth angle relative to the UAV's nose direction, denoted as . .

[0147] To comprehensively characterize the threat level of each effective obstacle, the system defines a composite index to represent the risk intensity. The calculation of this indicator incorporates three main elements:

[0148] First, the confidence level of the spatial effectiveness of the obstacle. This value comes from the confidence output of the corresponding obstacle in step S2. The higher the confidence, the stronger the certainty that it is a real obstacle.

[0149] Second, the shortest distance between the drone and the safety buffer zone boundary of the obstacle. The closer the distance, the greater the threat.

[0150] Thirdly, the drone's current flight speed, v, is crucial. Higher speeds mean shorter reaction times for evasive maneuvers, thus increasing the risk. Risk intensity. The specific calculation formula is as follows:

[0151]

[0152] Where m is the confidence amplification index, which is used to adjust the influence of high-confidence obstacles, and its value ranges from 5 to 6; The speed reference value, ranging from 15 to 20 meters per second, is used to normalize the speed term; n is the speed risk index, ranging from 1 to 4, used to control the amplification effect of speed on risk intensity. Through this calculation, each effective obstacle is assigned a dynamic, non-negative risk intensity value, which accurately quantifies its overall threat level to the drone's flight safety at the current moment.

[0153] S3.2: Integrate the UAV itself, inspection mission, and surrounding threat status to construct a standardized state vector.

[0154] The state of the UAV itself, the state of the inspection mission, and the state of surrounding threats are integrated from multiple dimensions to construct a standardized, fixed-dimensional state vector. Each component is then normalized to facilitate efficient processing by the subsequent reinforcement learning strategy model. The construction of the state vector mainly covers three aspects.

[0155] The first aspect is the flight state, which describes the UAV's own motion attitude and capability boundaries. This includes the UAV's position coordinates in the horizontal plane, its current heading angle, its velocity components in three dimensions, and the remaining percentage of its current velocity and acceleration relative to the maximum permissible values ​​of its physical actuators. This information collectively defines the UAV's current kinematic state and controllable potential.

[0156] The second aspect is the inspection mission status, used to guide the drone towards the mission target. Its core is calculating a continuous target proximity index. This metric also considers the straight-line distance between the drone and the current primary inspection target point. And the horizontal angle between the drone's current heading and the direction it is pointing towards the target point. The closer the drone gets to the target and the more its flight path aligns with the target, the closer this indicator value approaches 1. Its mathematical expression is:

[0157]

[0158] in, For the target approximation scale, it is used to control the attenuation rate of the distance effect, with a value range of 65 to 80 meters; The angle reference value is used to normalize the impact of heading deviation, with a value ranging from 0.8 to 1.2 radians; p is the angle penalty index, used to adjust the sensitivity of heading deviation, with a value ranging from 5 to 6. In addition, the mission status also includes the priority weight of the current target point and the number of remaining unvisited target points.

[0159] The third aspect is the surrounding threat state, which is directly derived from the risk intensity, relative distance, and relative azimuth angle of each effective obstacle calculated in sub-step S3.1. To ensure that the dimension of the state vector input to the policy network remains constant, the system sets a fixed number K, whose value ranges from 8 to 12. From all effective obstacles within the current local perception space, the risk intensity is selected... The top K highest-risk obstacles. For each selected obstacle, determine its risk intensity. Relative distance and relative azimuth These three data points are incorporated into the state vector. If the total number of currently valid obstacles is less than K, the empty positions are filled with zero values ​​to maintain a constant vector dimension.

[0160] After completing the acquisition and calculation of the raw data, all state components need to be standardized and preprocessed. For example, all distance values ​​are divided by the maximum sensing range. Normalized to the interval between 0 and 1; all angle values ​​are uniformly mapped to - To + Within the specified range; divide the speed value by the maximum permissible speed of the drone. This standardization process helps improve the stability and convergence speed of reinforcement learning model training.

[0161] S3.3: Define the environment state description object and the incremental action space, and establish the policy interaction interface.

[0162] The integrated state vector is encapsulated as an environment state description object, and the action space of the reinforcement learning agent is defined as an incremental control mode, including incremental commands for horizontal speed, heading angle and climb rate.

[0163] The multidimensional state vector constructed and standardized in step S3.2 is encapsulated into a complete environment state description object, denoted as . Where the subscript k represents the index of the discretized decision time. This state object is defined at fixed time intervals. Update and provide to the policy network. The decision step size, ranging from 0.05 seconds to 0.15 seconds, ensures the real-time nature of the decision.

[0164] The policy network receives the state Subsequently, its output is defined as action commands used to control the drone's flight. In this embodiment, the action space of the reinforcement learning agent is defined as the incremental control mode action space, that is, the actions output by the policy network are incremental commands relative to the current drone state, thereby directly connecting to the underlying flight controller and ensuring control smoothness. Specifically, the incremental control mode action space outputs a three-dimensional action increment vector, including: horizontal velocity increment. Its permissible range is between -2 and +2 meters per second; heading angle increment Its permissible range is between -0.6 and +0.6 radians; vertical climb rate increment Its permissible range is between -1.5 and +1.5 meters per second.

[0165] At each decision-making moment, the system superimposes the action increment output by the policy network with the actual executed action after smoothing and limiting processing from the previous moment, thereby obtaining the desired speed, heading, and climb rate commands for the current moment. This action definition method ensures both the direct executableness of the policy output and the smoothness of action changes through incremental control, meeting the requirements of engineering practice.

[0166] S3.4: Update and smooth the environment state with the decision step size as the period to generate a continuous time-series state sequence.

[0167] The environment state is continuously updated with the decision step size as the cycle, and key state indicators are smoothed over time to generate a continuous and stable sequence of environment states for online use by reinforcement learning strategies.

[0168] The system uses decision step size The system continuously drives the entire process from visual perception, consistency analysis, spatial validity determination to state feature extraction and integration, in a cyclical manner. This means that whenever a new frame of image sensor data and UAV state data is acquired, the system re-executes all calculations starting from step S1, thereby ensuring the environmental state description at each moment. They are all built upon the latest and most accurate sensory information.

[0169] However, raw perception data is inevitably affected by noise, especially in scenarios with strong specular reflection, which can lower the confidence level of certain obstacles deemed valid. Geometric features may also exhibit instantaneous fluctuations. To prevent these high-frequency jitters from directly causing drastic changes in the state vector, which in turn lead to instability or even oscillations in the policy output commands, it is necessary to perform smoothing filtering on key continuity indicators in the state vector.

[0170] The system specifically applies a first-order low-pass filter to the comprehensive collision risk index R and the target proximity index g. For example, it uses an exponential moving average for smoothing.

[0171]

[0172] in, These are the original calculated values. The smoothed value. This is a smoothing coefficient, with a value ranging from 0.1 to 0.12. By selecting an appropriate... The value can achieve a good balance between rapid response to changes in the real environment and effective filtering of instantaneous measurement noise.

[0173] This series of environmental state descriptions is generated chronologically and has undergone necessary smoothing. This constitutes a continuous sequence of environment states upon which the reinforcement learning policy relies for online decision-making. At each decision time k, the policy model is based on the latest state in this sequence. It generates corresponding flight action commands, enabling the drone to achieve safe, smooth, and efficient obstacle avoidance and inspection in a real environment full of uncertainty and mirror interference.

[0174] In the technical solution of this disclosure, a reinforcement learning environment state description based on effective spatial obstacles is constructed, transforming verified real obstacle information, UAV state, and mission objectives into structured and quantified state vectors. This step extracts local threat features of effective obstacles and calculates risk intensity, integrates multi-dimensional state components, and defines a policy interaction interface, thus constructing a clean, reliable, and directly usable environment state representation for the reinforcement learning model. This ensures that the input information of the decision-making model is free from the interference of specular reflections, enabling the model to make judgments based on real environmental threats, laying the foundation for safe decision-making.

[0175] S4: Based on the environmental state description, generate flight control actions online using a reinforcement learning strategy model, and output a dynamic obstacle avoidance and inspection path through rolling planning and safety verification.

[0176] This paper describes a dynamic obstacle avoidance and inspection path generated based on reinforcement learning, which is resistant to specular reflection interference. To achieve safe and continuous autonomous inspection in environments with strong reflection interference, this step utilizes a pre-trained reinforcement learning policy model to generate flight action commands online based on a clean environment. Through rolling planning and safety verification, the final output is a dynamic obstacle avoidance and inspection path that relies solely on real obstacle information. This is implemented through the following sub-steps.

[0177] S4.1: Configure online decision execution parameters, including decision step size, physical constraints, and smoothing coefficient.

[0178] Before initiating online flight decision-making, the strategy execution environment must be configured, and constraint parameters matching the UAV's physical characteristics and safety requirements must be set. First, obtain the real-time environmental state description set output from step S3, as well as the inherent dynamics and safety constraint parameters of the UAV platform.

[0179] Key configuration parameters include: decision time step Its value range is consistent with the definition in step S3.3, ranging from 0.05 seconds to 0.15 seconds, which determines the response frequency of the strategy. The rolling planning time domain is set, i.e., the number of steps the strategy model simulates and predicts forward for each decision, denoted as H, with a value range of 25 to 30. The total prediction time domain length is then... Seconds. This prediction time domain allows the policy to look ahead to a certain extent to assess the consequences of short-term action sequences. Understandably, rolling planning is an online short-view planning approach with decision step size as its rhythm: the system reads the latest environmental state description in each decision cycle, first generating flight actions online using a reinforcement learning policy model, then generating a short-term rolling prediction path segment within the set prediction time domain, and performing safety margin checks and comprehensive cost assessments on this path segment; subsequently, only the executable actions that pass the verification in the current cycle and their corresponding actual flight trajectory segments are executed, and this segment is spliced ​​into the global path record; the next cycle repeats the above process based on the updated environmental state, achieving continuous path correction in the dynamic environment. All the above online decision-making processes are based on the incremental control mode action space, ensuring that each generated action is a smooth incremental instruction relative to the current state.

[0180] At the same time, hard physical limits are set for drone flight, including maximum flight speed. The value ranges from 3 to 5 meters per second; maximum acceleration The value ranges from 5 to 8 meters per second squared; the maximum rate of change of heading angle. The value ranges from 1.2 to 1.5 radians per second; and the maximum rate of ascent. The values ​​range from 0.5 to 0.8 meters per second. These limits ensure that any generated motion commands are within the capabilities of the drone platform.

[0181] In addition, to avoid high-frequency jitter in the policy output actions caused by sensor noise or small fluctuations in state estimation, an action smoothing coefficient is introduced. Its value ranges from 0.5 to 0.7. This coefficient is used to perform first-order low-pass filtering on the action commands output in the continuous decision cycle, making the control output smoother and improving flight comfort and stability. After completing the above parameter configuration, the system enters the online decision execution loop.

[0182] S4.2: Generate a single-step action based on the current environment state and perform a safety margin check to ensure executability.

[0183] In each decision cycle, the current environmental state is input into the reinforcement learning policy network to generate flight action commands. After smoothing and limiting, executability verification based on single-step safety margin is performed to ensure action safety.

[0184] The system reads the latest state description from the environmental state set maintained in step S3.4. This state This includes the drone's own status, target proximity, and threat characteristics of the top K most dangerous effective obstacles. Then, Input to the deployed reinforcement learning policy network ,in, These are the network parameters. After forward computation, the policy network outputs a raw action vector. This action vector corresponds to the incremental control mode action space defined in step S3.3, which contains the original command with three dimensions: speed increment, heading angle increment, and climb rate increment.

[0185] For the original action Smoothing and limiting processes are performed to obtain the actual action to be executed. Smoothing utilizes actions already performed in the previous cycle. and smoothing coefficient The calculation formula is:

[0186]

[0187] Where clip is the component limiting function. and These represent the lower and upper limits for each action dimension. This step suppresses high-frequency jitter and strictly limits the action to a preset increment range.

[0188] To ensure the absolute safety of an action, executability verification is necessary. The core of this verification is calculating the next decision step size after executing the action. Within this area, the question is whether the drone will intrude into the safety buffer zone of any effective spatial obstacles. To this end, a safety margin called the single-step safety margin is calculated. Indicators:

[0189]

[0190] in, Based on the current state and the action to be performed Predicted, in The shortest distance from the UAV to the boundary of the i-th effective obstacle safety buffer zone (defined in step S3.1); The safe radius is represented by v; v represents the current speed of the drone. This represents the maximum acceleration. The last two terms of the formula constitute a conservative buffer, considering the maximum additional displacement that may result from velocity and control errors within a control cycle. The last term... This is a simplified form of the kinematics formula for uniform deceleration, corresponding to the UAV decelerating at its maximum speed. At the current decision-making pace The maximum braking distance that may occur within the range; the coefficient 0.5 is derived from the displacement formula for uniformly accelerated motion. In A constant. Only when Only then is the action considered. It is safe.

[0191] If action Failed to pass the safety margin check, i.e. The system will not execute the action directly. Instead, within the allowed action range, the system will perform the action based on the principle of minimum deviation. Adjustments may be made, such as proportionally reducing the speed increment or adjusting the course, until the new motion prediction meets the requirements. This mechanism ensures that even if the policy network outputs a dangerous action for some reason, the underlying security layer can correct it, fundamentally eliminating the risk of collision caused by planning errors or perception interference that has not been completely filtered out, such as the extremely rare case of virtual image misjudgment.

[0192] S4.3: Generate a rolling prediction path segment based on the verified actions and perform a comprehensive cost evaluation.

[0193] Once action By verifying and actually executing the action, a short-term rolling prediction path segment will be generated within the configured rolling time domain H steps based on the action and the UAV kinematic model, and the overall performance of the rolling path segment will be evaluated.

[0194] Using a simplified kinematic model of the drone, such as a point mass model or a model considering dynamics, starting from the current state, and... As the first step, and assuming that the policy can maintain a certain policy or continue with simple actions in subsequent prediction steps, a predicted state trajectory for the next H steps is generated. This trajectory consists of a series of discrete prediction points, each containing information such as position and velocity, forming a trajectory from the current moment to the future. The rolling path segment at time point. A comprehensive evaluation is performed on this predicted rolling path segment to calculate a comprehensive cost. This cost function balances three core objectives: obstacle avoidance safety, mission progress efficiency, and trajectory smoothness.

[0195]

[0196] in, This represents the safety margin of the h-th predicted point on the path segment. This is the distance from the predicted point to the current inspection target point; For the predicted h-th step, Can be regarded as ; , , These are the weighting coefficients for risk cost, target distance cost, and action change cost, respectively, with values ​​ranging from 0.1 to 50; p and q are exponential coefficients, with values ​​ranging from 1 to 6. The reference value for motion change ranges from 0.2 to 3. Among them, the risk cost is used to penalize prediction points with too small a safety margin to force the path away from effective obstacles; the target distance cost is used to encourage the path to move closer to the inspection target point to ensure coverage advancement; and the motion change cost is used to suppress drastic fluctuations in control commands of adjacent prediction steps to ensure smooth and executable trajectory.

[0197] The significance of this cost function lies in the following: the first term strongly penalizes points with low safety margins, forcing the path away from obstacles; the second term encourages the path to move closer to the target point; and the third term penalizes drastic changes in action between adjacent steps to promote the generation of a smooth flight trajectory. During online execution, although... Primarily used for optimizing strategy parameters during the offline training phase. However, its computational principles ensure that online-generated path segments naturally tend to have characteristics of low risk, high target proximity, and high smoothness. The system monitors whether the minimum safety margin of the path segment remains greater than zero and whether the target distance shows a decreasing trend, serving as a real-time reference for flight health. Furthermore, the comprehensive cost calculated online... The cost itself is used to quantify the overall performance of the current rolling path segment: the lower the cost, the better the path is in terms of obstacle avoidance safety, task efficiency, and motion smoothness. The system can use this cost as a real-time feedback signal to evaluate the quality of strategy decisions, trigger anomaly protection mechanisms, or as a basis for online learning and adaptive adjustment.

[0198] S4.4: Scroll and stitch together the trajectory segments of safe execution to output the final dynamic obstacle avoidance and inspection path.

[0199] By continuously splicing together the short path segments or the parts that have flown, which have passed safety verification and have been actually executed, generated in each decision cycle, a complete and continuous dynamic obstacle avoidance and inspection path is formed from the starting point to the end of the mission.

[0200] During flight, the system uses decision steps. To maintain rhythm, steps S4.2 and S4.3 are repeatedly executed. After each execution, the actual flight trajectory segment of the current cycle is added to the global path record, typically corresponding to... Displacement within time. At the junction of segments, due to the constraints of smooth motion and kinematic continuity, the path is naturally guaranteed to be geometrically continuous, including position and tangent direction.

[0201] Importantly, throughout the entire path generation and stitching process, the system consistently and exclusively uses the spatially valid obstacle region determined and transmitted in step S2 as the sole source of environmental occupancy constraints. For the non-spatially valid regions marked in step S2, i.e., the reflection interference candidate set, their information is only retained as a hint or log of environmental reflection intensity, and does not generate any repulsive or constraining force in any path planning or safety verification stage. This mechanism ensures that the mirror image cannot have any substantial impact on the final generated flight path.

[0202] The final output complete inspection path is a sequence of timestamped three-dimensional spatial coordinates, velocity, and heading information. This path meets the following core requirements: throughout the entire process, the minimum instantaneous safety margin calculated based on effective obstacles is always greater than zero; the drone's velocity, acceleration, and other state variables never exceed the configured physical limits; all preset inspection target points are visited in priority order, or, when unreachable, the maximum coverage is achieved. This path is the safe, reliable, and executable dynamic obstacle avoidance inspection path generated by this invention, capable of resisting strong specular reflection interference.

[0203] In the technical solution of this disclosure, a dynamic obstacle avoidance and inspection path resistant to specular reflection interference is generated based on reinforcement learning. A trained strategy model is used to generate flight actions online based on reliable environmental conditions. This step achieves online safe planning and execution by performing single-step action safety verification, rolling path segment generation, and comprehensive cost evaluation, while excluding non-spatial effective areas from constraints throughout the process. The final output flight path relies entirely on real obstacle information, thus being immune to the interference of specular images and ensuring the continuity, safety, and mission execution efficiency of the UAV's dynamic obstacle avoidance in strong reflection inspection scenarios.

[0204] According to embodiments of this disclosure, an electronic device is also provided, which may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, the communications interface, and the memory communicate with each other via the communication bus. The processor can invoke logical instructions stored in the memory to execute the methods provided in the above embodiments.

[0205] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0206] On the other hand, this disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0207] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0208] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0209] It should be understood that the above embodiments are only used to illustrate the technical solutions of this disclosure, and not to limit them; although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for dynamic obstacle avoidance and inspection path planning for unmanned aerial vehicles (UAVs) based on reinforcement learning, characterized in that, The method includes: S1. Perform temporal consistency analysis on the visible light image sequence acquired within a continuous time window to construct a visual observation consistency description result, which is used to characterize the stability of the candidate obstacle imaging structure. S2. Based on the visual observation consistency description results and combined with the UAV's own motion information, determine the spatial effectiveness of candidate obstacles, filter out specular reflection virtual images and output the spatially effective obstacle area. S3. Based on the effective obstacle area in the space, the UAV's own state and the inspection task state are integrated to construct an environmental state description for reinforcement learning decision-making. S4. Based on the environmental state description, generate flight control actions online using a reinforcement learning strategy model, and output a dynamic obstacle avoidance and inspection path through rolling planning and safety verification. S1 includes: Acquire a sequence of visible light images within a continuous time window and perform preprocessing; Extract and temporally correlate the contours of candidate obstacles from the preprocessed image sequence, and calculate the basic geometric features of each candidate obstacle in each frame within the time window; Based on the aforementioned fundamental geometric features, the position change energy, scale change energy, and contour continuity and fragmentation penalty energy of each candidate obstacle are calculated and synthesized into a visual stability score. Energy of position change As shown below: in, , Let be the pixel coordinates of the centroid of the obstacle's outline in frame t. , The pixel coordinates of the centroid of the obstacle's outline in frame t-1; As a scale reference quantity, The length of the time window; Scale change energy As shown below: in, Let the outline area be the area of ​​the t-th frame and the (t-1)-th frame. The perimeter of the contour in frame t and frame (t-1); contour continuity and fragmentation penalty energy. As shown below: in, Let be the overlap rate of the obstacle contour filling region between frame t and frame t-1. The number of connected components of the obstacle's outline in frame t; The visual stability score S is shown below: in, , , These are the weighting coefficients; Based on the visual stability score, a visual observation consistency description result is generated for each candidate obstacle; S2 includes: Based on the visual observation consistency description results, the initial screening score is calculated and candidate obstacles that are initially stable are selected. For candidate obstacles that pass the initial screening, verify whether their pixel motion and scale changes conform to the camera motion geometry based on the UAV pose change estimation, and calculate the geometric mapping consistency score. By combining the initial screening score and the geometric mapping consistency score, spatial validity is determined, and the spatial validity confidence level is calculated. Output the spatial validity determination results, which include the region of effective obstacles in the space and their confidence levels. The spatial validity determination adopts a hysteresis-based dual-threshold mechanism, specifically as follows: If the geometric mapping consistency score If so, it is marked as a valid occupied area; like If so, it is marked as a non-valid spatial region; like If so, the previous marked state will be maintained; in, To enable the threshold, To close the threshold, and .

2. The method for dynamic obstacle avoidance and inspection path planning of unmanned aerial vehicles based on reinforcement learning according to claim 1, characterized in that, S3 includes: Extract effective obstacles located within the local perception space of the UAV, and calculate the risk intensity of each effective obstacle relative to the UAV. The risk intensity integrates the spatial effectiveness confidence of the obstacle, the relative distance between the UAV and the obstacle, and the speed of the UAV. The drone's own status, inspection mission status, and surrounding threat status based on the risk intensity are integrated to construct a standardized state vector; The state vector is encapsulated as an environment state description object, and the incremental control mode action space of the reinforcement learning agent is defined.

3. The method for dynamic obstacle avoidance and inspection path planning of unmanned aerial vehicles based on reinforcement learning according to claim 1, characterized in that, S4 includes: In each decision cycle, the current environmental state description is input into the reinforcement learning policy network, and flight action commands are generated based on the incremental control mode action space. A single-step safety margin check is performed on the flight maneuver command, and the safety margin check is based on the safety buffer zone of the effective obstacle area in the space; Perform the actions that pass the verification and generate the predicted path segment in the rolling time domain; The trajectory segments executed in each cycle are spliced ​​together to form a complete dynamic obstacle avoidance and inspection path.

4. The method for dynamic obstacle avoidance and inspection path planning of unmanned aerial vehicles based on reinforcement learning according to claim 3, characterized in that, The single-step safety margin check is as follows: Calculate and verify the single-step safety margin Is it greater than zero, where The calculation formula is: in, To predict the shortest distance from the UAV to the boundary of the i-th effective obstacle safety buffer zone after performing the action, Where v is the safety radius and v is the current speed of the drone. For decision step size, This is the maximum acceleration.

5. The method for dynamic obstacle avoidance and inspection path planning of unmanned aerial vehicles based on reinforcement learning according to claim 3, characterized in that, The generation of the predicted path segment in the rolling time domain includes: Based on the UAV kinematic model and flight action commands that have passed safety margin verification, a predicted state trajectory is generated as a rolling path segment within a preset number of rolling planning steps. Calculate the comprehensive cost of the rolling path segment, which is a weighted sum of three factors: risk cost representing obstacle avoidance safety, target distance cost representing task advancement efficiency, and motion change cost representing trajectory smoothness. The overall performance of the current rolling path segment is evaluated based on the comprehensive cost, and this performance is used as a real-time basis for flight health monitoring, decision quality assessment, or online strategy adjustment.

6. An electronic device, characterized in that, The electronic device includes a memory and at least one processor, the memory storing a computer program, and the processor executing the computer program to implement the reinforcement learning-based UAV dynamic obstacle avoidance and inspection path planning method as described in any one of claims 1-5.

7. A computer storage medium, characterized in that, It stores a computer program, which, when executed, implements the UAV dynamic obstacle avoidance and inspection path planning method based on reinforcement learning according to any one of claims 1-5.

Images