Airplane skin autonomous inspection method and system based on double-layer deep reinforcement learning

Abstract

The application discloses an airplane skin autonomous inspection method and system based on double-layer deep reinforcement learning, comprising the following steps: S1, acquiring an initial airplane skin surface image and performing pretreatment; S2, obtaining an equidistance scattered grid map; S3, constructing a defect detection intelligent agent and training; performing single-grid airplane skin defect detection and uncertainty evaluation; S4, calculating the overall defect uncertainty value of the current grid, updating the uncertainty heat map and the defect recognition confidence heat map, and updating the grid coverage flag; S5, constructing a coverage planning intelligent agent; judging whether the termination condition is reached, if yes, outputting the inspection result, if not, using the trained coverage planning intelligent agent to optimize and control the multi-dimensional pose and focal length of the camera, and turning to step S3 for a new round of inspection; the application relies on the double-intelligent-agent cooperative architecture and the uncertainty heat map technology, realizes the parallel linkage of path planning, automatic focusing and defect detection.

Description

Technical Field

[0001] This invention relates to the field of aircraft skin inspection technology, and more specifically, to an autonomous aircraft skin inspection method and system based on two-layer deep reinforcement learning. Background Technology

[0002] As the core external structure of an aircraft, the aircraft skin directly withstands the impact of airflow, temperature changes, and external forces during flight. After long-term service, it is highly susceptible to various defects such as cracks, surface scratches, localized corrosion, coating peeling, and loose rivets. If such damage is not detected and addressed in a timely manner, it will continue to expand and seriously threaten the overall structural strength and flight safety of the aircraft. For a long time, the aviation industry has mainly relied on ground staff to carry out traditional manual inspections: personnel need to work with auxiliary equipment such as ladders and aerial ladders to visually inspect the skin condition area by area at close range, and some minor defects also require the use of auxiliary tools for judgment. This purely manual inspection mode is time-consuming and has low overall work efficiency, making it difficult to meet the maintenance needs of large-scale, high-frequency inspections of the fleet; on the other hand, the inspection results are highly dependent on the experience, focus, and subjective judgment of the staff, and the accuracy and consistency of the inspection results cannot be guaranteed.

[0003] To address the various problems associated with traditional manual inspections, the aviation operations and maintenance industry is currently vigorously promoting the research, development, and application of robotic automated inspection technologies. At present, mainstream automated inspection platforms encompass various types of intelligent equipment, including inspection drones, intelligent inspection vehicles, and wall-climbing robots. These robots can be equipped with high-definition vision cameras, LiDAR, 3D scanning sensors, and other sensing devices, relying on machine vision, laser measurement, and intelligent recognition technologies to automatically complete full-coverage scanning of the fuselage skin, image acquisition, data transmission, and preliminary defect assessment. However, current robotic inspection solutions still face many practical challenges and are difficult to adapt to airport on-site operational scenarios.

[0004] Specifically, current robot-based inspection solutions have the following limitations:

[0005] (1) Poor adaptability to the on-site environment and low on-site operation efficiency:

[0006] Airport aprons are high-dynamic operating environments with frequent personnel movement and continuous shuttle of various ground support vehicles. Intelligent inspection vehicles need to continuously identify and avoid dynamic obstacles during their movement, and this obstacle avoidance process consumes a significant amount of time. Meanwhile, aircraft inspections are concentrated in short intervals between flight turns, resulting in very limited effective operating windows. This further compresses equipment operation and inspection time, leading to low efficiency of the entire inspection process and making it difficult to meet the high-frequency, fast-paced operational needs of airports.

[0007] (2) Airspace and safety regulations:

[0008] The operation of drones is strictly constrained by civil aviation regulations, airport airspace control, and aviation safety management systems. For overall operational safety reasons, the vast majority of airports in China explicitly prohibit drones from entering their premises for operations. Due to these restrictions, the application of drone inspection models has an extremely high barrier to entry, preventing large-scale application in frontline civil aviation scenarios.

[0009] (3) There are risks associated with close-range operations:

[0010] Intelligent inspection vehicles, wall-climbing robots, and other equipment all need to be close to the aircraft fuselage to carry out inspection operations. During the movement and positioning of these devices, accidental collisions and scrapes are very likely to occur, which could damage the aircraft's skin. In addition, the movement and parking of these devices on the tarmac will occupy the already congested work area, disrupt ground crew work routes, and adversely affect the airport's normal support operations.

[0011] Currently, remote operation can be achieved with the help of telephoto autofocus lenses, eliminating the need for equipment to be close to the machine body. This fundamentally avoids the safety and scenario limitations of traditional robot inspections, such as on-site collisions, airspace control, and interference from personnel movement. However, long-range inspection with long focal lengths still faces several technical challenges, requiring solutions to the following issues: First, it is difficult to balance scanning accuracy and inspection efficiency. While short focal lengths offer a wide field of view and fast scanning speed, their insufficient resolution makes it impossible to identify minute defects in the skin. Increasing the focal length to improve imaging accuracy reduces the area covered by the camera in a single scan, leading to a decrease in overall scanning efficiency. Second, traditional defect detection algorithms employ a two-stage architecture of scanning and sampling followed by target detection. This requires full-area image acquisition before unified analysis, resulting in large data volumes, processing delays, and slow computation, making it difficult to meet the requirements of rapid inspection. Third, existing inspection planning relies solely on geometric information such as point cloud models and grid coverage to design routes, representing an open-loop architecture that separates planning and detection. When facing difficult-to-identify areas such as shadows, reflections, and minute defects, proactive re-inspection and supplementary testing are not possible, failing to form a closed-loop mode of detection-feedback-re-inspection, resulting in poor detection performance in difficult areas. Summary of the Invention

[0012] The purpose of this invention is to propose an autonomous inspection method and system for aircraft skin based on two-layer deep reinforcement learning, aiming to solve problems such as the difficulty in balancing accuracy and efficiency in long-range inspection with long focal length, slow operation of traditional serial inspection process, and the separation between inspection planning and defect detection.

[0013] Specifically, this invention provides an autonomous inspection method for aircraft skin based on two-layer deep reinforcement learning, comprising the following steps:

[0014] S1. Obtain an initial image of the aircraft skin surface and preprocess the image;

[0015] S2, Based on the acquired preprocessed image The aircraft skin is segmented to obtain a discrete grid map with equal spacing. Construct and initialize the uncertainty heatmap for the corresponding grid. Compare the defect identification confidence heatmap C with the actual defect identification confidence level, and determine the starting point of the inspection equipment's operation.

[0016] S3. Construct a defect detection intelligent agent The agent is trained, and the Actor policy weights are saved after training to obtain the trained defect detection agent. ; Utilizing the trained defect detection agent Conduct single-grid aircraft skin defect detection and uncertainty assessment.

[0017] S4. Based on the complete defect information obtained in step S3, calculate the overall defect uncertainty value of the current grid and update the uncertainty heatmap. Compare the defect identification confidence heatmap C with the grid coverage flags and update the grid coverage flags. ;

[0018] S5, Constructing a Coverage Planning Intelligent Agent And train it to obtain the trained coverage planning agent. Determine whether all grids are covered and the overall defect uncertainty of all grids cannot be reduced. If yes, output the inspection result; otherwise, use the trained coverage planning agent. After optimizing and adjusting the camera's pose and focal length in multiple dimensions, proceed to step S3 for a new round of inspection.

[0019] Furthermore, the defect detection intelligent agent The input observation state space is expressed as ;in, The current angle of the gimbal represents the horizontal position. This represents the current tilt angle of the gimbal. The feature vector after image compression is obtained by preprocessing the image. The input is a pre-trained lightweight ResNet18 visual feature extraction network, which outputs a 512-dimensional normalized feature vector as an image observation representation; a defect detection agent... The action space is continuous control output In the formula, This represents the increment of a single horizontal rotation of the gimbal. This represents the increment of a single pitch rotation of the gimbal. The maximum single rotation angle specified by the hardware; when the defect detection agent... After completing the posture adjustment, the pre-trained YOLO model is called to perform multi-class defect detection, and the coordinate position, confidence score and category of each defect target are output.

[0020] Furthermore, defect detection intelligent agents reward function A composite segmented reward system is adopted, expressed as follows: ;in, , These represent the weighting coefficients for the main reward and the stable reward, respectively. The main reward represents the confidence score of object detection. The average of the confidence scores of all object detections in the image is used as the quantitative score of image quality to improve the final image quality. The reward for stable pose convergence is used to incentivize the defect detection agent to converge to the optimal observation pose. If the amplitude of two consecutive actions is less than a threshold, the reward is applied. Then a fixed positive reward is assigned, which can be expressed mathematically as:

[0021] (1);

[0022] In the formula: These are manually adjusted coefficients used to balance the influence weights of the two rewards; , For defect detection intelligent agents Continuous control output at steps t and t-1.

[0023] Furthermore, the method for calculating the overall defect uncertainty value of the current grid based on obtaining all defect information within the current grid in step S3 is as follows:

[0024] Defect detection intelligent agent Adjust the camera tilt angle Deflection angle The camera pose is continuously adjusted iteratively as an action variable until the camera viewpoint and detection results converge, thus forming a defect detection agent. After the observation perspective is optimized and converged, all defect information in the current grid is output to form a complete defect set. Based on the identification confidence of all defects in the defect set, the uncertainty value of all defects in the current grid can be solved. The maximum value of all defect uncertainty values ​​is taken as the overall defect uncertainty value of the current grid.

[0025] Furthermore, covering the planning of intelligent agents Input observation state space Heatmap of mesh uncertainty U and heatmap of mesh defect identification confidence With real-time focal length Segmentation and composition; coverage planning intelligent agent The action space is a set of global traversal and focus control actions, which can be expressed as follows: All output actions match the device motion constraints.

[0026] Furthermore, covering the planning of intelligent agents The reward function expression is: ,in, To achieve the core reward for grid coverage completion, when the coverage planning agent... A fixed positive reward will be given after moving to an uncovered grid and performing a detection process on that grid; The reward for traversal efficiency is calculated using the following formula: In the formula, Representative coverage planning intelligent agent The Euclidean distance between the two positions. As an indicator function, a penalty term is triggered when the target mesh has been fully covered, constraining invalid repeated access behavior. Representing the grid Coverage identifier; Rewards for exploring unknown areas are calculated using the following formula: ,in, The overall defect uncertainty value of the target mesh; The calculation formula for motion smoothing reward is as follows: Among them, when covering the planning intelligent agent Current action Actions at the previous moment Maintaining consistency will result in positive rewards.

[0027] Furthermore, the equally spaced discrete grid map obtained in step S2 The i-th grid The information includes: spatial coordinates Coverage mark The overall defect uncertainty value of the i-th grid The confidence level of defect identification for the i-th grid Defect type {Cracks, dents, missing rivets, paint peeling, scratches}.

[0028] Furthermore, the method for obtaining the initial aircraft skin surface image in step S1 and preprocessing the image is as follows:

[0029] Acquire raw color images of the aircraft skin surface in the current field of view. Convert the original color image to a grayscale image to obtain a single-channel grayscale image. Then use Gaussian kernel for single-channel grayscale images After filtering and denoising, histogram equalization is applied to the filtered image for enhancement, resulting in the preprocessed image. Where H and W represent the width and height of the image, respectively. The pixel value represents a real number.

[0030] Furthermore, the standard deviation is used. The Gaussian bimodal concave function is used as the uncertainty mapping formula to determine the defect uncertainty value. The calculation formula is:

[0031] (2);

[0032] in, The uncertainty value represents the j-th defect within the grid; the defect identification confidence level. ,when Approaching 0 or When it approaches 1, the defect uncertainty value All values ​​approach 0, indicating extremely high reliability of the test results; when When the value approaches 0.5, the uncertainty in defect detection is at its maximum.

[0033] An autonomous aircraft skin inspection system based on two-layer deep reinforcement learning is used to implement the aforementioned autonomous aircraft skin inspection method based on two-layer deep reinforcement learning, including:

[0034] The image preprocessing module is used to acquire initial images of the aircraft skin surface and to preprocess the images.

[0035] The grid map acquisition and initialization module is used to acquire preprocessed images. The aircraft skin is segmented to obtain a discrete grid map with equal spacing. Construct and initialize the uncertainty heatmap for the corresponding grid. Compare the defect identification confidence heatmap C with the actual defect identification confidence level, and determine the starting point of the inspection equipment's operation.

[0036] The defect detection agent construction and defect detection module are used to build the defect detection agent. The agent is trained, and the Actor policy weights are saved after training to obtain the trained defect detection agent. ; Utilizing the trained defect detection agent Conduct single-grid aircraft skin defect detection and uncertainty assessment.

[0037] The overall defect uncertainty calculation and information update module for the mesh is used to calculate the overall defect uncertainty value of the current mesh and update the uncertainty heatmap based on all defect information within the current mesh obtained from the defect detection agent construction and defect detection module. Compare the defect identification confidence heatmap C with the grid coverage flags and update the grid coverage flags. ;

[0038] A module for constructing and optimizing coverage planning agents is included. And train it to obtain the trained coverage planning agent. Determine whether all grids are covered and the overall defect uncertainty of all grids cannot be reduced. If yes, output the inspection result; otherwise, use the trained coverage planning agent. The camera's multi-dimensional pose and focal length are optimized and adjusted, and then the process is transferred to the defect detection agent construction and defect detection module for a new round of inspection.

[0039] The beneficial technical effects of this invention are as follows:

[0040] 1. To address the difficulty in balancing scanning accuracy and inspection efficiency, and the low efficiency of traditional two-stage inspection processes, this invention designs a two-layer deep reinforcement learning architecture with clearly defined responsibilities and parallel collaboration between the upper and lower layers, decoupling macroscopic path planning, focusing, and microscopic viewpoint control; intelligent agent The intelligent agent is responsible for global path and direction planning for the camera, as well as automatic focus adjustment. Focusing on lens adjustment and defect detection, both are performed simultaneously, eliminating the need for sequential operations and improving overall speed. Simultaneously, a real-time communication mechanism based on detection uncertainty is established, enabling intelligent agents to... The identification results and uncertainties are transmitted back to the intelligent agent in real time. intelligent agent Based on feedback, the scanning strategy is dynamically adjusted to flexibly adapt the focal length and shooting range under different working conditions, effectively balancing imaging accuracy and inspection efficiency.

[0041] 2. To address the issues of lacking a feedback loop in inspection planning and the inability to proactively re-inspect difficult areas, this invention constructs an uncertainty heatmap aligned with the aircraft segmentation results. This heatmap serves as a shared global state for the two agents, recording and storing the detection confidence and uncertainty data for each area in real time. After the defect detection agent completes identification, it synchronously updates the heatmap information. Read the uncertainty heatmap data and proactively plan secondary inspection tasks for areas with high uncertainty; after the re-inspection is completed, refresh the heatmap status again to form a complete closed loop of detection, feedback, re-inspection and update. Relying on the long-term state memory of uncertainty, the uncertainty-driven dual-agent collaborative decision-making is realized, which greatly improves the defect detection effect in difficult areas.

[0042] 3. This invention aims to solve the problems of difficulty in balancing accuracy and efficiency in long-range inspection with long focal length, slow operation of traditional serial inspection process, and separation between inspection planning and defect detection. Relying on dual-agent collaborative architecture and uncertainty heat map technology, it realizes parallel linkage of path planning, automatic focusing and defect detection, builds a complete inspection closed-loop system, and finally achieves safe, efficient and high-precision long-range defect detection of aircraft skin. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the autonomous aircraft skin inspection method based on two-layer deep reinforcement learning provided in Embodiment 1 of the present invention. Detailed Implementation

[0044] Example 1

[0045] like Figure 1 As shown, this embodiment discloses an autonomous aircraft skin inspection method based on two-layer deep reinforcement learning, including the following steps:

[0046] S1. Acquire initial images of the aircraft skin surface and preprocess them; specifically, acquire raw color images of the aircraft skin surface at the current field of view using a fixed-point zoom PTZ camera at the lowest magnification. Convert the original color image to a grayscale image to obtain a single-channel grayscale image. Then use Gaussian kernel for single-channel grayscale images After filtering and denoising, histogram equalization is applied to the filtered image for enhancement, resulting in the preprocessed image. Where H and W represent the width and height of the image, respectively. The pixel value represents a real number;

[0047] Specifically, during image preprocessing, the original color image is first converted into a grayscale image. Then, a 5×5 Gaussian filter is used to smooth the grayscale image to suppress image noise. After filtering, histogram equalization is introduced to eliminate interference such as uneven lighting and differences in brightness, and finally, the image after preprocessing is obtained.

[0048] S2, Based on the acquired preprocessed image The aircraft skin is segmented to obtain a discrete grid map with equal spacing. And determine the starting point for the inspection equipment;

[0049] Specifically, the acquired preprocessed images Input after pre-training Aircraft skin segmentation model (it should be noted that this is a pre-trained model) The aircraft skin segmentation model is an existing network model that obtains pixel-level mask images of the aircraft skin. (1 represents the skinned area, 0 represents the background), mask image The skinned area in the map is divided into equally spaced discrete grid maps. The i-th grid The information includes: spatial coordinates Coverage mark (0 represents uncovered, 1 represents covered), the overall defect uncertainty value of the i-th grid. Defect identification confidence level Defect type {Cracks, dents, missing rivets, paint peeling, scratches}; Construct a corresponding grid map Uncertainty heatmap Heatmap of confidence level for defect identification , The defect identification confidence score represents the Nth grid cell, where N represents the number of grid cells. Each grid cell has an initial uncertainty of 1 and a defect identification confidence score of 0. Finally, the skin mask image is calculated. geometric center The starting point is the grid closest to the geometric center. The control pan-tilt unit moves the inspection equipment to... Adjust the focal length to the maximum focal length and align the center of the camera's view with the grid.

[0050] S3. Construct a defect detection intelligent agent The agent is trained, and the Actor policy weights are saved after training to obtain the trained defect detection agent. ; Utilizing the trained defect detection agent Conducting single-grid aircraft skin defect detection and uncertainty assessment: Defect detection intelligent agent Adjust the camera tilt angle Deflection angle As a core action variable, the camera attitude is continuously adjusted iteratively. After each attitude adjustment, the inspection equipment acquires images and performs defect identification tasks. Simultaneously, it updates the attitude parameters based on the confidence level of the defect identification in this round, iterating repeatedly until the camera viewpoint and detection results converge, locking in the optimal imaging viewpoint. Specifically, when the defect detection agent... After the pose adjustment is completed, the pre-trained YOLO model is called to perform multi-class defect detection. The model outputs the coordinate position, confidence score and category of each defect target.

[0051] Specifically, defect detection intelligent agents It comprises four independent networks. The network input is a 514-dimensional state vector obtained by concatenating gimbal pose and image features, where the pose dimension is 2-dimensional and the image feature dimension is 512-dimensional (the image features are obtained through a lightweight ResNet18 pre-trained visual feature extraction network). The four independent networks are as follows: The first network is the Actor policy network, which takes a 514-dimensional state vector as input. The input is processed sequentially through three hidden layers containing 514, 256, and 128 neurons, with ReLU activation and layer normalization, before splitting into two output branches. The mean branch is activated by Tanh and outputs a 2D corner mean. It is then mapped to the interval [-1, 1], and the variance branch is activated by Softplus to output the 2D variance. To ensure that the variance is always greater than 0, a biaxial rotation Gaussian distribution is ultimately generated. After sampling to obtain the control action, numerical trimming and hardware limit matching are completed; the second and third groups are two completely independent dual-Q value networks. , The first two are used to mitigate the overestimation bias of value. The network input is a 516-dimensional vector obtained by concatenating a 514-dimensional state vector with a 2-dimensional action vector. The number of hidden layers is set to three because the input information is not much. The number of neurons, activation function and normalization method are consistent with the Actor network. Finally, the output is a single-dimensional state-action value. The fourth group is the value target network V, which is used to fit the state value baseline to stabilize the training process. The network input is only a 514-dimensional state vector. The hidden layer structure is the same as the above networks. The output is a single-dimensional state value.

[0052] Defect detection intelligent agent Input observation state space It is composed of the gimbal attitude and the high-dimensional feature vector of the image after dimensionality reduction, and is expressed as follows: ;in, This represents the current horizontal angle of the gimbal, and its original value range. , This represents the current tilt angle of the gimbal, and its original value range. Both are normalized to the interval [-1, 1]. The feature vector after image compression is obtained by preprocessing the image. The input is a pre-trained lightweight ResNet18 visual feature extraction network (this pre-trained lightweight ResNet18 visual feature extraction network belongs to the existing network structure), and the output is a 512-dimensional normalized feature vector as the image observation representation; the defect detection agent... The action space is for continuous control output The mathematical expression for the incremental rotation control of the gimbal's horizontal and pitch axes is as follows: In the formula, Represents the increment of a single horizontal rotation. Represents the increment of pitch rotation in a single stroke. The maximum single rotation angle specified by the hardware, such as the maximum adjustment in a single action. After the network outputs the action, it will be limited to a range through numerical clipping. Within the limits, prevent the gimbal from moving beyond the mechanical safety boundaries;

[0053] Defect detection intelligent agent reward function A composite segmented reward system is adopted, consisting of a weighted sum of the main reward for object detection confidence and the reward for pose convergence stability. The overall expression is as follows: ;in, , These represent the weighting coefficients for the main reward and the stable reward, respectively. The main reward represents the confidence score of object detection. To optimize the core indicator, the higher the value, the better the confidence score of object detection in the current image. Considering that multiple objects are usually detected in a single image, the average of the confidence scores of all objects in the image is used as the quantitative score of image quality. The reward function is designed based on this score to improve the final image quality. The reward for stable pose convergence is used to incentivize the defect detection agent to converge to the optimal observation pose. If the amplitude of two consecutive actions is less than a threshold, the reward is applied. Then a fixed positive reward is assigned, which can be expressed mathematically as:

[0054] (1);

[0055] In the formula: These are manually adjusted coefficients used to balance the influence weights of the two rewards; , They represent defect detection intelligent agents respectively. Continuous control output at steps t and t-1; defect detection agent There are three types of episode termination conditions: a positive termination condition is that the amplitude of the output action is less than a threshold for 10 consecutive steps. At this point, it is determined that the gimbal has been adjusted to the optimal imaging posture, and the current episode ends normally. The negative termination condition is that the gimbal movement touches the mechanical limit switch, which triggers the premature termination of the current episode. At the same time, a maximum step size termination constraint is set, limiting a single episode to a maximum of 20 control operations, thereby avoiding the infinite loop problem caused by the defect detection agent's continuous meaningless search.

[0056] In the construction of the defect detection intelligent agent During training, the standard training process is carried out iteratively in episodes. Each training round first performs an environment reset, randomly initializes the gimbal pose, and then preprocesses the acquired initial images before inputting them into the pre-trained lightweight ResNet18 visual feature extraction network to obtain 512-dimensional image features. The pose vector and image features are then concatenated to obtain the initial state. Subsequently, a single-step interactive loop is entered, with each step first inputting from the Actor network. Output action Gaussian distribution and sample continuous control action Then, the motion is input to the gimbal, the gimbal angle is updated, and the camera generates images from the corresponding viewpoints. Features are then extracted and stitched together to obtain the next state. Next, the reward function is calculated. And determine the termination flag. , sample The data is stored in the experience replay pool. A network batch update is performed every 50 interactive steps. The network batch update process includes: randomly selecting 256 samples from the replay pool to form a batch; sequentially calculating the double-Q network loss and updating the parameters; fixing the Q network and solving the Actor policy gradient loss and updating the policy network; and adaptive entropy coefficients. Iteration, with soft update coefficients The four operations of soft update target value network are as follows: when the interaction process triggers limit constraints, the pose converges and stabilizes after multiple consecutive steps, or the maximum number of steps in a single round is reached, the current episode ends, the total reward of this round is recorded, the environment is reset and the next round of iteration is entered, until the average total reward of 50 consecutive episodes converges and there is no significant improvement, the simulation pre-training process ends and the Actor policy weights are saved.

[0057] This step relies on a pre-trained defect detection agent. Completed single-grid aircraft skin defect detection and uncertainty assessment; defect detection intelligent agent The core function is to minimize the uncertainty of defect detection in the mesh area through multiple rounds of camera viewpoint iteration optimization from the current perspective. The system uses the current mesh center as the reference and the current focal length... The skin image acquired at the corresponding magnification is used as input material, and the defect detection agent... Adjust the camera tilt angle Deflection angle As the core action variable, the camera pose is continuously adjusted iteratively. After each pose adjustment is completed, an image is acquired and a defect recognition task is performed. At the same time, the pose parameters are updated based on the confidence of the defect recognition in this round. The cycle is iterated until the camera viewpoint and the detection result converge, and the optimal imaging viewpoint is locked.

[0058] S4, Defect Detection Intelligent Agent After the observation perspective is optimized and converged, all defect information within the current grid is output, forming a complete defect set. : ,in, The coordinates of the defect center are... For the circumscribed moment of the defect, Defect type The defect identification confidence score for the j-th defect in the current grid, where n represents the number of defects in the defect set;

[0059] To accurately quantify the detection uncertainty, the standard deviation is used. The Gaussian bimodal concave function is used as the uncertainty mapping formula to determine the defect uncertainty value. The calculation formula is:

[0060] (2);

[0061] in, A higher value indicates higher detection uncertainty; defect identification confidence level. ,when Approaching 0 (confirming a defect-free target) or When the value approaches 1 (confirming the existence of a defective target), the defect uncertainty value is... All values ​​approach 0, indicating extremely high reliability of the test results; when When the value approaches 0.5 (the test result is ambiguous), the defect uncertainty value Reaching the peak value indicates the highest detection uncertainty. Based on the identification confidence of all defects within the defect set, the uncertainty value of all defects in the current grid can be calculated. The maximum value of all defect uncertainties is taken as the overall defect uncertainty value of the current grid. :

[0062] (3);

[0063] It should be noted that the maximum value of all defect uncertainty values ​​in the current grid is taken as the overall uncertainty of the grid. As long as any defect in the grid has a high uncertainty, it means that there is a significant risk in the detection results of that area. Using the maximum value can conservatively reflect the worst-case reliability of the detection results of that grid.

[0064] Based on the obtained overall defect uncertainty value of the current mesh Real-time updates of global uncertainty heatmap and update the grid coverage flag. ;

[0065] S5, Constructing a Coverage Planning Intelligent Agent And train it to obtain the trained coverage planning agent. Using the trained coverage to plan intelligent agents Optimize and adjust camera pose and focal length;

[0066] In this embodiment, the coverage planning intelligent agent The network consists of five modules: a grid embedding layer, a state feature concatenation module, a shared fully connected hidden layer, a policy output head, and a value output head. The overall network has a policy-value dual structure, enabling decision-making for discrete traversal actions and state value evaluation. The specific structure is as follows:

[0067] Grid Embedding Layer: This layer is used to implement dense feature mapping of discrete grid indices, integrating the global grid index. Mapped to 16-dimensional high-dimensional dense feature vectors Among them, the trainable parameter matrix It mines the differences in positional features between different grids through embedding learning; the state feature splicing module: the network input state It is obtained by splicing and combining multi-dimensional environmental observation features, with an overall dimension of 35, and is fully expressed as follows: ,in, This is a feature indicating the coverage of eight neighborhoods within a grid. The eight-neighborhood normalized uncertainty characteristics read from the uncertainty heatmap; The global grid coverage ratio of the scene, where, This represents the number of grids that have been covered in the entire scene; This represents the current grid's normalized historical access count. The network uses the average reward feature of the agent's last 5 steps, and multi-class feature fusion to achieve a complete representation of the grid traversal state. A shared fully connected hidden layer is used: the network has three shared fully connected layers with 256, 128, and 64 neurons respectively, all uniformly paired with the ReLU activation function to achieve deep extraction and dimensional mapping of state features, providing a unified high-order feature representation for the policy and value branches. The policy output head: based on the deep features extracted by the shared fully connected hidden layer, a 7-dimensional logarithmic probability of the original action is output through the fully connected layer. After normalization by the Softmax activation function, the probability distribution of discrete traversal actions is obtained. This corresponds to seven types of spatial traversal and fine-tuning actions; the value output head: based on the same shared depth features, it outputs a one-dimensional scalar state value through an independent fully connected layer. , which is used to fit the value baseline of the current mesh traversal state.

[0068] Coverage planning intelligent agent Input observation state space Heatmap of mesh uncertainty U and heatmap of mesh defect identification confidence With real-time focal length The splicing structure can be characterized as Coverage planning intelligent agents The action space is a discrete control output, defined as a set of global traversal and focus control actions, expressed as follows: All output actions strictly match the device motion constraints.

[0069] Coverage planning intelligent agent reward function A composite reward system is adopted, integrating four factors: grid coverage, traversal efficiency, unknown exploration, and motion smoothness. The overall expression is: ,in, Achieving the core reward for grid coverage is the primary optimization metric for the task, and this is achieved by the coverage planning agent. After moving to an uncovered grid and performing step S3 detection on that grid, a fixed positive reward of 1.0 will be awarded to incentivize the coverage planning agent. Complete the task of covering the entire unknown grid. The traversal efficiency constraint reward is used to suppress repetitive traversal behavior and improve overall task efficiency. Its calculation formula is as follows: In the formula, Representative coverage planning intelligent agent The Euclidean distance between the two positions. As an indicator function, when the target mesh has been fully covered ( When a penalty is triggered, it constrains invalid repeated access behavior. Representing the grid Coverage identifier; Rewards are given for exploring unknown areas to guide the coverage planning agent. Prioritize detection of high-value areas; the calculation formula is as follows: ,in, The overall defect uncertainty value of the target grid represents the overall defect uncertainty value. The higher the overall defect uncertainty value of the target grid, the higher the exploration reward, which can effectively incentivize the coverage planning agent. Prioritize traversing regions of high uncertainty that are not fully perceived; The motion smoothing reward is used to constrain and cover the planned agent's motion trajectory, avoiding frequent direction changes and jitter issues. The calculation formula is as follows: Among them, when covering the planning intelligent agent Current action Actions at the previous moment Maintaining consistency will result in positive rewards, ensuring a smooth operational trajectory and consistent movement of the intelligent agent. This covers the planning of intelligent agents. The task termination condition is set as global grid convergence judgment: when all grids in the scene are fully covered, and the overall defect uncertainty value of all grids no longer increases iteratively and tends to stabilize, the single traversal task is considered complete, and the coverage planning agent is deemed to have completed its task. Terminate the current interactive iteration process.

[0070] It should be noted that in training the coverage planning agent... The overall training process is carried out iteratively in units of episodes, aiming to drive the coverage planning agent. The system learns efficient traversal and movement capabilities on grid maps and proactive exploration capabilities in high-uncertainty regions. Each training round begins with environment initialization, randomly generating a grid map with complete defect distribution characteristics to ensure all grids in the scene contain valid defect information. Simultaneously, a coverage planning agent is randomly selected. Traverse the starting grid Complete the pre-configuration for single-round training. Set the maximum number of interaction steps for single-round training. The current iteration can end early when the preset termination condition for the task is met. After entering the single-step environment interaction loop, the agent uses the grid traversal state at the current moment. The input is fed into the policy network to complete the forward propagation, and the output is the probability distribution of multiple spatial traversal actions. The current interaction action is obtained through probability sampling. After executing the corresponding movement and traversal action, the coverage planning agent updates its own grid position and global environment observation status in real time, and calculates the real-time reward for each step according to a preset composite reward function. For newly traversed grids, the coverage-related reward is calculated with a delay, and unified calculation is required after grid detection and correction. During the traversal, the grid coverage flag, overall defect uncertainty value, and global heatmap information can be updated in real time, and the corresponding exploration rewards and coverage rewards can be calculated simultaneously to dynamically correct and optimize the exploration state. Each completed single-step interaction enables the coverage planning agent to... collection Training samples, among which, This represents the instantaneous reward value returned by the environment after the action is performed at step t. Represents the execution of actions The next moment's observed state is obtained after the environment update. Indicate whether the current interaction step is the final step of this round of coverage tasks.

[0071] Training samples are stored in a dedicated experience replay buffer, and iterative interaction continues until the number of samples in the buffer reaches the batch threshold. Then, the PPO algorithm is activated to complete the batch update of the network, iteratively optimizing the parameters of the policy network and value network. Model convergence is judged by the average metric of the most recent 100 training rounds, when the coverage planning agent... When the global traversal coverage efficiency and global uncertainty optimization indicators tend to stabilize and there is no significant performance improvement, the model training is determined to be converged, the training process is terminated, and the optimal network parameters are saved.

[0072] In this step, the planning agent is covered. The heatmap of grid uncertainty U and the heatmap of grid confidence C generated in real time by the algorithm are compared with the real-time focal length. Based on the input, the system perceives the difference between the detection uncertainty distribution and the recognition confidence in the current skin mesh region, and executes multi-dimensional pose adjustment actions for the camera; covering the planning intelligent agent. The control actions include dynamically adjusting the camera focal length. By increasing the focal length, the camera's field of view can be expanded, allowing a single image to cover more areas of the grid to be inspected, thus improving global inspection efficiency. Alternatively, the focal length can be reduced to focus on local areas, achieving high-precision supplementary inspection. At the same time, it supports four-way translation adjustment of the camera corresponding to the grid, combined with focal length scaling.

[0073] S6. Termination Decision: The system continuously monitors the state changes of the global grid uncertainty heatmap and confidence heatmap in real time, covering the planning agent. After each pose optimization action such as camera focusing, translation, or field of view scaling is completed, the system checks the global termination condition: when the uncertainty of all grids cannot be reduced during continuous iteration, it is determined that the global detection has reached the optimal stable state and the iterative detection process is automatically terminated; when the termination condition is not met, the process proceeds to step S3 to start a new round of operations.

[0074] After the inspection is completed, the system will output the following: inspection path sequence and uncertainty heatmap. Each grid outputs the overall defect uncertainty value; a defect statistics table includes the defect type, location coordinates, detection confidence level, and the grid to which it belongs. At this point, the entire task of global grid defect detection and uncertainty assessment for aircraft skin is completed.

[0075] As an example, this embodiment compares the method of this application with traditional coverage planning algorithms and traditional threshold-based target detection algorithms. Detection speed and the complete execution effect of defect detection are used as comparison dimensions, with detection time and overall defect detection completion rate selected as core evaluation indicators. A dataset of aircraft skin collected on the tarmac was used, and all tests were completed on a single-camera platform. Experimental results show that, compared to traditional multi-layer coverage planning algorithms, the automatic focus coverage planning method designed in this application does not require repeated acquisition of aircraft skin data at multiple focal lengths; a single planning operation is sufficient to complete full-area coverage acquisition. Taking a camera with 5 focal length settings as an example, the method of this application can reduce coverage planning time by approximately 81.92%. Meanwhile, compared to traditional threshold-based target detection algorithms, the defect detection confidence of the method of this application is improved by approximately 32.34%. The method of this application integrates the advantages of both traditional algorithms, coverage planning and target detection, effectively improving both the efficiency of full-area skin coverage planning and the performance of complete defect detection execution.

[0076] In addition, the method of this application has the following additional advantages:

[0077] The built-in attitude fine-tuning mechanism of the defect detection agent can autonomously optimize the camera imaging angle and shooting focal length, and adjust the gimbal dual-axis rotation angle to the optimal observation attitude in real time. This effectively avoids problems such as image blurring, exposure imbalance, and target offset, improving the quality of aircraft skin image acquisition from the source and providing highly reliable input data for subsequent defect identification.

[0078] The entire algorithm can complete the full skin detection operation using only a single camera, with no strict spatial constraints on camera installation points, allowing for high deployment flexibility. The underlying architecture naturally supports multi-device expansion; simply adding camera instances and supplementing global coordination logic allows for the rapid construction of a multi-camera joint observation system, adaptable to large-scale, multi-target full-coverage planning scenarios.

[0079] Example 2

[0080] Based on the aircraft skin autonomous inspection method based on two-layer deep reinforcement learning provided in Example 1, this example provides an aircraft skin autonomous inspection system based on two-layer deep reinforcement learning, including:

[0081] The image preprocessing module is used to acquire initial images of the aircraft skin surface and to preprocess the images.

[0082] The grid map acquisition and initialization module is used to acquire preprocessed images. The aircraft skin is segmented to obtain a discrete grid map with equal spacing. Construct and initialize the uncertainty heatmap for the corresponding grid. Compare the defect identification confidence heatmap C with the actual defect identification confidence level, and determine the starting point of the inspection equipment's operation.

[0083] The defect detection agent construction and defect detection module are used to build the defect detection agent. The agent is trained, and the Actor policy weights are saved after training to obtain the trained defect detection agent. ; Utilizing the trained defect detection agent Conduct single-grid aircraft skin defect detection and uncertainty assessment.

[0084] The overall defect uncertainty calculation and information update module for the mesh is used to calculate the overall defect uncertainty value of the current mesh and update the uncertainty heatmap based on all defect information within the current mesh obtained from the defect detection agent construction and defect detection module. Compare the defect identification confidence heatmap C with the grid coverage flags and update the grid coverage flags. ;

[0085] A module for constructing and optimizing coverage planning agents is included. And train it to obtain the trained coverage planning agent. Determine whether all grids are covered and the overall defect uncertainty of all grids cannot be reduced. If yes, output the inspection result; otherwise, use the trained coverage planning agent. The camera's multi-dimensional pose and focal length are optimized and adjusted, and then the process is transferred to the defect detection agent construction and defect detection module for a new round of inspection.

[0086] A non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the autonomous aircraft skin inspection method based on two-layer deep reinforcement learning as described above.

[0087] Furthermore, the present invention adopts the following technical solution:

[0088] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described autonomous aircraft skin inspection method based on two-layer deep reinforcement learning.

[0089] From the above description of the embodiments, those skilled in the art will clearly understand that the facilities of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Embodiments of the present invention can be implemented using existing processors, or by dedicated processors used for this or other purposes for suitable systems, or by hardwired systems. Embodiments of the present invention also include non-transitory computer-readable storage media, comprising machine-readable media for carrying or having machine-executable instructions or data structures stored thereon; such machine-readable media can be any available medium accessible by a general-purpose or special-purpose computer or other machine with a processor. For example, such machine-readable media can include RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of machine-executable instructions or data structures and is accessible by a general-purpose or special-purpose computer or other machine with a processor. When information is transmitted or provided to a machine via a network or other communication connection (hardwired, wireless, or a combination of hardwired and wireless), that connection is also considered a machine-readable medium.

[0090] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. An autonomous inspection method for aircraft skin based on two-layer deep reinforcement learning, characterized in that, Includes the following steps: S1. Obtain an initial image of the aircraft skin surface and preprocess the image; S2, Based on the acquired preprocessed image The aircraft skin is segmented to obtain a discrete grid map with equal spacing. Construct and initialize the uncertainty heatmap for the corresponding grid. Compare the defect identification confidence heatmap C with the actual defect identification confidence level, and determine the starting point of the inspection equipment's operation. S3. Construct a defect detection intelligent agent The agent is trained, and the Actor policy weights are saved after training to obtain the trained defect detection agent. ; Utilizing the trained defect detection agent Conduct single-grid aircraft skin defect detection and uncertainty assessment. S4. Based on the complete defect information obtained in step S3, calculate the overall defect uncertainty value of the current grid and update the uncertainty heatmap. Compare the defect identification confidence heatmap C with the grid coverage flags and update the grid coverage flags. ; S5, Constructing a Coverage Planning Intelligent Agent And train it to obtain the trained coverage planning agent. Determine whether all grids are covered and the overall defect uncertainty of all grids cannot be reduced. If yes, output the inspection result; otherwise, use the trained coverage planning agent. After optimizing and adjusting the camera's pose and focal length in multiple dimensions, proceed to step S3 for a new round of inspection.

2. The autonomous aircraft skin inspection method based on two-layer deep reinforcement learning according to claim 1, characterized in that: The defect detection intelligent agent The input observation state space is expressed as ;in, The current angle of the gimbal represents the horizontal position. This represents the current tilt angle of the gimbal. The feature vector after image compression is obtained by preprocessing the image. The input is a pre-trained lightweight ResNet18 visual feature extraction network, which outputs a 512-dimensional normalized feature vector as an image observation representation; a defect detection agent... The action space is for continuous control output In the formula, This represents the increment of a single horizontal rotation of the gimbal. This represents the increment of the gimbal's pitch rotation in a single operation. The maximum single rotation angle specified by the hardware; when the defect detection agent... After completing the posture adjustment, the pre-trained YOLO model is called to perform multi-class defect detection, and the coordinate position, confidence score and category of each defect target are output.

3. The autonomous aircraft skin inspection method based on two-layer deep reinforcement learning according to claim 1, characterized in that: Defect detection intelligent agent reward function A composite segmented reward system is adopted, expressed as follows: ;in, , These represent the weighting coefficients for the main reward and the stable reward, respectively. The main reward represents the confidence score of object detection. The average of the confidence scores of all object detections in the image is used as the quantitative score of image quality to improve the final image quality. The reward for stable pose convergence is used to incentivize the defect detection agent to converge to the optimal observation pose. If the amplitude of two consecutive actions is less than a threshold, the reward is applied. Then a fixed positive reward is assigned, which can be expressed mathematically as: （1）； In the formula: These are manually adjusted coefficients used to balance the influence weights of the two rewards; , For defect detection intelligent agents Continuous control output at steps t and t-1.

4. The autonomous aircraft skin inspection method based on two-layer deep reinforcement learning according to claim 1, characterized in that: The method for calculating the overall defect uncertainty value of the current grid based on obtaining all defect information within the current grid in step S3 is as follows: Defect detection intelligent agent Adjust the camera tilt angle Deflection angle The camera pose is continuously adjusted iteratively as an action variable until the camera viewpoint and detection results converge, thus forming a defect detection agent. After the observation perspective is optimized and converged, all defect information in the current grid is output to form a complete defect set. Based on the identification confidence of all defects in the defect set, the uncertainty value of all defects in the current grid can be solved. The maximum value of all defect uncertainty values ​​is taken as the overall defect uncertainty value of the current grid.

5. The autonomous aircraft skin inspection method based on two-layer deep reinforcement learning according to claim 1, characterized in that: Coverage planning intelligent agent Input observation state space Heatmap of mesh uncertainty U and heatmap of mesh defect identification confidence With real-time focal length Segmentation and composition; coverage planning intelligent agent The action space is a set of global traversal and focus control actions, which can be expressed as follows: All output actions match the device motion constraints.

6. The autonomous aircraft skin inspection method based on two-layer deep reinforcement learning according to claim 1, characterized in that: Coverage planning intelligent agent The reward function expression is: ,in, To achieve the core reward for grid coverage completion, when the coverage planning agent... A fixed positive reward will be given after moving to an uncovered grid and performing a detection process on that grid; The reward for traversal efficiency is calculated using the following formula: In the formula, Representative coverage planning intelligent agent The Euclidean distance between the two positions. As an indicator function, a penalty term is triggered when the target mesh has been fully covered, constraining invalid repeated access behavior. Representing the grid Coverage identifier; The reward for exploring unknown areas is calculated using the following formula: ,in, The overall defect uncertainty value of the target mesh; The calculation formula for motion smoothing reward is as follows: Among them, when covering the planning intelligent agent Current action Actions at the previous moment Maintaining consistency will result in positive rewards.

7. The autonomous aircraft skin inspection method based on two-layer deep reinforcement learning according to claim 1, characterized in that: The equally spaced discrete grid map obtained in step S2 The i-th grid The information includes: spatial coordinates Coverage mark The overall defect uncertainty value of the i-th grid The confidence level of defect identification for the i-th grid Defect type {Cracks, dents, missing rivets, paint peeling, scratches}.

8. The autonomous aircraft skin inspection method based on two-layer deep reinforcement learning according to claim 1, characterized in that: The method for obtaining the initial aircraft skin surface image in step S1 and preprocessing the image is as follows: Acquire raw color images of the aircraft skin surface in the current field of view. Convert the original color image to a grayscale image to obtain a single-channel grayscale image. Then use Gaussian kernel for single-channel grayscale images After filtering and denoising, histogram equalization is applied to the filtered image for enhancement, resulting in the preprocessed image. Where H and W represent the width and height of the image, respectively. The pixel value represents a real number.

9. An autonomous aircraft skin inspection system based on two-layer deep reinforcement learning, used to implement the method described in any one of claims 1-8, characterized in that, include: The image preprocessing module is used to acquire initial images of the aircraft skin surface and to preprocess the images. The grid map acquisition and initialization module is used to acquire preprocessed images. The aircraft skin is segmented to obtain a discrete grid map with equal spacing. Construct and initialize the uncertainty heatmap for the corresponding grid. Compare the defect identification confidence heatmap C with the actual defect identification confidence level, and determine the starting point of the inspection equipment's operation. The defect detection agent construction and defect detection module are used to build the defect detection agent. The agent is trained, and the Actor policy weights are saved after training to obtain the trained defect detection agent. ; Utilizing the trained defect detection agent Conduct single-grid aircraft skin defect detection and uncertainty assessment. The overall defect uncertainty calculation and information update module for the mesh is used to calculate the overall defect uncertainty value of the current mesh and update the uncertainty heatmap based on all defect information within the current mesh obtained from the defect detection agent construction and defect detection module. Compare the defect identification confidence heatmap C with the grid coverage flags and update the grid coverage flags. ; A module for constructing and optimizing coverage planning agents is included. And train it to obtain the trained coverage planning agent. Determine whether all grids are covered and the overall defect uncertainty of all grids cannot be reduced. If yes, output the inspection result; otherwise, use the trained coverage planning agent. The camera's multi-dimensional pose and focal length are optimized and adjusted, and then the process is transferred to the defect detection agent construction and defect detection module for a new round of inspection.

Images