Automatic configuration method for outdoor inspection robot

Abstract

The invention belongs to the field of intelligent robots, and discloses an automatic configuration method for an outdoor inspection robot, which comprises the following steps of: acquiring point cloud data and an environment texture image, judging an environment grade after preprocessing, constructing a three-dimensional electronic map and establishing a dynamic coordination rule; a robot walks along a planned route to trigger a dynamic coordination rule, after double cameras shoot inspection equipment, a suspected inspection target is primarily screened and framed, a type is labeled, a primary screening result table is generated, a corrected labeled target set is generated through manual recheck and correction, an environment weighting factor is introduced, shooting angle deviation is compensated, and a full-amount unified labeled photo set is generated; constructing an inspection recognition model, outputting frame selection coordinates, target categories and target confidence coefficients, and judging target frame selection consistency; constructing a three-dimensional constraint condition, removing the same target picture, marking an effective inspection target, and calculating and adjusting the attitude compensation amount of 3D positioning; a correlation mapping table is generated based on large auxiliary target positioning and small target candidate area amplification identification, and the inspection target identification precision and configuration efficiency are improved.

Description

Technical Field

[0001] This invention relates to an automatic configuration method for outdoor inspection robots, belonging to the field of intelligent robot technology. Background Technology

[0002] Traditional outdoor inspection robots rely on full human intervention for configuration. This requires manual measurement of roads in the inspection area to create electronic maps and plan inspection routes. It also necessitates manual inspection of each robot-taken photo, manually selecting and labeling inspection targets – a tedious and time-consuming process. Manual configuration not only struggles to achieve 24 / 7 inspection coverage but is also prone to human error, leading to insufficient map accuracy, missed or incorrect target labeling, and impacting the robot's subsequent inspection accuracy. Furthermore, fixed shooting strides and camera postures cannot adapt to changes in the outdoor environment, affecting image quality. Data recording is mostly in non-digital form, hindering rapid transmission and analysis. Model training relies on repeated manual corrections, resulting in long iteration cycles and low overall automation and intelligence levels, failing to meet the demands of efficient outdoor inspections.

[0003] Chinese patent application CN121053200A discloses a method for automatic target configuration during robot outdoor inspections. First, an electronic map of the substation, including charging stations and default inspection routes, is manually measured and drawn. Then, the robot travels along these routes, periodically taking photos of equipment and recording its location information. Next, staff examine the photos, select inspection targets, and calculate SIFT feature points. Subsequently, target annotation and model training are performed on the photos, and errors are eliminated by comparing the recognition results with the SIFT algorithm output. After multiple iterations of training, accurate recognition is achieved. Finally, the system automatically identifies targets from all photos, analyzes their size, distance, and angle, and selects suitable photos for annotation based on the shooting angle. The gimbal then captures images of the targets based on the inspection route and target location, completing the configuration. This invention enables rapid target location, status assessment, fault diagnosis, and real-time transmission and analysis of related data, significantly improving the intelligence and automation of inspection work.

[0004] Although there is an existing method for automatic target configuration of robots in outdoor inspections, which achieves automatic identification and configuration of inspection targets by manually drawing electronic maps containing charging points and inspection routes, using robot fixed-step-length shooting equipment, manually selecting and marking targets, using SIFT algorithm feature matching and model iterative training, calculating target parameters with binocular vision, and adjusting shooting with 3D positioning, thus improving the intelligence and automation of outdoor inspections, there are still problems such as large workload and high labor costs for manual surveying and marking, low robot recognition accuracy for inspection targets and insufficient adaptability to complex outdoor environments, resulting in low efficiency of automatic configuration. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide an automatic configuration method for outdoor inspection robots. By integrating binocular vision, environmental perception, and SIFT feature matching, and combining AI initial screening with a human collaborative correction mechanism, the method achieves high-precision unified labeling and adaptive shooting of inspection targets. Furthermore, based on three-dimensional spatial constraints and small target context enhancement strategies, the method dynamically optimizes gimbal posture and shooting parameters, thereby solving the problems of low robot recognition accuracy for inspection targets, insufficient adaptability to complex outdoor environments, reliance on manual configuration, and low efficiency of automatic configuration.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An automatic configuration method for an outdoor inspection robot includes:

[0008] Collect point cloud data and environmental texture images, preprocess them, calculate image texture entropy and obstacle density to determine the environmental level, construct a 3D electronic map, and collect environmental parameters and combine them with inspection behavior parameters to establish dynamic collaborative rules.

[0009] The robot walks along the planned route and triggers dynamic collaborative rules based on real-time environmental parameters; it takes pictures of the inspection equipment with dual cameras and performs AI initial screening, automatically selects suspected inspection targets, labels them, and outputs the initial screening confidence level, generates an initial screening result table, and manually reviews and corrects them to generate a corrected labeled target set.

[0010] The corrected bounding box targets are subjected to environment-adaptive SIFT operation. An environment weighting factor is introduced to correct the feature point extraction weights and compensate for the shooting angle deviation, generating a full set of uniformly labeled photos. An inspection and recognition model is constructed to output bounding box coordinates, target categories and target confidence, and the bounding box overlap is calculated to determine the consistency of target bounding boxes.

[0011] Three-dimensional constraints are constructed to screen valid inspection photos. An improved ORB algorithm is used to remove photos with the same target and label valid inspection targets. The bat algorithm is used to calculate and adjust the pose compensation amount of 3D positioning. Based on the large auxiliary target, the candidate region of small target is located and enlarged for recognition, and an association mapping table is generated.

[0012] Specifically, the steps for determining the environmental level include:

[0013] Point cloud data of outdoor inspection areas are collected to obtain key parameters; environmental texture images are collected; and image texture entropy is calculated after denoising. ;

[0014] Set texture entropy threshold , ,like If the texture complexity is low, then the texture complexity is determined to be low; if If the texture complexity is determined to be medium; if If so, the texture complexity is determined to be high;

[0015] Establish a data association index table to determine the effective sensing area and calculate obstacle density. ;

[0016] Set density threshold , ,like If the obstacle density level is low, then the obstacle density level is determined to be low; if If the obstacle density level is determined to be medium; if If so, the obstacle density level is determined to be high;

[0017] If the texture complexity is high and the obstacle density level is high, it is determined to be a level 3 environment; if the texture complexity is low and the obstacle density level is low, it is determined to be a level 1 environment; otherwise, it is determined to be a level 2 environment.

[0018] Specifically, the steps for establishing dynamic collaborative rules include:

[0019] Based on the environmental level, candidate charging areas are selected, with priority given to first-level environmental areas with low texture complexity and low obstacle density, while third-level environmental areas are avoided, and smooth road sections are selected for second-level environmental areas.

[0020] The number of obstacle point clouds in each charging candidate area is counted, and charging candidate areas with fewer than the judgment threshold are selected. Point cloud data is collected and the coordinates of the charging position are determined. The corresponding environmental level and charging position attributes are associated and labeled. The point clouds of each charging candidate area are stitched together to form a global point cloud dataset.

[0021] Based on the A* algorithm, a default inspection route is planned, prioritizing the passage through first-level environmental road sections and avoiding third-level environmental road sections. The route also passes through all selected charging candidate areas and charging locations. The environmental level, charging location, and inspection route are added to the global point cloud dataset to construct a three-dimensional electronic map.

[0022] Collect environmental parameters of the inspection area, and combine the environmental level with the inspection behavior parameters to establish dynamic collaborative rules.

[0023] Specifically, the steps to trigger dynamic collaboration rules include:

[0024] The system receives the dynamic coordination rules, controls the robot to walk along the planned route in the three-dimensional electronic map, and obtains the robot's current position coordinates in real time.

[0025] Set the dual cameras to a fixed elevation angle of 30 degrees. To cover the key areas of the inspection equipment;

[0026] Construct three-dimensional trigger conditions. In the lighting dimension, when the light intensity is less than or equal to the preset light threshold, start the exposure parameter optimization.

[0027] In terms of device spacing, when the device spacing is less than or equal to a preset spacing threshold, the stop shooting step size is shortened.

[0028] In terms of visibility, when the visibility is less than or equal to the preset visibility threshold, increase exposure compensation;

[0029] If all three trigger conditions are met, the dynamic coordination rule is triggered, and shooting is performed using a shortened stop shooting step and optimized exposure parameters;

[0030] If the triggering conditions of any single dimension or any two dimensions are met, the adjustment rules corresponding to the met dimensions will be executed; otherwise, no parameter adjustment will be performed.

[0031] Specifically, the steps for generating the corrected annotation target set include:

[0032] The robot continues to walk along the planned route according to the adjusted stop shooting step length. It performs real-time positioning calibration by combining GPS / IMU fusion positioning data with the route coordinates of the 3D electronic map. When it reaches the shooting point, it stops, and the dual cameras load optimized exposure parameters and maintain a fixed elevation angle to inspect the equipment on both sides of the shooting route.

[0033] Calculate the disparity map variance. If the disparity map variance is greater than a preset variance threshold, the image is determined to be blurry; otherwise, the image is determined to be clear.

[0034] Calculate the occlusion area. If the occlusion area is greater than or equal to a preset area threshold, it is determined to be occlusion; otherwise, it is determined to be unoccluded.

[0035] Images that are clear and unobstructed are initially retained and integrated into a preliminary effective image set. This set is then input into a trained YOLOv8n model for initial AI screening, and the output includes the category, bounding box, and initial screening confidence of the inspected target.

[0036] The initial screening confidence levels are divided into three levels: Level 1, Level 2, and Level 3. The initial screening results are integrated and generated into a table. The Level 2 confidence items are then manually reviewed to generate a set of corrected and annotated targets.

[0037] Specifically, the steps for generating a complete set of uniformly labeled photos include:

[0038] Obtain the corrected bounded target, quantify the influence weight of each environmental parameter, normalize the light intensity, equipment spacing and visibility, and calculate the environmental weighting factor in combination with the influence weight;

[0039] The actual shooting angle of the target is obtained, and the angle deviation value is calculated by combining it with a fixed elevation angle. The actual height of the target, the image scaling factor and the device spacing are obtained to calculate the image offset factor. The focal length of the dual cameras is obtained and the angle deviation value is combined to calculate the angle deviation compensation amount. The target bounding box coordinates are adjusted to generate an angle-compensated target area image.

[0040] Feature points are extracted to generate a template feature vector library. For the target area of ​​the full inspection photos, the response value of the feature points is corrected by combining the environmental weighting factor. The feature vector to be matched is extracted, and the matching distance between the target area to be matched and the template target is calculated.

[0041] If the matching distance is less than or equal to the preset distance threshold, the matching score is calculated, and the effective matching feature point pairs are retained according to the preset score threshold. The target contour is fitted and the bounding box coordinates are combined with the corrected bounding box coordinates to uniformly select and label the target, generating a full set of uniformly labeled photos.

[0042] Specifically, the steps for determining the consistency of target selection include:

[0043] Construct an inspection and identification model, including an input layer, a feature fusion layer, a feature decision layer, and an output layer;

[0044] The input layer is used to receive image features and environmental features;

[0045] The feature fusion layer is used to weightedly fuse the image features and the environmental features through a multi-head self-attention mechanism to generate a fused feature vector;

[0046] The feature decision layer is used to receive the fused feature vector, extract key features, fit the target bounding box coordinates using a regression branch, and determine the target category and recognition reliability through a classification branch.

[0047] The output layer is used to output the bounding coordinates of the target, the target category, and the target confidence level;

[0048] Using the full set of uniformly labeled photos as the training set, the inspection and recognition model is trained by combining the total loss function and the AdamW optimizer. After training, the overlap of the bounding boxes is calculated and the target confidence is combined to determine the consistency of the target bounding boxes.

[0049] If the overlap of the selected boxes is greater than or equal to the preset overlap threshold, and the target confidence is greater than or equal to the preset confidence threshold, then the target selection consistency is determined to be up to standard; otherwise, the inspection and recognition model is updated.

[0050] Specifically, the steps for constructing three-dimensional constraints include:

[0051] For the full range of labeled inspection photos, based on the binocular vision ranging formula, camera parameters and environmental parameters are fused and an atmospheric scattering correction term is introduced. The target disparity is calculated by matching the left and right views of the binocular camera.

[0052] The atmospheric scattering correction term is determined based on the current environmental parameters, and the straight-line distance between the target and the robot is calculated.

[0053] Calculate the actual size of the target based on the target pixel coordinates and camera intrinsic parameters in the labeled full-scale inspection photos; calculate the horizontal and vertical angles based on the deviation angle between the target center pixel coordinates and the center coordinates of the labeled full-scale inspection photos.

[0054] Construct three-dimensional constraints, including shooting angle within the horizontal and vertical ranges, image clarity greater than or equal to a preset clarity threshold, and target key feature point coverage greater than or equal to a preset coverage threshold.

[0055] Specifically, the steps for generating the association mapping table include:

[0056] Valid inspection photos that simultaneously meet the three-dimensional constraints are selected, and the improved ORB algorithm is used to extract and match feature points of the valid inspection photos to calculate image similarity.

[0057] Automatically delete duplicate photos with image similarity greater than or equal to a preset similarity threshold, and combine the straight-line distance, the actual size of the target, the horizontal angle and the vertical angle to mark all valid inspection targets and generate a set of valid inspection targets;

[0058] Based on the effective inspection target set and inspection route, an inspection and shooting task list is generated. The robot is controlled to call the initial gimbal attitude parameters in the inspection and shooting task list during the inspection process and shoot the effective inspection targets in sequence.

[0059] If the effective inspection target is outside the preset photo center area, the attitude compensation amount is calculated using the bat algorithm based on real-time environmental parameters, and the gimbal direction is adjusted and the photo is retaken using the 3D positioning algorithm until the effective inspection target is within the preset photo center area.

[0060] Specifically, the steps for generating the association mapping table also include:

[0061] If the actual size of the target is smaller than the preset small target threshold, the target is determined to be a small target; otherwise, it is determined to be a large auxiliary target.

[0062] For small targets, extract the contextual features of large auxiliary targets in the captured photos, locate the candidate region of the small target using the sliding window method, perform double interpolation magnification, extract the feature points of the small target and match them with the preset target template, and calculate the matching recognition rate.

[0063] If the matching recognition rate is greater than or equal to the preset matching recognition threshold, the recognition is considered successful. The gimbal angle and shooting position are adjusted by 3D positioning to place the small target in the center of the photo and take a clear photo of the small target.

[0064] For each valid inspection target, a hash mapping is used to generate an association mapping table, which is then imported into the robot control system. Based on the parameters collected in real time, the system automatically matches the configuration parameters of the optimal inspection step size and the optimal gimbal angle from the association mapping table to complete the automatic configuration of robot inspection.

[0065] The beneficial effects of this invention are:

[0066] 1. This invention significantly improves the image quality and target recognition reliability of outdoor inspections through the collaborative optimization of environmental level determination and dynamic shooting parameters. First, environmental levels are classified based on point cloud data and image texture entropy to select suitable charging areas and inspection routes, avoiding highly complex and interfering environments. Then, through three-dimensional triggering conditions of light intensity, device spacing, and visibility, the shooting step size, exposure parameters, and compensation strategies are dynamically adjusted to specifically address image blurring caused by variable outdoor lighting and occlusion. Combining binocular vision ranging and angle compensation technology, a unified target selection and annotation standard is established. Through deduplication, small target magnification, bat algorithm calculation of posture compensation, and 3D positioning adjustment, clear imaging and accurate target selection are ensured, improving the accuracy of target recognition and guaranteeing the precision of inspections in complex outdoor scenarios.

[0067] 2. This invention achieves fully automated configuration of outdoor inspection robots, effectively reducing manual intervention costs and improving inspection collaboration efficiency. It uses the A* algorithm to plan the optimal inspection route covering charging stations, constructs a 3D electronic map integrating environmental level and equipment location, and uses an association mapping table to automatically match parameters such as inspection step length and gimbal angle. A labeling mechanism combining initial screening with YOLOv8n model and manual review generates a unified set of corrected label targets. Combined with an improved ORB algorithm and feature fusion technology, it enhances the consistency of target matching and bounding box selection. Dynamic collaboration rules adaptively adjust inspection behavior based on real-time environmental parameters, ensuring stable robot operation in different environments, reducing the workload of manual route planning, parameter adjustment, and label correction, and improving the stability and efficiency of long-term outdoor inspection operations. Attached Figure Description

[0068] Figure 1 A schematic diagram of an automatic configuration method for an outdoor inspection robot;

[0069] Figure 2 This is a flowchart of the dynamic collaboration rules triggered in this invention;

[0070] Figure 3 This is a flowchart illustrating the generation of the corrected annotation target set in this invention;

[0071] Figure 4 This is a flowchart for determining the consistency of target selection in this invention. Detailed Implementation

[0072] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0073] Example 1

[0074] This embodiment describes an automatic configuration method for an outdoor inspection robot, including the following steps:

[0075] The length of roads in the outdoor inspection area is measured and an electronic map containing charging point markings and default inspection routes is drawn. The electronic map information is then input into the robot control system for outdoor inspection. The robot is equipped with two cameras, including a gimbal camera and a fixed camera. The parameters of the two cameras are the same, and the line connecting the two cameras is perpendicular to the robot's forward direction, at the same height, and arranged side by side.

[0076] The robot is controlled to walk along the planned route from the starting point. The dual cameras maintain a fixed elevation angle perpendicular to the direction of travel. It stops at fixed steps and takes pictures of the inspection equipment on one side of the route, simultaneously recording the current road number, step length value, and corresponding photo number. After the pictures are taken, all the pictures are manually checked, various inspection targets are selected and labeled. Small targets need to be selected simultaneously for their corresponding large auxiliary targets. SIFT operation is performed on the selected targets to extract feature points. The SIFT algorithm is used to match target features in all inspection pictures and complete unified selection and labeling. The full set of inspection pictures is obtained from the inspection pictures that have been manually checked and preliminarily selected and labeled.

[0077] The labeled photos are input into the recognition model for training. The trained model is then used to re-identify all the labeled photos. The model's recognition results are compared with the SIFT algorithm's annotation results. Inconsistencies are manually identified and erroneous annotations are corrected. The corrected data is then used to retrain the model until the model's recognition results are completely consistent with the labeled data. A binocular vision formula is used on all the labeled photos to calculate the actual size, distance from the robot, and angle of each inspection target. Photos are filtered based on the target's shooting angle, and duplicate photos of the same target within the same step length range are removed. Finally, all valid inspection targets are labeled.

[0078] Based on the inspection route and target location, control the gimbal camera to take pictures of the targets in sequence; if the target is outside the center area of ​​the photo, activate the 3D positioning function to adjust the camera angle to center the target; if it is a small target, first zoom in on the designated area of ​​the large auxiliary target where the small target is located to identify the small target, and then adjust it to the center of the photo through 3D positioning, and simultaneously record the step length, gimbal angle and photo position information to complete the robot's automatic configuration.

[0079] Example 2

[0080] refer to Figures 1 to 4 As shown, as an improvement to Embodiment 1, an automatic configuration method for an outdoor inspection robot includes the following steps:

[0081] Point cloud data of the outdoor inspection area is collected using LiDAR to obtain key parameters and is preprocessed, including noise removal through voxel grid filtering and statistical outlier removal algorithms. Combined with environmental texture images collected by dual cameras, the image texture entropy and obstacle density are calculated after preprocessing to determine the environmental level. A 3D electronic map containing charging position coordinates, default inspection route and environmental level is constructed. Environmental parameters, such as light intensity, are collected by environmental sensors, and dynamic coordination rules between environmental parameters and inspection behavior parameters are established. Key parameters include road length, slope, curvature and obstacle distribution. Environmental sensors include, but are not limited to, photosensors, temperature and humidity sensors. Inspection behavior parameters include stop shooting step length and camera exposure parameters.

[0082] The robot control center receives dynamic coordination rules and controls the robot to walk along the planned route on the 3D electronic map from the starting point. The dual cameras maintain a fixed elevation angle. Based on the collected real-time environmental parameters, the dynamic coordination rules are triggered, and the stopping shooting step size and camera exposure parameters are adjusted to ensure optimal shooting conditions. After the robot stops, it takes pictures of the inspection equipment on both sides of the route through the dual cameras. According to the parallax principle of the dual cameras, the invalid images that are blurred or occluded are initially filtered out. The initially valid images are retained and input into the trained YOLOv8n target detection model for AI screening. Suspected inspection targets are automatically selected, labeled, and the initial screening confidence level is output to determine the initial screening confidence level, including first-level confidence, second-level confidence, and third-level confidence. The initial screening result table is generated by combining the initial screening confidence level. Only the second-level confidence items are manually reviewed to correct the misselection, omission, and labeling errors in the initial screening result table and generate a corrected labeled target set.

[0083] The corrected bounding boxes are subjected to environment-adaptive SIFT operations. An environment weighting factor is introduced to correct the feature point extraction weights, and the shooting angle deviation of the feature points is compensated according to the slope data. The environment-adaptive SIFT algorithm is used to match the target features in the full set of inspection photos, and uniform bounding box annotation is performed to generate a full set of uniformly annotated photos. An inspection recognition model is constructed to learn the recognition adaptation rules in different environments, output the bounding box coordinates, target category and target confidence, and calculate the bounding box overlap to determine the consistency of the target bounding box. If the consistency is not met, the inspection recognition model is updated.

[0084] An optimized binocular vision formula, which integrates camera and environmental parameters, is used to calculate 3D parameters for all labeled inspection photos. Atmospheric scattering correction terms are introduced to calculate these parameters, including the actual size of the target, the straight-line distance between the target and the robot, and the horizontal and vertical angles. 3D constraints are constructed, including shooting angle, photo clarity, and target feature integrity. Valid inspection photos are then selected, and an improved ORB algorithm is used to calculate image similarity within the same step length range. Photos of the same target with similarity exceeding a similarity threshold are automatically removed, and all valid inspection targets are labeled. Based on the inspection route, target position, and real-time environmental parameters, the gimbal camera is controlled to sequentially capture images of valid inspection targets. If a valid inspection target exceeds the center area of ​​the photo, the bat algorithm is used to calculate and adjust the 3D positioning posture compensation based on real-time environmental parameters to avoid positioning deviations caused by environmental interference. For small targets, candidate regions are located based on the contextual features of large auxiliary targets, and then region magnification and recognition are performed. After successful recognition, the target is adjusted to the center of the photo using 3D positioning. Step length, gimbal angle, photo position, and environmental parameters are recorded, generating a mapping table between the environment and configuration parameters to complete the robot's automatic configuration.

[0085] Specifically, the steps for establishing dynamic coordination rules between environmental parameters and inspection behavior parameters include:

[0086] LiDAR is used to collect point cloud data of the outdoor inspection area to obtain key parameters, including road length, slope, curvature and obstacle distribution. Noise is removed by voxel grid filtering and statistical outlier removal algorithm. Dual cameras are used to collect environmental texture images and Gaussian filtering is used to remove image noise.

[0087] For each denoised environmental texture image, it is converted into a single-channel grayscale image and subjected to grayscale compression. The calculation step size and calculation window of the grayscale co-occurrence matrix are set. , , Four computational directions are used to traverse the image pixels and count the occurrences of adjacent gray-level value pairs in corresponding directions. After normalizing the frequency matrix, a gray-level co-occurrence matrix is ​​obtained. Based on the gray-level co-occurrence matrix, the texture entropy in each of the four directions is calculated. The arithmetic mean of the texture entropy in the four directions is used to obtain the initial image texture entropy of the environment texture image. The average of the initial image texture entropy is taken as the image texture entropy. Set the texture entropy threshold based on historical inspection scene data statistics. , To determine texture complexity;

[0088] like If the texture complexity is low, then the texture complexity is determined to be low; if If the texture complexity is determined to be medium; if If so, the texture complexity is determined to be high;

[0089] By mapping the acquisition time and spatial coordinates of the LiDAR point cloud with the shooting time and imaging field of view of the dual-camera images, a data association index table is established. Based on the data association index table, obstacle point clouds in the corresponding regions of each denoised environmental texture image are matched. The effective perception area is determined with the image shooting range as the boundary. The number of obstacle point clouds in the effective perception area is counted, and the obstacle density is calculated by combining the area of ​​the effective perception area. Density thresholds are set based on historical obstacle distribution statistics. , To determine the obstacle density level;

[0090] like If the obstacle density level is low, then the obstacle density level is determined to be low; if If the obstacle density level is determined to be medium; if If so, the obstacle density level is determined to be high;

[0091] For example, for each denoised environmental texture image, it is converted into a single-channel grayscale image and subjected to 16 levels of grayscale compression processing. The calculation step size of the gray-level co-occurrence matrix is ​​set to 1 pixel, and the calculation window is 512×512 pixels. , , Four computational directions are used to traverse the image pixels and count the occurrence frequency of adjacent gray-level value pairs in corresponding directions. After normalizing the frequency matrix, a gray-level co-occurrence matrix is ​​obtained. Based on the gray-level co-occurrence matrix, the texture entropy in each of the four directions is calculated. The texture entropy in the four directions is then arithmetically averaged to obtain the initial image texture entropy of the environment texture image. The average of the initial image texture entropy of all 24 images, which is 0.85, is taken as the image texture entropy. Based on the statistical analysis of historical inspection scene data, which shows that image texture entropy less than 0.8 indicates low texture complexity, between 0.8 and 1.5 indicates medium texture complexity, and greater than 1.5 indicates high texture complexity, a texture entropy threshold is set. , To determine texture complexity; because If so, the texture complexity is determined to be medium;

[0092] By mapping the acquisition time and spatial coordinates of the LiDAR point cloud with the shooting time and imaging field of view of the dual-camera images, a data association index table is established. Based on the data association index table, obstacle point clouds corresponding to the region of each denoised environmental texture image are matched. According to the transformation relationship between the imaging field of view of the dual-camera images and the spatial coordinates of the LiDAR point cloud, the pixel boundary of the imaging field of view is transformed by perspective projection using the camera intrinsic and extrinsic parameters, and mapped to the three-dimensional spatial boundary in the geographic coordinate system of the inspection scene. The image shooting range is determined as the projection area of ​​the imaging field of view in three-dimensional space. With the image shooting range as the boundary, the effective perception area area is determined. The number of obstacle point clouds within the effective sensing area is 32. Based on the area of ​​the effective sensing area, the obstacle density is calculated to be... When the historical obstacle distribution statistics indicate a low obstacle density level When the obstacle density level is medium When the obstacle density level is high Set the density threshold to , To determine the obstacle density level; because If the obstacle density level is determined to be medium, then the obstacle density level is determined to be medium.

[0093] The environment level is determined based on texture complexity and obstacle density level. If both texture complexity and obstacle density level are high, it is classified as a level 3 environment; if both texture complexity and obstacle density level are low, it is classified as a level 1 environment; otherwise, it is classified as a level 2 environment.

[0094] Based on environmental level selection, candidate charging areas are chosen, prioritizing Level 1 environmental areas with low texture complexity and low obstacle density, while avoiding Level 3 environmental areas. Level 2 environmental areas are selected from smooth road sections; specifically, in this embodiment, smooth road sections are defined as having a slope of less than 2. And the curvature is less than 0.01 ;

[0095] The number of obstacle point clouds in each charging candidate area was counted, based on the physical space required for the charging robot to safely dock (length 1.8). Width 1.2 0.6 meters high With point cloud resolution A threshold of 40 was set to filter charging candidate regions where the number of obstacle point clouds was less than the threshold. For the selected charging candidate regions, LiDAR was used to process them according to a 5-10... Sampling interval, coverage area boundary extension 1 The system collects point cloud data of the candidate charging areas, uses the 3D coordinates of the point cloud cluster centers to determine the charging location coordinates, associates and labels the corresponding environmental level, and labels the charging location attributes, such as the starting charging location and the intermediate replenishment charging location. The point cloud data of the LiDAR after noise reduction is stitched together with the ICP algorithm to form a full-domain point cloud dataset covering the entire outdoor inspection area. The system plans the default inspection route based on the A* algorithm, prioritizing the passage through the first-level environmental road section and avoiding the third-level environmental road section, and passing through the charging locations of all selected charging candidate areas. The environmental level, charging location and inspection route are added to the full-domain point cloud dataset to construct a 3D electronic map containing charging location coordinates, default inspection route and environmental level.

[0096] Environmental parameters such as light intensity and visibility in the inspection area are collected in real time by environmental sensors. Combined with environmental level and inspection behavior parameters, dynamic coordination rules between environmental parameters and inspection behavior are established. For example, in Level 1 environments, if the light intensity is below 200 Lux, a supplementary light is triggered, and the shooting stops with a step size of 2. The camera exposure parameters are set to automatic mode with HDR enabled; in secondary ambient light conditions, if the light intensity is below 500 Lux, the fill light is triggered and the stop-shoot step is shortened to 1.5. The camera exposure parameters are set to manual mode, shutter priority; in Level 3 environment, there is no limit to light intensity and the fill light and infrared auxiliary illumination are enabled, with the stop-shoot step shortened to 0.5 seconds. The camera exposure parameters are set to a fixed high ISO and noise reduction mode is enabled. In this embodiment, the fixed high ISO is set to ISO 3200.

[0097] Specifically, the steps for generating the corrected annotation target set include:

[0098] The robot control center receives dynamic coordination rules and controls the robot to move from the starting point along the planned route on the 3D electronic map. It uses GPS and IMU fusion positioning to provide real-time feedback of the robot's current position coordinates; the dual-camera fixed elevation angle is set to 30 degrees. Ensure that the shooting angle covers the key areas of the inspected equipment, such as equipment bolts and wiring terminals;

[0099] Based on real-time collected environmental parameters and the distance between the robot and equipment on both sides of the route, a three-dimensional triggering condition for dynamic collaborative rules is constructed. In the illumination dimension, the illumination threshold is set to 150 Lux based on the minimum illumination requirement for clear identification of surface details on the inspected equipment. When the illumination intensity is less than or equal to the illumination threshold, exposure parameter optimization is initiated. In the equipment distance dimension, the distance threshold is set to 3.0 based on the maximum effective distance for clearly capturing equipment details under the principle of dual-camera parallax. When the distance between devices is less than or equal to the distance threshold, the robot will stop shooting by default, with a step size of 0.8. Reduced to 0.4 To increase the density of close-up shooting locations; in terms of visibility, the visibility threshold is set to 500 based on the minimum visibility standard of no fog and no dust blurring. When visibility is less than or equal to the visibility threshold, increase exposure compensation;

[0100] If all three trigger conditions are met, the dynamic coordination rule is triggered, and the shooting is performed using a shortened stop shooting step size and optimized exposure parameters to ensure optimal shooting conditions; if any single dimension or any two dimensions trigger conditions are met, the adjustment rule corresponding to the met dimension is executed; otherwise, no parameter adjustment is performed.

[0101] The robot continues to walk along the planned route according to the adjusted stop-shooting step length. It performs real-time positioning calibration by combining GPS / IMU fusion positioning data with the route coordinates of the 3D electronic map. When the positioning data shows that the robot has reached the shooting point, the robot stops walking, and the dual cameras load the optimized exposure parameters and maintain a 30° exposure. With a fixed elevation angle, the robot synchronously photographs the inspection equipment on both sides of the route. After the photographing is completed, the robot continues to move to the next photographing point. The GPS / IMU fusion positioning data is obtained by fusing position and velocity data collected in real-time by the robot's GPS receiver and three-axis angular velocity and acceleration data output in real-time by the IMU sensor, using an extended Kalman filter algorithm. In this embodiment, the time difference between the two cameras is set to be less than or equal to 10 seconds. ;

[0102] For each set of images captured by the dual cameras, a semi-global block matching algorithm is used. Based on the baseline distance between the dual cameras and the shooting distance of the inspection equipment, a reasonable disparity search range of 0-64 pixels is set. The matching cost for corresponding pixels in the left and right images is calculated using fixed-size image blocks. Dynamic programming is used to aggregate the matching costs across multiple path directions to obtain the globally optimal matching result. The disparity value of each pixel is determined to generate a dual-camera disparity map. The arithmetic mean of the disparity values ​​of all pixels in the dual-camera disparity map is calculated. The square of the difference between each pixel's disparity value and the arithmetic mean is calculated, and the arithmetic mean of all squared values ​​is obtained to obtain the disparity map variance. Based on the disparity stability requirements for clear image imaging, a variance threshold of 8.0 pixels is set. 2 If the variance of the disparity map is greater than the variance threshold, the image is judged to be blurry; otherwise, the image is judged to be clear.

[0103] The depth abrupt change features of the device are extracted from the dual-camera disparity map. Specifically, the depth difference between adjacent pixels in the dual-camera disparity map is calculated, based on the depth change corresponding to the device's minimum occlusion size. Set the depth threshold to Pixels with depth differences exceeding a depth threshold are marked as candidate mutation points, and the contour features formed by the continuous distribution of candidate mutation points are defined as depth mutation features. Based on the depth mutation features, occlusion regions are identified and the occlusion area is calculated. According to the requirement that the key areas of the device be completely visible without any obstructions, an area threshold of 20 is set to ensure the key areas of the device are fully visible and unobstructed. If the occlusion area is greater than or equal to the area threshold, it is determined to be occlusion; otherwise, it is determined to be no occlusion.

[0104] Clear and unobstructed dual-camera images are initially retained and integrated into a preliminary effective image set. This preliminary effective image set is then input into the trained YOLOv8n object detection model for AI initial screening.

[0105] The system matches the inspection equipment type based on the category ID, automatically selects suspected inspection targets, labels their type, and outputs the initial screening confidence score. Set grading thresholds according to the target detection credibility grading standard. , To classify the initial screening confidence level, if If it is, then it is determined to be a level 1 confidence level; if If it is, then it is determined to be a level 2 confidence level; if If so, it is determined to be a level three confidence level;

[0106] For example, the system matches the inspection equipment type according to category ID, where category ID1 corresponds to distribution boxes, ID2 to cable joints, ID3 to insulators, and ID4 to grounding stakes. Suspected inspection targets are automatically selected and their types are labeled, and the initial screening confidence level is output. Set grading thresholds according to the target detection credibility grading standard. , This is used to classify the initial screening confidence level; for example, when a suspected target matching category ID2 is found, the suspected target is automatically selected and labeled as a cable connector, and the initial screening confidence level is output. ,satisfy If the match is within the second level of confidence, it is determined to be a level 2 confidence level; when a suspected target of category ID1 is matched, the suspected target is automatically selected and marked as a distribution box, and the initial screening confidence level is output. ,satisfy If the match is positive, it is determined to be a level 1 confidence level; when a suspected target with category ID3 is matched, the suspected target is automatically selected and marked as an insulator, and the initial screening confidence level is output. ,satisfy If so, it is determined to be a level three confidence level.

[0107] Based on the image IDs, the detection results are integrated to generate a preliminary screening result table, including image ID, shooting point coordinates, target coordinates, inspection equipment type, preliminary screening confidence level, and preliminary screening confidence grade. Since the detection accuracy of level 1 confidence targets is high, no manual review is required. Level 3 confidence targets have low credibility and are directly judged as invalid targets and removed without manual review. Only level 2 confidence items are selected from the preliminary screening result table for manual review, effectively reducing the workload of manual review and annotation costs. The preliminary screening result table is manually corrected for misselected, missing, and labeled errors, including deleting target boxes of non-inspection equipment, manually drawing target boxes to supplement the annotation of undetected inspection equipment, and modifying the inspection equipment type annotation, generating a corrected labeled target set.

[0108] Specifically, the steps for determining the consistency of target selection include:

[0109] The corrected bounding boxes are obtained from the corrected labeled target set. The corrected bounding boxes are associated with the corresponding photos and environmental parameters according to the image ID. The number of successfully matched feature points under different light intensities, equipment spacing and visibility conditions in the historical inspection data is statistically analyzed. Multiple linear regression is used to fit the influence coefficient of each environmental factor on the number of feature points extracted. Each influence coefficient is normalized to obtain the corresponding weight value, ensuring that the sum of the weight values ​​is 1, so as to quantify the influence weight of each environmental parameter on feature point extraction, including light intensity weight, equipment spacing weight and visibility weight.

[0110] Min-max normalization is used to map light intensity, equipment spacing, and visibility to light intensity value, equipment spacing value, and visibility value, respectively. Combining light intensity weight, equipment spacing weight, and visibility weight, the environmental weighting factor is calculated as: Environmental weighting factor = Light intensity value × Light intensity weight + Equipment spacing value × Equipment spacing weight + Visibility value × Visibility weight.

[0111] Based on a fixed elevation angle of dual cameras, and incorporating the slope from key parameters, the shooting angle deviation of feature points is compensated, and the actual shooting angle of the target is calculated using trigonometric functions. Obtain the actual height of the target from the corrected labeled target set. Dual-camera focal length is obtained through camera calibration. With image scaling factor Obtain device spacing from environmental parameters The angle deviation is calculated based on the actual shooting angle of the target and the fixed elevation angle of the dual cameras, taking into account the device spacing. With image scaling factor The product of the two, combined with the actual height of the target. Calculate the image offset coefficient corresponding to a unit angle, using the angle deviation value and the focal length of the dual cameras. Calculate the angular deviation compensation amount using the image offset coefficient. The expression is as follows:

[0112]

[0113] Compensation amount based on angle deviation Adjust the y-axis coordinates of the target bounding box to obtain the corrected angle-compensated target bounding box coordinates, ensuring that the feature point extraction area is aligned with the actual position of the target, and then generate the corrected angle-compensated target area image;

[0114] For the corrected angle-compensated target area image, environment-adaptive SIFT operation is used to extract feature points. Neighborhood gradient statistics and direction normalization are performed on the feature points to form a 128-dimensional SIFT feature descriptor. This descriptor is then categorized according to the inspection target type to generate a template feature vector library. For the target area of ​​all inspection photos, the original feature point response values ​​are extracted. The corrected feature point response values ​​are obtained by multiplying the environment weighting factor by the original feature point response values ​​to enhance the saliency of weak feature points in low-quality environments. Matching feature vectors are extracted using the same SIFT parameters. The FLANN matching algorithm is used to perform feature matching between the template feature vector and the matching feature vector and calculate the matching distance. This includes: extracting the values ​​of each dimension of the template feature vector and the matching feature vector, calculating the sum of squared differences of the corresponding dimension values, and taking the square root to obtain the matching distance. A distance threshold of 50 is set according to the target feature stability requirements. If the matching distance is less than or equal to the distance threshold, the target area to be matched is determined to match the template target; otherwise, it is determined not to match.

[0115] The matching score is obtained by comparing the number of matched feature points with the total number of template feature points in the template feature vector library. The ratio of the number of matched feature points to the total number of template feature points is calculated. The score threshold is set to 0.6 based on the minimum feature point coverage requirement for effective matching. If the matching score is greater than or equal to the score threshold, it is determined to be a valid match and the corresponding feature point pair is retained; otherwise, it is determined to be an invalid match and marked as an unidentified target.

[0116] For effectively matched feature point pairs, the least squares method is used to fit the target contour. Combined with the corrected bounding box coordinates, the targets in the full inspection photos are uniformly selected and labeled to generate a full set of uniformly labeled photos.

[0117] Based on a fully unified labeled photo set, environmental parameters, and feature fusion requirements, an inspection and recognition model is constructed, including an input layer, a feature fusion layer, a feature decision layer, and an output layer. The feature fusion requirement is to perform weighted fusion of image features and environmental features to suppress the interference of changes in illumination, angle, and distance on feature representation and improve the accuracy and robustness of target localization and classification in complex inspection scenarios.

[0118] The input layer is used to receive image features and environmental features; the image features are extracted from the RGB images of all uniformly labeled photos, and the environmental features are extracted from environmental parameters.

[0119] The feature fusion layer is used to perform weighted fusion of image features and environmental features through a multi-head self-attention mechanism, learn the adaptive influence of environmental parameters on the recognition results, and generate a fused feature vector of image and environment.

[0120] The feature decision layer is used to receive the fused feature vector and extract key features, such as the edge structure features of the inspection equipment, through three convolutional layers. According to the number of target types and the accuracy requirements of the bounding box in the inspection scenario, regression branches and classification branches are set. The regression branch is used to fit the target bounding box coordinates, and the classification branch is used to determine the target category and recognition reliability.

[0121] The output layer is used to output the bounding coordinates of the target, the specific target category, and the target confidence level;

[0122] Using a fully uniformly labeled set of photos as the training set, the target classification loss and bounding box regression loss are used as the total loss function. The AdamW optimizer is combined to train the inspection and recognition model. After training, the bounding box coordinates, target category and target confidence are output.

[0123] For the same target in each photo, the overlap between the bounding box coordinates output by the inspection and recognition model and the uniformly labeled bounding box coordinates is calculated. The target bounding box consistency is determined by combining the target confidence score. An overlap threshold is set according to the target detection industry standard and inspection accuracy requirements, and a confidence threshold is set according to the model recognition reliability requirements. If the bounding box overlap is greater than or equal to the overlap threshold and the target confidence score is greater than or equal to the confidence threshold, the bounding box consistency is determined to be up to standard. Otherwise, the bounding box consistency is determined to be down to standard. The down to standard samples are screened, merged with the corrected labeled target set, and then incrementally trained on the inspection and recognition model to update the inspection and recognition model.

[0124] For example, an inspection identification model is constructed based on a complete set of uniformly labeled photos containing 2,000 outdoor inspection equipment photos, covering three core inspection targets: distribution boxes, cable joints, and insulators, as well as environmental parameters including three-dimensional environmental features such as light intensity, visibility, and equipment spacing, and feature fusion requirements. The model includes an input layer, a feature fusion layer, a feature decision layer, and an output layer.

[0125] The input layer is used to receive image features and environmental features; MobileNetV2 is used to extract 1280-dimensional image features from the RGB images of all uniformly labeled photos, and normalized 3-dimensional environmental features are extracted from the environmental parameters.

[0126] The feature fusion layer is used to perform weighted fusion of image features and environmental features through a multi-head self-attention mechanism with 4 attention heads, learn the adaptive influence of environmental parameters on the recognition results, and generate a fusion feature vector of image and environment with a dimension of 512.

[0127] The feature decision layer is used to receive the fused feature vector. It extracts key features, such as the edge structure features of the inspection equipment, through three convolutional layers with a kernel size of 3×3 and output channels of 256, 128 and 64 respectively. The regression branch and the classification branch are set according to the number of target types and the accuracy requirements of the bounding box in the inspection scenario. The regression branch is used to fit the target bounding box coordinates and outputs 4-dimensional coordinates as the upper left x / y and the lower right x / y. The classification branch is used to determine the target category and the recognition reliability.

[0128] The output layer is used to output the bounding coordinates of the target, the specific target category, and the target confidence level;

[0129] Using a fully uniformly labeled set of photos as the training set, the object classification loss and bounding box regression loss are combined into a total loss function. The target classification loss expression is: , , For the number of target categories, For real category labels, The bounding box regression loss expression is used to express the class probabilities predicted by the inspection and identification model. ,in, These are the center coordinates of the bounding box. and width ,high , Smoothing for bounding box regression loss function The bounding box parameters predicted by the inspection and identification model. The parameters are the true bounding boxes, with the weights of the two loss classes being 0.5 and 0.5 respectively. The inspection and recognition model is trained using the AdamW optimizer with a learning rate of 1e-5 and a weight decay coefficient of 0.001. Training is completed after 50 epochs, outputting bounding box coordinates, target category, and target confidence. For the same target in each image, the overlap between the bounding box coordinates output by the inspection and recognition model and the uniformly labeled bounding box coordinates is obtained by calculating the ratio of the intersection area to the union area of ​​the model's predicted bounding box and the manually labeled bounding box. The target bounding box consistency is determined by combining this with the target confidence. Based on industry standards for target detection and inspection accuracy requirements, an overlap threshold of 0.75 is set, and a confidence threshold of 0.85 is set based on model recognition reliability requirements. If the bounding box overlap is greater than or equal to 0.75 and the target confidence is greater than or equal to 0.85, the target bounding box consistency is considered satisfactory; otherwise, the bounding box consistency is considered unsatisfactory.

[0130] For example, the coordinates of the manually labeled "distribution box" in the target photo are (100, 80, 200, 180), and the coordinates of the box output by the inspection recognition model are (105, 85, 198, 175). The calculated intersection area coordinates are (105, 85, 198, 175), and the intersection area = (198-105) × (175-85) = 93 × 90 = 8370 pixels. 2 Union area = (200-100)×(180-80)+(198-105)×(175-85)-8370=100×100+8370-8370=10000 pixels 2 The overlap of the selected boxes is 8370 ÷ 10000 = 0.837. The target confidence score of the distribution box output by the inspection and recognition model is 0.92. Since 0.837 ≥ 0.75 and 0.92 ≥ 0.85, the target selection consistency is deemed to meet the standard. In another target photo, the manually labeled "cable joint" has selection coordinates of (300, 250, 350, 300). The selection coordinates output by the inspection and recognition model are (310, 260, 360, 310). The intersection area is calculated to be 40 × 40 = 1600 pixels. 2The area of ​​the union of the sets = 50×50 + 50×50 - 1600 = 2500 + 2500 - 1600 = 3400 pixels 2 The bounding box overlap is approximately 0.471 (1600 ÷ 3400). The target confidence score is 0.78. Since 0.471 < 0.78, the threshold requirement is not met, and the bounding box consistency is deemed unsatisfactory. The 200 unsatisfactory samples are screened and merged with the corrected labeled target set of 50 supplementary labeled samples. The inspection recognition model is then incrementally trained for 10 epochs, with the learning rate adjusted to 5e-6, and the inspection recognition model is updated.

[0131] Specifically, the steps for generating the mapping table between the environment and configuration parameters include:

[0132] For the labeled full-volume inspection photos, based on the binocular vision ranging formula that integrates camera parameters and environmental parameters, an atmospheric scattering correction term is fitted using the current environmental parameters and incorporated into the binocular vision formula to obtain an optimized binocular vision formula. ,in, The binocular baseline distance, Focal length For parallax, This is the scattering correction factor. This is the theoretical value for binocular visual ranging without incorporating atmospheric scattering correction. The atmospheric scattering correction term is calculated by matching the left and right views of the binocular camera to determine the target disparity. The value of the atmospheric scattering correction term is determined by combining the current environmental parameters. The straight-line distance between the target and the robot is calculated by combining the optimized binocular vision formula.

[0133] Based on camera imaging principles, and combining the pixel coordinates of the target in the labeled full-scale inspection photos with camera intrinsic parameters, the target pixel size is extracted. Substitution Calculate the actual size of the target; calculate the horizontal and vertical angles based on the deviation angle between the target center pixel coordinates and the center coordinates of the marked full inspection photos, including: calculating the difference between the horizontal coordinates of the target center and the horizontal coordinates of the photo center to obtain the first pixel deviation, calculating the ratio of the camera's horizontal field of view to the total number of horizontal pixels to obtain the single-pixel horizontal angle, and multiplying the first pixel deviation by the single-pixel horizontal angle to obtain the horizontal angle; calculate the difference between the vertical coordinates of the target center and the vertical coordinates of the photo center to obtain the second pixel deviation, divide the camera's vertical field of view by the total number of vertical pixels to obtain the single-pixel vertical angle, and calculate the product of the second pixel deviation and the single-pixel vertical angle to obtain the vertical angle;

[0134] Construct three-dimensional constraints including shooting angle, image sharpness, and target feature integrity, including the shooting angle in the horizontal direction. Within range and vertical Within the specified range, provided the image clarity is greater than or equal to a clarity threshold and the target key feature point coverage is greater than or equal to a coverage threshold, the target parameters of each labeled full-scale inspection photo are compared with the three-dimensional constraints. Labeled full-scale inspection photos that simultaneously meet the three-dimensional constraints are selected and defined as valid inspection photos. In this embodiment, the following settings are provided: , Based on the historical average resolution of 35 pixels and the minimum resolution requirement for target recognition of 25 pixels, the weight of the average resolution is set to 0.6 according to the reliability priority of historical effective inspection data, and the weight of the minimum resolution requirement for target recognition is set to 0.4 according to the minimum accuracy requirement for target recognition. The resolution threshold is determined to be 31 pixels by 0.6×35+0.4×25. The target key feature point coverage rate is obtained by calculating the ratio of the number of target key feature points covered to the total number of target key feature points. The minimum effective target recognition feature coverage rate verified by historical inspection data is 75%. After adding 5% to the target feature redundancy recognition requirement, the coverage rate threshold is determined to be 80%. Among them, the number of target key feature points covered is the number of target key feature points successfully detected in the current inspection photo, and the total number of target key feature points is determined based on the experience of experts in historical effective inspections.

[0135] In the feature point extraction stage of the ORB algorithm, Gaussian blur preprocessing is added to reduce noise interference. The matching strategy of feature point descriptors is optimized by KD tree. Hamming distance combined with RANSAC algorithm is used to remove mismatched pairs. Valid inspection photos are grouped according to the robot's stopping step length. For photos within the same step length group, the improved ORB algorithm is used to calculate image similarity. The calculation process includes: applying the standard deviation parameter to the valid inspection photos within the same step length group. Gaussian blur preprocessing was performed, and feature points were extracted and 512-dimensional binary descriptors were generated by improving the ORB algorithm. Taking the first photo in the group as a reference, a fast retrieval index was built using a KD tree for the descriptors. The descriptors of the remaining photos in the group were traversed and candidate matching sets were retrieved. The Hamming distance between descriptors in the candidate matching set was calculated. Based on the distribution characteristics of the Hamming distance of the 512-dimensional binary descriptors, the statistical mean of the Hamming distance of historical effective matching pairs (48), and the allowable distance deviation of less than or equal to 16 under noise interference, a Hamming distance threshold of less than or equal to 64 was set to screen preliminary matching pairs. The RANSAC algorithm was used to fit the fundamental matrix to eliminate erroneous matching pairs and obtain high-precision matching pairs. The ratio of the number of high-precision matching pairs to the total number of feature points in the reference photo was calculated, and the image similarity was obtained by weighting the ratio with the normalized value of the average Hamming distance of the matching pairs.

[0136] Based on historical similarity statistics of duplicate photos within the same step length range, the average similarity of duplicate photos is 92%, and the average similarity of non-duplicate photos is 78%. The median value between the two is selected, and combined with the requirement of a false deletion rate of less than or equal to 5%, the similarity threshold is set to 85%. Valid inspection photos with image similarity less than the similarity threshold are retained, while duplicate photos with image similarity greater than or equal to the similarity threshold are automatically deleted, resulting in deduplicated valid inspection photos. All valid inspection targets are labeled with straight-line distance, actual target size, and horizontal and vertical angles between the target and the robot, generating a set of valid inspection targets with labeled information. In this embodiment, 20cm is set as a step length group.

[0137] Based on the target location information in the effective inspection target set, the robot control system is input into the inspection route and the initial gimbal attitude parameters corresponding to each target to generate an inspection shooting task list. The robot travels according to the inspection route. After reaching the shooting position of each target, the control system calls the initial gimbal attitude parameters in the inspection shooting task list and controls the gimbal camera to shoot the effective inspection targets in sequence. The pixel coordinates of the target center in the captured photo are compared with the preset photo center area range. If the effective inspection target is outside the preset photo center area range, attitude compensation adjustment is performed. The attitude compensation amount of 3D positioning is calculated and adjusted using the bat algorithm in combination with real-time environmental parameters. For example, when the wind speed is greater than 3m / s, the horizontal attitude compensation amount is increased; when the visibility is less than 500m, the vertical pitch angle compensation amount is adjusted based on the atmospheric scattering correction term. The gimbal horizontal rotation angle and vertical pitch angle are adjusted by combining the 3D positioning algorithm with the attitude compensation amount, and the shooting is repeated until the target is within the preset photo center area range. In this embodiment, the preset photo center area range is determined by the image normalization resolution and the center positioning requirement for clear target recognition. The bat algorithm calculation process includes: initial... The initial bat population was established, with each individual bat corresponding to a set of horizontal rotation and vertical pitch compensation values ​​for the gimbal. The frequency range was set to 20kHz~50kHz based on the actual needs of gimbal attitude adjustment and the algorithm's optimization accuracy requirements. The loudness attenuation coefficient was set to 0.9 based on the algorithm's convergence speed and optimization stability requirements. The pulse emission rate update rule was set to increase with each iteration, with an initial pulse emission rate of 0.1, gradually increasing by a factor of 0.05 after each iteration. The squared Euclidean distance between the current target center pixel coordinates and the preset central area of ​​the image was used as the fitness. During iteration, each bat emitted pulses according to its own frequency and updated its position. The balance between local and global searches was achieved by adjusting the loudness and pulse emission rate. Fitness was evaluated after each iteration. A fitness threshold of 9 pixels was set based on the allowable deviation for center positioning in clear target identification. The iteration terminated when the fitness was less than the fitness threshold, retaining the optimal individual and outputting the optimal horizontal rotation and vertical pitch compensation values ​​as the final attitude compensation values ​​for gimbal attitude compensation.

[0138] Based on the typical size range of common small targets such as bolts in inspection scenarios, and the minimum effective recognition size of the camera image is greater than or equal to Take the upper limit of the typical size range. As a small target threshold, if the actual size of the target is smaller than the small target threshold, the target is determined to be a small target; otherwise, the target is determined to be a large auxiliary target. For small targets, the contextual features of the large auxiliary targets in the captured image are first extracted, such as the texture association between the large auxiliary targets and the small targets. The sliding window method is used to set the sliding window size to 80×80 pixels according to the pixel size corresponding to the small target threshold, based on the window side length. The step size is used to slide through the context feature region around the large auxiliary target row by row and column by column, calculate the similarity between the region within each window and the typical texture features of the small target, and select windows with a similarity greater than or equal to 60% as candidate regions of the small target. The candidate regions are then double-interpolated and enlarged. Among them, the average similarity of correctly located windows based on statistical history is 72%, and the average similarity of incorrectly located windows is 45%. Combining the requirement of a positioning accuracy of greater than or equal to 90% and a false negative rate of less than or equal to 8%, the positioning accuracy and false negative rate corresponding to different similarity thresholds are tested to construct an association fitting curve and select the optimal similarity threshold of 60% that simultaneously meets the requirements of positioning accuracy and false negative rate.

[0139] An improved ORB algorithm is used to extract feature points of small targets and match them with a preset target template. The number of matched feature points is counted against the total number of feature points of the small targets, and the ratio of the number of matched feature points to the total number of feature points of the small targets is calculated to obtain the matching recognition rate. Based on historical accuracy data for small target recognition scenarios, when the matching recognition rate is greater than or equal to 75%, the correct recognition probability of the small targets reaches over 92%. At the same time, a fault-tolerant scenario with partial occlusion of small target feature points is set with a matching recognition threshold of 75%. If the matching recognition rate is greater than or equal to the matching recognition threshold, the recognition is considered successful. The gimbal angle and shooting position are adjusted through 3D positioning to place the small target in the center of the photo and take a clear photo of the small target. The preset target template is obtained by selecting high-definition, feature-complete target photos from various types of effective inspection targets, extracting standard feature point sets, and establishing a template library.

[0140] For each valid inspection target, using environmental parameters as keys and inspection step length, adjusted gimbal angle, and shooting position as values, a hash mapping is used to generate an association mapping table between environmental parameters and configuration parameters. This ensures that the corresponding optimal configuration parameters are quickly matched under different environmental scenarios. The association mapping table is then imported into the robot control system. Based on the parameters collected by the real-time environmental sensors, the system automatically matches the optimal inspection step length and optimal gimbal angle configuration parameters from the association mapping table, thus completing the automatic configuration of the robot inspection.

[0141] In summary, both embodiments revolve around the automatic configuration of outdoor inspection robots. Embodiment 2 is an improvement and optimization of Embodiment 1. Both embodiments follow the entire process of map construction, data processing, target recognition, and automatic configuration, gradually improving the adaptability and accuracy of inspections. Embodiment 1 serves as the basic solution. First, an electronic map containing charging locations and inspection routes is drawn. The robot is equipped with dual cameras that take pictures at fixed steps. After manually labeling target types, the SIFT algorithm is used for unified labeling. The recognition model is trained and iteratively optimized. Target parameters are calculated using binocular vision formulas. After filtering and deduplication, valid targets are obtained. Finally, the camera angle is adjusted through 3D positioning, and the parameters are recorded to complete the automatic configuration. Embodiment 2 further optimizes this by using LiDAR and dual-camera fusion. After denoising preprocessing, image texture entropy and obstacle density are calculated to classify environmental levels and construct a 3D electronic map, establishing dynamic collaborative rules between the environment and inspection behavior. Invalid images are filtered by dual-camera parallax, and a corrected and labeled target set is generated using a trained YOLOv8n model for initial screening and manual verification with secondary confidence. An environment-adaptive SIFT algorithm is introduced, and an inspection recognition model is constructed to optimize feature extraction. An optimized binocular vision formula with atmospheric scattering correction term and an improved ORB algorithm are used to screen for duplicate and valid photos. Small targets are localized and magnified for recognition through large auxiliary targets. An association mapping table between the environment and configuration parameters is generated to achieve automatic matching of robot parameters, effectively adapting to complex outdoor environments and improving inspection accuracy and efficiency.

[0142] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An automatic configuration method for an outdoor inspection robot, characterized in that, include: Collect point cloud data and environmental texture images, preprocess them, calculate image texture entropy and obstacle density to determine the environmental level, construct a 3D electronic map, and collect environmental parameters and combine them with inspection behavior parameters to establish dynamic collaborative rules. The robot walks along the planned route and triggers dynamic collaborative rules based on real-time environmental parameters; it takes pictures of the inspection equipment with dual cameras and performs AI initial screening, automatically selects suspected inspection targets, labels them, and outputs the initial screening confidence level, generates an initial screening result table, and manually reviews and corrects them to generate a corrected labeled target set. The corrected bounding box targets are subjected to environment-adaptive SIFT operation. An environment weighting factor is introduced to correct the feature point extraction weights and compensate for the shooting angle deviation, generating a full set of uniformly labeled photos. An inspection and recognition model is constructed to output bounding box coordinates, target categories and target confidence, and the bounding box overlap is calculated to determine the consistency of target bounding boxes. Three-dimensional constraints are constructed to filter valid inspection photos. An improved ORB algorithm is used to remove photos with the same target and mark valid inspection targets. The bat algorithm is used to calculate and adjust the pose compensation amount for 3D localization; based on the large auxiliary target, the candidate region of small target is located and magnified for recognition, and an association mapping table is generated.

2. The automatic configuration method for an outdoor inspection robot according to claim 1, characterized in that, The specific steps for determining the environmental level include: Point cloud data of outdoor inspection areas are collected to obtain key parameters; environmental texture images are collected; and image texture entropy is calculated after denoising. ; Set texture entropy threshold , ,like If the texture complexity is low, then the texture complexity is determined to be low; if If the texture complexity is determined to be medium; if If so, the texture complexity is determined to be high; Establish a data association index table to determine the effective sensing area and calculate obstacle density. ; Set density threshold , ,like If the obstacle density level is low, then the obstacle density level is determined to be low; if If the obstacle density level is determined to be medium; if If so, the obstacle density level is determined to be high; If the texture complexity is high and the obstacle density level is high, it is determined to be a level 3 environment; if the texture complexity is low and the obstacle density level is low, it is determined to be a level 1 environment; otherwise, it is determined to be a level 2 environment.

3. The automatic configuration method for an outdoor inspection robot according to claim 2, characterized in that, The specific steps for establishing dynamic collaborative rules include: Based on the environmental level, candidate charging areas are selected, with priority given to first-level environmental areas with low texture complexity and low obstacle density, while third-level environmental areas are avoided, and smooth road sections are selected for second-level environmental areas. The number of obstacle point clouds in each charging candidate area is counted, and charging candidate areas with fewer than the judgment threshold are selected. Point cloud data is collected and the coordinates of the charging position are determined. The corresponding environmental level and charging position attributes are associated and labeled. The point clouds of each charging candidate area are stitched together to form a global point cloud dataset. Based on the A* algorithm, a default inspection route is planned, prioritizing the passage through first-level environmental road sections and avoiding third-level environmental road sections. The route also passes through all selected charging candidate areas and charging locations. The environmental level, charging location, and inspection route are added to the global point cloud dataset to construct a three-dimensional electronic map. Collect environmental parameters of the inspection area, and combine the environmental level with the inspection behavior parameters to establish dynamic collaborative rules.

4. The automatic configuration method for an outdoor inspection robot according to claim 3, characterized in that, The specific steps to trigger dynamic collaboration rules include: The system receives the dynamic coordination rules, controls the robot to walk along the planned route in the three-dimensional electronic map, and obtains the robot's current position coordinates in real time. Set the dual cameras to a fixed elevation angle of 30 degrees. To cover the key areas of the inspection equipment; Construct three-dimensional trigger conditions. In the lighting dimension, when the light intensity is less than or equal to the preset light threshold, start the exposure parameter optimization. In terms of device spacing, when the device spacing is less than or equal to a preset spacing threshold, the stop shooting step size is shortened. In terms of visibility, when the visibility is less than or equal to the preset visibility threshold, increase exposure compensation; If all three trigger conditions are met, the dynamic coordination rule is triggered, and shooting is performed using a shortened stop shooting step and optimized exposure parameters; If the triggering conditions of any single dimension or any two dimensions are met, the adjustment rules corresponding to the met dimensions will be executed; otherwise, no parameter adjustment will be performed.

5. The automatic configuration method for an outdoor inspection robot according to claim 4, characterized in that, The specific steps for generating the corrected annotation target set include: The robot continues to walk along the planned route according to the adjusted stop shooting step length. It performs real-time positioning calibration by combining GPS / IMU fusion positioning data with the route coordinates of the 3D electronic map. When it reaches the shooting point, it stops, and the dual cameras load optimized exposure parameters and maintain a fixed elevation angle to inspect the equipment on both sides of the shooting route. Calculate the disparity map variance. If the disparity map variance is greater than a preset variance threshold, the image is determined to be blurry; otherwise, the image is determined to be clear. Calculate the occlusion area. If the occlusion area is greater than or equal to a preset area threshold, it is determined to be occlusion; otherwise, it is determined to be unoccluded. Images that are clear and unobstructed are initially retained and integrated into a preliminary effective image set. This set is then input into a trained YOLOv8n model for initial AI screening, and the output includes the category, bounding box, and initial screening confidence of the inspected target. The initial screening confidence levels are divided into three levels: Level 1, Level 2, and Level 3. The initial screening results are integrated and generated into a table. The Level 2 confidence items are then manually reviewed to generate a set of corrected and annotated targets.

6. The automatic configuration method for an outdoor inspection robot according to claim 5, characterized in that, The specific steps for generating a complete set of uniformly labeled photos include: Obtain the corrected bounded target, quantify the influence weight of each environmental parameter, normalize the light intensity, equipment spacing and visibility, and calculate the environmental weighting factor in combination with the influence weight; The actual shooting angle of the target is obtained, and the angle deviation value is calculated by combining it with a fixed elevation angle. The actual height of the target, the image scaling factor and the device spacing are obtained to calculate the image offset factor. The focal length of the dual cameras is obtained and the angle deviation value is combined to calculate the angle deviation compensation amount. The target bounding box coordinates are adjusted to generate an angle-compensated target area image. Feature points are extracted to generate a template feature vector library. For the target area of ​​the full inspection photos, the response value of the feature points is corrected by combining the environmental weighting factor. The feature vector to be matched is extracted, and the matching distance between the target area to be matched and the template target is calculated. If the matching distance is less than or equal to the preset distance threshold, the matching score is calculated, and the effective matching feature point pairs are retained according to the preset score threshold. The target contour is fitted and the bounding box coordinates are combined with the corrected bounding box coordinates to uniformly select and label the target, generating a full set of uniformly labeled photos.

7. The automatic configuration method for an outdoor inspection robot according to claim 6, characterized in that, The specific steps for determining the consistency of target selection include: Construct an inspection and identification model, including an input layer, a feature fusion layer, a feature decision layer, and an output layer; The input layer is used to receive image features and environmental features; The feature fusion layer is used to weightedly fuse the image features and the environmental features through a multi-head self-attention mechanism to generate a fused feature vector; The feature decision layer is used to receive the fused feature vector, extract key features, fit the target bounding box coordinates using a regression branch, and determine the target category and recognition reliability through a classification branch. The output layer is used to output the bounding coordinates of the target, the target category, and the target confidence level; Using the full set of uniformly labeled photos as the training set, the inspection and recognition model is trained by combining the total loss function and the AdamW optimizer. After training, the overlap of the bounding boxes is calculated and the target confidence is combined to determine the consistency of the target bounding boxes. If the overlap of the selected boxes is greater than or equal to the preset overlap threshold, and the target confidence is greater than or equal to the preset confidence threshold, then the target selection consistency is determined to be up to standard; otherwise, the inspection and recognition model is updated.

8. The automatic configuration method for an outdoor inspection robot according to claim 7, characterized in that, The specific steps for constructing three-dimensional constraints include: For the full range of labeled inspection photos, based on the binocular vision ranging formula, camera parameters and environmental parameters are fused and an atmospheric scattering correction term is introduced. The target disparity is calculated by matching the left and right views of the binocular camera. The atmospheric scattering correction term is determined based on the current environmental parameters, and the straight-line distance between the target and the robot is calculated. Calculate the actual size of the target based on the target pixel coordinates and camera intrinsic parameters in the labeled full-scale inspection photos; calculate the horizontal and vertical angles based on the deviation angle between the target center pixel coordinates and the center coordinates of the labeled full-scale inspection photos. Construct three-dimensional constraints, including shooting angle within the horizontal and vertical ranges, image clarity greater than or equal to a preset clarity threshold, and target key feature point coverage greater than or equal to a preset coverage threshold.

9. The automatic configuration method for an outdoor inspection robot according to claim 8, characterized in that, The specific steps for generating the association mapping table include: Valid inspection photos that simultaneously meet the three-dimensional constraints are selected, and the improved ORB algorithm is used to extract and match feature points of the valid inspection photos to calculate image similarity. Automatically delete duplicate photos with image similarity greater than or equal to a preset similarity threshold, and combine the straight-line distance, the actual size of the target, the horizontal angle and the vertical angle to mark all valid inspection targets and generate a set of valid inspection targets; Based on the effective inspection target set and inspection route, an inspection and shooting task list is generated. The robot is controlled to call the initial gimbal attitude parameters in the inspection and shooting task list during the inspection process and shoot the effective inspection targets in sequence. If the effective inspection target is outside the preset photo center area, the attitude compensation amount is calculated using the bat algorithm based on real-time environmental parameters, and the gimbal direction is adjusted and the photo is retaken using the 3D positioning algorithm until the effective inspection target is within the preset photo center area.

10. The automatic configuration method for an outdoor inspection robot according to claim 9, characterized in that, The specific steps for generating the association mapping table also include: If the actual size of the target is smaller than the preset small target threshold, the target is determined to be a small target; otherwise, it is determined to be a large auxiliary target. For small targets, extract the contextual features of the large auxiliary target in the captured image, locate the candidate region of the small target using the sliding window method, perform double interpolation magnification, and extract the feature points of the small target and compare them with the preset target template. Matching and recognition; calculate the matching and recognition rate. If the matching recognition rate is greater than or equal to the preset matching recognition threshold, the recognition is considered successful. The gimbal angle and shooting position are adjusted by 3D positioning to place the small target in the center of the photo and take a clear photo of the small target. For each valid inspection target, a hash mapping is used to generate an association mapping table, which is then imported into the robot control system. Based on the parameters collected in real time, the system automatically matches the configuration parameters of the optimal inspection step size and the optimal gimbal angle from the association mapping table to complete the automatic configuration of robot inspection.

Images