Path planning method and system for inspection robot

Abstract

The invention relates to the technical field of path planning, in particular to a path planning method and system for an inspection robot, and provides the following scheme: executing an inspection task through an initial electronic map, collecting first image data and pose information, recognizing an inspection target, completing image registration, and generating second image data; inverting and calibrating the optimal observation direction based on the structural sensitivity distribution of the local visual features and the illumination variation trend, and determining the shooting position and angle to form a path node; a multi-objective optimization model is adopted, the path length, the view angle continuity and the illumination interference are comprehensively considered, and an optimal inspection path is generated; the method is suitable for outdoor inspection scenes with complex structures and non-uniform target distribution.

Description

Technical Field

[0001] The present application relates to the field of path planning technology, and in particular to a path planning method and system for a patrol robot. Background Art

[0002] Inspection robots are currently widely used in industrial applications such as power generation, petrochemicals, and transportation, replacing manual labor in equipment status monitoring and troubleshooting. However, most existing path planning methods rely on manually preset inspection target coordinates or static task differentiation, making them difficult to adapt to real-world environments with unclear target boundaries, complex spatial structures, and uneven target distribution. Furthermore, traditional image acquisition typically uses fixed angles and distances, failing to fully consider the sensitivity of target structures and the impact of ambient lighting conditions on image quality. This can easily lead to blurring or occlusion of key details, thus affecting the accuracy of subsequent recognition and discrimination.

[0003] For example, Chinese patent application CN111781936B discloses a robot path planning method, apparatus, robot, and computer-readable storage medium. Applicable to a robot, the method comprises: determining a positioning map of the current working area; receiving prohibited area data transmitted by a host computer; planning a path based on the positioning map and the prohibited area data; and moving according to the path. This technical solution eliminates the need for additional sensors and enables the robot to avoid prohibited areas. Furthermore, the addition of a prohibited area map layer does not affect the original positioning system, maintaining the original positioning accuracy.

[0004] The above patents have the problem raised by this background technology: under restricted moving path conditions such as rail-type mobile platforms, how to achieve effective modeling and information acquisition of the target with limited posture choices is still a technical difficulty in current inspection path planning. In order to solve the above problems, this application designs a path planning method and system for inspection robots. Summary of the Invention

[0005] The technical problem to be solved by this application is to address the shortcomings of existing technologies and provide a path planning method and system for inspection robots. This method uses an initial electronic map to execute inspection tasks, collects first image data and position information, identifies inspection targets, completes image registration, and generates second image data. Based on the structural sensitivity distribution of local visual features and the trend of illumination changes, the optimal observation direction is inverted and calibrated, and the shooting positions and angles that constitute the path nodes are determined. A multi-objective optimization model is used to comprehensively consider path length, perspective continuity, and illumination interference to generate the optimal inspection path.

[0006] To achieve the above objectives, this application provides the following technical solutions:

[0007] A path planning method for an inspection robot, the method comprising:

[0008] Driving the inspection robot to perform a first phase inspection according to a preset initial electronic map to obtain first image data and corresponding posture data;

[0009] Processing the first image data to identify inspection targets, and registering the first image data acquired from multiple shooting positions for each inspection target to obtain second image data;

[0010] Based on the second image data, determining a shooting position and a shooting angle for acquiring inspection target information as a shooting configuration of the corresponding inspection target;

[0011] Inputting the shooting configurations of all inspection targets as path nodes into a preset path optimization model, and outputting the inspection path of the inspection robot, wherein the path optimization model generates the inspection path through a multi-objective optimization algorithm, and the multi-objective optimization algorithm includes the kinematic constraints of the inspection robot;

[0012] Motion control instructions are generated according to the inspection path to control the inspection robot to perform inspection tasks according to the inspection path.

[0013] The shooting configurations of all inspection targets are input into the preset path optimization model as path nodes, and the inspection path of the inspection robot is output, including:

[0014] The shooting position and shooting angle of each inspection target are used to form path nodes, and a cost function is constructed between the nodes. The cost function includes the spatial movement distance between nodes, the angle of view switching amplitude, and the degree of change of shooting lighting conditions;

[0015] All path nodes are input into a multi-objective optimization algorithm, which comprehensively minimizes the total inspection path length, the perspective switching cost, and the interference effect of lighting changes, and outputs an inspection path that meets the shooting continuity constraint. The constraints of the multi-objective optimization algorithm include the kinematic constraints, dynamic constraints of the inspection robot, and the time constraints of the inspection task.

[0016] Generating motion control instructions according to the inspection path includes:

[0017] According to the path nodes of the inspection path and in combination with the kinematic model of the inspection robot, a control parameter set including a speed curve, a steering angle command, and a torque command is generated;

[0018] The control parameter set is converted into an electrical signal and sent to the inspection robot.

[0019] Processing the first image data to identify an inspection target includes:

[0020] Processing the first image data using a pre-trained target detection model to extract candidate inspection target areas, wherein the target detection model is a convolutional neural network structure trained on a power equipment image dataset;

[0021] According to the inspection target candidate area, the corresponding contour features and texture features are calculated through edge detection, and the contour features and texture features are matched with a preset template library for similarity to identify the inspection target.

[0022] Registering first image data of each inspection target acquired from multiple shooting positions to obtain second image data includes:

[0023] Determining an image sequence for the same inspection target based on a timestamp and position information when the first image data is collected, wherein each first image data in the image sequence corresponds to a different shooting position;

[0024] The first image data in the image sequence are registered by feature point matching to obtain second image data, wherein the feature point matching includes extracting corner points of the first image data, constructing local descriptors, and obtaining corresponding point pairs between images by bidirectional matching based on the local descriptors.

[0025] Determine the shooting position and angle for obtaining inspection target information, including:

[0026] For each inspection target, identifying local visual features of the corresponding inspection target in the second image data, wherein the local visual features include nameplate characters, connection terminals, and equipment seams;

[0027] Performing structural sensitivity analysis on the local visual features to obtain a structural sensitivity distribution;

[0028] Inverting a first observation direction of the local visual feature according to the structural sensitivity distribution, wherein the shooting angle corresponding to the first observation direction is a shooting direction that makes the comprehensive value of the structural sensitivity of the multiple local visual features reach a maximum value;

[0029] The first observation direction is calibrated according to lighting information in the second image data to determine a second observation direction, the second observation direction is used as a shooting angle, and the intersection of the second observation direction and the initial electronic map is used as a shooting position, wherein the lighting information is a brightness variation range of the second image data at different viewing angles.

[0030] Performing a structural sensitivity analysis on the local visual features to obtain a structural sensitivity distribution includes:

[0031] Calculating pixel gradients and spatial arrangement features of the local visual features;

[0032] Comparing the image structure changes of the pixel gradients and spatial arrangement features frame by frame to obtain a structural change amount, wherein the structural change amount includes a clarity change, a structural integrity change, and an image detail preservation degree;

[0033] According to the variation trend of the structural variation in the viewing angle dimension, a corresponding sensitivity curve is constructed, and according to the slope of the sensitivity curve, a structural sensitivity distribution is generated.

[0034] Inverting a first observation direction of the local visual feature according to the structural sensitivity distribution includes:

[0035] The sensitivity values ​​corresponding to each shooting direction in the structural sensitivity distribution are combined into a spatial sensitivity point set, and the structural sensitivity change rate between each shooting direction is calculated based on the spatial sensitivity point set to generate a spatial sensitivity gradient field;

[0036] In the spatial sensitivity gradient field, a structural response path is calculated based on the gradient rising direction, wherein the structural response path is a direction sequence with the largest response change in the gradient rising path;

[0037] A change rate analysis is performed on the structural response path to obtain an extreme point of the structural sensitivity change rate, and a direction corresponding to the extreme point is used as the first observation direction of the local visual feature.

[0038] Calibrating the first observation direction according to illumination information in the second image data to determine the second observation direction includes:

[0039] Extracting brightness gradient values ​​of corresponding illumination information according to a plurality of adjacent viewing angles in the first observation direction to obtain a local brightness change sequence;

[0040] Calculating the brightness fluctuation amplitude corresponding to the first observation direction, and if the brightness fluctuation amplitude is less than the amplitude mean of the local brightness change sequence, using the first observation direction as the second observation direction;

[0041] If the brightness fluctuation amplitude is greater than or equal to the amplitude mean of the local brightness change sequence, constructing a brightness fluctuation trend vector field based on the local brightness change sequence;

[0042] calculating a directional distribution trend of a minimum brightness disturbance amplitude in the brightness fluctuation trend vector field, and calculating a calibration vector based on the directional distribution trend, wherein the calibration vector represents a minimum illumination interference adjustment direction relative to the first observation direction;

[0043] Direction calibration is performed on the first observation direction according to the calibration vector to obtain a second observation direction.

[0044] A path planning system for an inspection robot, the system comprising:

[0045] An image acquisition module is used to control the inspection robot to perform a first-stage inspection according to the initial electronic map and obtain first image data and corresponding posture data;

[0046] an image processing module, configured to process the first image data, identify an inspection target, and register the first image data acquired from multiple shooting positions of the same inspection target to obtain second image data;

[0047] a shooting configuration generation module, which performs structural sensitivity analysis and observation direction inversion based on local visual features in the second image data, calibrates the first observation direction in combination with illumination information, and determines the shooting position and shooting angle of each inspection target as the shooting configuration;

[0048] The path planning module is used to construct a cost function using the shooting configuration of each inspection target as a path node and generate an inspection path through a multi-objective optimization algorithm;

[0049] The control execution module is used to send the inspection path to the inspection robot and control it to complete the target information collection in sequence according to the inspection path.

[0050] Compared with the prior art, the present invention has the following advantages:

[0051] This application can dynamically identify inspection targets and their optimal observation configurations without prior information about the target, automatically generating inspection paths with consistent capture and perceptual stability. By constructing a structural sensitivity distribution and inverting the first observation direction, it can filter out observation points with the clearest structural information from multiple perspectives. Furthermore, the shooting direction is calibrated based on illumination disturbance trends, improving the validity and consistency of the image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0053] Figure 1 This is a schematic diagram of an exemplary inspection scenario in an embodiment of the present application;

[0054] Figure 2 This is a flow chart of a path planning method for an inspection robot according to an embodiment of the present application;

[0055] Figure 3 This is a schematic diagram of the first image data processing flow in an embodiment of the present application;

[0056] Figure 4 This is a schematic diagram of the principle of the second image processing in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0058] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of a phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It will be understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0059] This application is applicable to an outdoor autonomous inspection system based on free navigation control and equipped with dual-stage image acquisition and path optimization capabilities. The inspection system is designed for the following scenarios. The characteristics of the application environment include but are not limited to:

[0060] Wheeled or tracked inspection robots with autonomous navigation capabilities can complete initial path exploration and secondary planning in unstructured or semi-structured outdoor environments;

[0061] Target inspection objects are irregularly arranged along outdoor inspection routes, and most lack clear positioning markers. Targets vary in spatial position, orientation, and occlusion status.

[0062] The target is power equipment components with typical outdoor installation characteristics, including pole-mounted transformers, switchgear, outdoor distribution boxes, and photovoltaic modules. Their common characteristics include, but are not limited to: surface structural details are the key judgment basis, and image acquisition is limited by angle, lighting, and occlusion conditions;

[0063] It should be noted that this application does not rely on preset coordinates or statically collected position information for all inspection targets. Instead, it uses the images and pose data collected in the first stage to dynamically identify the targets and their structural contour information, and then infer the most suitable shooting direction and position.

[0064] It can be understood that the inspection path planning method proposed in this application is applicable to both the first inspection of a certain outdoor scene and the periodic daily inspection of the scene. In an optional embodiment, the inspection robot can move along a fixed track.

[0065] See also Figure 1 , Figure 1 A schematic diagram of an exemplary inspection scenario provided in an embodiment of the present application.

[0066] Figure 1 The robot moves along a pre-set inspection track. During the initial inspection phase, it captures images at a fixed stepping distance, acquiring first image data and corresponding pose information. At this stage, the spatial information of the inspection target is unclear, so the system relies on the first image data for target recognition and contour extraction.

[0067] Figure 1 It further shows that based on the results obtained after the first stage of alignment, the system identifies the inspection target (such as the equipment cabinet panel in the figure), and combines image structure sensitivity analysis and lighting condition deduction to invert the optimal observation direction and determine the appropriate shooting position and angle for the second inspection.

[0068] It should be noted that the inspection targets of this application are often equipment units installed in open spaces, which lack uniform dimensions, standardized anchor points or clear boundary markings, and have structural characteristics of both extensibility and obstruction in space.

[0069] The drawings provided in this application are only illustrative examples. The specific target type, quantity and track configuration can be flexibly adjusted according to the actual deployment environment and do not constitute a limitation on the type of equipment, identification method or installation scheme.

[0070] In this embodiment, the inspection robot performs the first phase of inspection image acquisition along the track based on a preset initial electronic map. The electronic map provides basic path information such as track structure, relative spatial position, and acquisition step distance, but does not include the specific spatial coordinates or boundary configuration of the inspection target.

[0071] In actual inspection environments, inspection targets are usually installed on the side of the track, such as switch cabinets, electrical control boxes, or equipment panels. The characteristics of their spatial layout include but are not limited to:

[0072] There is no unified size standard between the targets, the spatial spacing is uneven, and there is a lack of unified markings;

[0073] The inspection robot's shooting direction deflects during the acquisition process, and the target's structure in different frames of image is significantly different.

[0074] The boundary of the target in the image may be connected to the background or be obstructed, making it difficult to statically define the target image;

[0075] As you can understand, this embodiment achieves dynamic identification of inspection targets by analyzing structural features and aligning spatial positions of the images collected in the first phase, without requiring pre-set target coordinates or labels. The system clusters and groups local visual structures extracted from the images (including nameplate characters, terminal blocks, structural seams, etc.), assigning images with structural consistency across consecutive frames to the same set of inspection target images.

[0076] Furthermore, based on the image set, the optimal shooting configuration for obtaining target information is derived through structural sensitivity analysis and observation direction inversion. Combined with illumination gradient analysis, the shooting angle is calibrated and input as a node into the path optimization module to generate the second-stage inspection path.

[0077] Next, in conjunction with the accompanying drawings, a path planning method for an inspection robot provided in an embodiment of the present application is introduced. Figure 2 The method shown includes the following steps S1-S4, and the specific steps are as follows:

[0078] S1: Performing a first-stage inspection according to a preset initial electronic map to obtain first image data and corresponding posture data;

[0079] In this embodiment, the inspection robot performs image acquisition tasks within a track path or designated movement area according to a pre-configured electronic map, following a set stepping rule. During the acquisition process, the robot simultaneously records the spatial pose information of each image frame. The first image data is a sequence of images acquired at stepping intervals along the inspection track. The pose data includes information such as displacement, posture, and shooting direction at the time of capture, which is used for subsequent image alignment and target attribution determination.

[0080] Understandably, the purpose of the first phase of inspection is to construct a set of spatially referenced image data, given the unknown location of the inspection target, to provide a foundation for target identification and information inversion. Electronic maps only provide a framework for navigation paths; this application does not rely on pre-set target coordinates for subsequent modeling.

[0081] S2: Processing the first image data to identify inspection targets, and registering the first image data acquired from multiple shooting positions for each inspection target to obtain second image data;

[0082] In this embodiment, an object detection network trained on a dataset of power equipment images is used to extract candidate inspection target regions from the first image data. A combination of edge features and texture matching is then used to construct a target identification criterion, eliminating background interference and non-structural feature areas. For each identified inspection target, image segments are captured from multiple corresponding shooting positions and registered using the pose relationships between the images. This creates a unified set of images of the target from different perspectives, which serve as the second image data.

[0083] S3: Determine, based on the second image data, a shooting position and a shooting angle for acquiring inspection target information as a corresponding shooting configuration of the inspection target;

[0084] In this embodiment, typical local visual structural features, such as nameplate characters, terminal blocks, or equipment corners and gaps, are first extracted from the second image data. Their pixel gradients and structural clarity are calculated in images from different viewpoints. Combined with the pose information, the spatial directional response relationships of each structure are derived to establish a structural sensitivity distribution model. Subsequently, within the spatial sensitivity distribution field, the direction of the structural response peak is identified based on the gradient variation path, and the level of illumination interference is determined based on the brightness variation trend, enabling automatic calibration of the shooting perspective. The resulting shooting angle and corresponding spatial position constitute the target's shooting configuration.

[0085] S4: inputting the shooting configurations of all inspection targets as path nodes into a preset path optimization model, and outputting the inspection path of the inspection robot;

[0086] In this embodiment, the shooting position and angle corresponding to each inspection target are used as path nodes, and a cost function is constructed based on the spatial movement distance between nodes, the angle of view switching amplitude, and the degree of illumination change. A multi-objective optimization method is used to comprehensively consider the total path length, angle of view continuity, and the stability of the shooting conditions to solve the optimal inspection sequence and movement path, generating the task trajectory for the second stage. The path planning model can use optimization algorithms such as genetic algorithms, ant colony optimization, or local search. Operational data can be collected according to actual scenario settings, which are not limited in this application.

[0087] It can be understood that the inspection path in this application can be the initial planned path autonomously generated by the robot before the first mission execution, or it can be a daily maintenance path dynamically updated based on existing mission data. The specific path planning strategy can be flexibly configured according to factors such as deployment frequency, target change frequency and scene stability.

[0088] Those skilled in the art will understand that the final formation of the inspection path not only depends on the spatial distribution of the shooting configuration of each inspection target, but is also affected by non-structural factors such as environmental dynamic conditions. Therefore, the path optimization strategy should have a certain degree of dynamic adaptability and configurability to adapt to the diverse actual inspection scenario requirements.

[0089] In an optional embodiment, the method further includes:

[0090] S5: Generate motion control instructions according to the inspection path to control the inspection robot to perform the inspection task according to the inspection path.

[0091] In actual outdoor power inspection applications, target equipment is usually distributed in irregular terrain or non-standard installation environments, and inspection paths and shooting strategies often cannot fully rely on pre-built high-precision maps or static target coordinate systems. Traditional path planning methods are mostly based on preset target coordinates or structured area divisions for inspection point layout. This method requires detailed modeling of the scene before the system is put into operation, which obviously has limitations in scenarios of initial deployment or frequent changes in equipment location. In addition, the shooting configuration for a certain inspection target in conventional image acquisition methods usually adopts fixed parameters or empirical presets, which fails to fully incorporate external disturbance factors such as spatial perspective differences, target surface structural characteristics, and lighting changes formed by actual image acquisition. As a result, the shooting results cannot be reused stably for a long time, and frequent adjustments and reshoots are required, making it difficult to uniformly guarantee inspection efficiency and image quality.

[0092] Taking the inspection of outdoor busbars in substations as an example, this type of target generally has the characteristics of surface reflection, structural cable obstruction, and dense spatial layout. If the shooting point selection logic based on the geometric model is directly used, it is often difficult to take into account both image quality and traffic feasibility. However, this application starts from multi-perspective image registration, reversely judges the image structure integrity and brightness disturbance trend under different perspectives, and automatically adjusts the shooting parameters and posture. It does not require external manual annotation or fixed coordinate system support, and can achieve autonomous modeling of the target structure level and generation of the optimal shooting configuration.

[0093] Next, the part of the method of the present application regarding the first image data processing is further expanded.

[0094] See also Figure 3 , Figure 3 This is a schematic diagram of the first image data processing flow in an embodiment of the present application. Figure 3 It shows that the processing of the first image data of this application includes two parts: inspection target recognition and data registration.

[0095] It can be understood that the first image data of this application refers to all images obtained by the inspection robot during the first inspection phase, and are sorted according to the timestamp of the shooting. When the same inspection target is identified, the data alignment part defaults to aligning the first image data of the same inspection target.

[0096] In an example, the specific steps for inspection target identification are as follows:

[0097] S2.1: Processing the first image data using a pre-trained target detection model to extract candidate inspection target areas, wherein the target detection model is a convolutional neural network structure trained on a power equipment image dataset;

[0098] In this embodiment, the target detection model employed is a pre-trained convolutional neural network (CNN) trained on a dataset of images featuring power equipment. This dataset covers a variety of common power equipment configurations, including but not limited to switchgear, cable boxes, and terminal blocks. The dataset features diverse content from multiple angles, varying lighting conditions, and occlusions, enabling the model to adapt to complex real-world inspection scenarios. The network model utilizes deep learning architectures with target localization capabilities, such as YOLOv5, Faster R-CNN, or RetinaNet. Through end-to-end regression detection, the model outputs information about all rectangular boxes in the image that may contain electrical targets as candidate regions.

[0099] S2.2: Based on the candidate inspection target area, calculate the corresponding contour features and texture features through edge detection, and perform similarity matching between the contour features and texture features and a preset template library to identify the inspection target;

[0100] Specifically, edge features are extracted from the interior of the candidate region. In an optional embodiment, feature extraction is performed using a Canny operator or a Sobel operator to construct the contour boundary shape of the target.

[0101] Furthermore, the texture features of the candidate region are extracted through the gray-level co-occurrence matrix to describe the surface structural characteristics of the target region in the image space.

[0102] Furthermore, the contour features and texture features are compared with a locally maintained template library, which pre-collects and organizes multi-view image features of standard target devices and establishes corresponding structural feature files for each type of device.

[0103] Furthermore, during the comparison process, a multi-dimensional similarity matching logic can be used to comprehensively evaluate the similarity between the candidate region and the template based on multiple indicators such as shape matching coefficient, texture direction consistency score, and edge closure. If the similarity exceeds a preset similarity threshold, the candidate region is confirmed as a valid inspection target area and marked with a specific target number. The similarity threshold can be determined by those skilled in the art through experimentation.

[0104] It is understandable that the number of inspection targets is determined by the specific application scenario. The identification of inspection targets in this application is based on existing related technologies and does not limit the number, type or appearance structure of inspection targets.

[0105] In an example, the specific steps of data registration are as follows:

[0106] S2.3: Determine an image sequence for the same inspection target based on the timestamp and pose information of the first image data when it is captured, wherein each first image data in the image sequence corresponds to a different shooting position;

[0107] S2.4: Registering each first image data in the image sequence by feature point matching to obtain second image data, wherein the feature point matching includes extracting corner points from the first image data, constructing local descriptors, and obtaining corresponding point pairs between the images by bidirectional matching based on the local descriptors;

[0108] In this embodiment, the first step of the registration is to use the timestamp and camera pose data attached at the time of acquisition to preliminarily aggregate image frames with similar shooting time, high consistency in shooting direction, and pose differences within an acceptable threshold range to form a candidate image sequence. The aggregation process can eliminate image frames in which the target deviates from the center of the field of view or completely moves out of the field of view due to robot movement, while ensuring target consistency, thereby improving the quality of subsequent image matching. Next, for each image in the image sequence, the Harris corner point or FAST feature point extraction operator is used to extract stable corner point features, and a scale-invariant local descriptor is constructed in the neighborhood of the feature point, such as lightweight descriptors such as ORB, BRIEF or AKAZE to adapt to edge computing scenarios. A two-way matching strategy is used to match image frames in the image sequence.

[0109] Furthermore, after completing the feature point matching, the RANSAC algorithm is used to estimate the homography matrix between images, and all image frames are uniformly transformed and reprojected into the reference frame coordinate system, thereby constructing second image data with consistent perspective and high boundary overlap.

[0110] Preferably, the estimation results can be optimized in combination with the posture information to make the transformation parameters more suitable for the actual shooting state and avoid the matching drift problem caused by weak texture or repeated structure of the image content.

[0111] Next, the part of the present application method regarding the shooting configuration is further expanded.

[0112] It can be understood that the second image data obtained in this application refers to the corresponding image of each inspection target after inspection target identification and alignment. The second image data can be a single image after image stitching or multiple images. This application does not limit this.

[0113] Furthermore, the present application obtains the shooting configurations of all inspection targets by traversing the second image data, and this embodiment is carried out based on the shooting configuration of a single inspection target.

[0114] Those skilled in the art will understand that when inspection robots are used in outdoor scenarios, the spatial layout of inspection targets is often complex and irregular. This is especially true in construction sites lacking unified installation specifications and explicit signage. Not only do the distribution of inspection targets present issues of unequal spacing and misalignment, but they can also suffer from varying degrees of visual information loss or noise interference due to external occlusion, background interference, or natural aging. In this context, relying solely on initial electronic maps or static pose information to pre-set shooting positions and angles can easily lead to problems such as out-of-focus capture, partial target loss, and insufficient information, impacting the accuracy and usability of inspection data.

[0115] This application is based on the image sequence that has completed the registration in the second image data. Without relying on external three-dimensional modeling equipment, it independently determines the most informative observation direction and shooting position by extracting the local visual features of the inspection target, analyzing the structural sensitivity, and reversely inferring the lighting information. The local visual features include but are not limited to typical structural units such as nameplate characters, terminal interfaces, and boundary gaps. These units usually have strong task relevance and image expression sensitivity in actual use, and are an important basis for determining equipment status and structural changes. By introducing a combination of structural sensitivity distribution inversion and illumination interference calibration, not only the effectiveness of the shooting angle is improved, but also the potential impact of actual on-site lighting conditions on image quality is taken into account, thereby achieving automatic reasoning and precise positioning of the optimal shooting configuration without the need for complex three-dimensional modeling or laser scanning.

[0116] Taking the second image data as an example, you can refer to Figure 4 To understand, Figure 4 This is a schematic diagram of the principle of the second image processing in an embodiment of the present application.

[0117] Figure 4 The structure sensitivity distribution relationship is constructed by extracting the corresponding structural characteristics of the local visual feature area in the second image data. The structural sensitivity distribution takes the viewing angle as the horizontal axis and the structural sensitivity in the structural response index as the vertical axis, which can be expressed as a one-dimensional curve (such as Figure 4 ) or a two-dimensional / three-dimensional distribution diagram. The two-dimensional form is used to represent the response changes of local features in the two angular dimensions of pitch and azimuth, while the three-dimensional structural sensitivity distribution further introduces a distance indicator to form a complete observation space sensitivity map, which is not limited in this application.

[0118] Figure 4 The inversion process of the structural sensitivity distribution is further shown. It can be understood that in the inversion process, the structural response path represents the optimal imaging response area of ​​the current local visual feature under a specific shooting angle. In a typical case, it should contain at least two extreme points, such as Figure 4The extreme point 1 and the extreme point 2 in the image correspond to clear structural peaks in different spatial directions, respectively. The embodiment analyzes the structural response path to identify the sequence with the largest structural change rate in the gradient ascent path, and combines the connectivity between the local feature position and the observation direction to select the direction connected by the extreme points as the first observation direction.

[0119] Figure 4 The image further illustrates the distribution of brightness fluctuations in the adjacent areas of the first viewing direction. By constructing a brightness fluctuation trend vector field, an adjustment vector with minimal illumination disturbance and high directional consistency is identified and combined with the first viewing direction to calibrate the final second viewing direction. While maintaining the advantages of structural response, image quality degradation caused by outdoor natural light interference or high-contrast shadows is minimized, thereby improving the stability and practicality of the shooting angle.

[0120] In an example, the specific steps for S3 are as follows:

[0121] S3.1: For each inspection target, identifying local visual features of the inspection target in the second image data, wherein the local visual features include nameplate characters, connection terminals, and equipment seams;

[0122] Specifically, due to the complex surface structure of the equipment, it is difficult to locate the key inspection areas based solely on the overall outline. It is necessary to accurately extract tiny visual features such as nameplate characters, connection terminals, and equipment seams to ensure subsequent positioning and image clarity judgment.

[0123] Those skilled in the art understand that the recognition method of local visual features can be achieved through existing technologies, for example, through the combination of feature point detection based on convolutional neural networks and traditional edge detection, first using a CNN trained on power equipment data to extract candidate areas, and then performing Canny edge and texture filtering operations on each area to generate a fine local visual feature mask, which will not be elaborated in this application.

[0124] S3.2: Performing a structural sensitivity analysis on the local visual features to obtain a structural sensitivity distribution;

[0125] Specifically, the image performance (clarity, boundary integrity, detail contrast) of local visual features at different shooting angles varies significantly. Using only typical angles will lead to missed detections or false alarms, and the response changes need to be quantified from multiple perspectives.

[0126] In this embodiment, pixel-level gradient and texture alignment analysis is performed on the located feature regions within a preset viewing angle window. Image detail preservation, edge connectivity, and confidence metrics are compared frame by frame to generate a structural sensitivity curve. Each structural sensitivity curve corresponds to a feature and is further normalized within the viewing angle dimension using slope change as a weight to generate a structural sensitivity distribution.

[0127] Furthermore, the structural sensitivity distribution can be expressed in two-dimensional or three-dimensional space, comprehensively characterizing the observability of equipment details from a spatial perspective and providing data support for the inversion of stable and effective observation directions.

[0128] In one example, performing a structural sensitivity analysis on the local visual features to obtain a structural sensitivity distribution includes:

[0129] Calculating pixel gradients and spatial arrangement features of the local visual features;

[0130] Comparing the image structure changes of the pixel gradients and spatial arrangement features frame by frame to obtain a structural change amount, wherein the structural change amount includes a clarity change, a structural integrity change, and an image detail preservation degree;

[0131] According to the variation trend of the structural variation in the viewing angle dimension, a corresponding sensitivity curve is constructed, and according to the slope of the sensitivity curve, a structural sensitivity distribution is generated.

[0132] S3.3: Inverting a first observation direction of the local visual feature based on the structural sensitivity distribution, wherein the shooting angle corresponding to the first observation direction is a shooting direction that maximizes the combined structural sensitivity value of the multiple local visual features;

[0133] Those skilled in the art will understand that if the selection is made simply based on the maximum value of the structural sensitivity distribution, it is easy to be affected by local noise. The purpose of this application is to find the angle with the best comprehensive performance in structural clarity and integrity through path continuity and gradient ascent analysis to ensure that the inversion direction is real and feasible.

[0134] It can be understood that path continuity ensures that the direction selection does not deviate from the optimal shooting range due to occasional feature enhancement, and the gradient ascent mechanism explores the stable growth area from the overall change trend, so that the inversion direction has both global optimality in visual clarity and continuous feasibility of shooting posture.

[0135] In one example, inverting the first viewing direction of the local visual feature according to the structural sensitivity distribution includes:

[0136] S3.3.1: Combine the sensitivity values ​​corresponding to each shooting direction in the structural sensitivity distribution into a spatial sensitivity point set, and calculate the structural sensitivity change rate between each shooting direction based on the spatial sensitivity point set to generate a spatial sensitivity gradient field;

[0137] Specifically, a single extreme value may originate from noise or local microstructural differences. Directly selecting the highest point can easily lead to inversion direction deviation or breakage, which is detrimental to the stability of the next observation. By mapping discrete sensitivity values ​​into a spatial gradient field, irrelevant fluctuations can be smoothed, the overall trend is preserved, and a continuous search foundation is provided for subsequent path tracing.

[0138] In this embodiment, for each viewing angle direction containing the target in the second image data, uniform sampling is performed at a preset angle step, and the structural sensitivity value is calculated respectively. Each sampled viewing angle and its sensitivity value are mapped to a three-dimensional directional coordinate system to form a spatial sensitivity point set. Then, the sensitivity difference between adjacent directions is divided by the angle difference to form a discrete change rate. Then, a three-dimensional interpolation algorithm is used to obtain a spatial sensitivity gradient field. Each point in the spatial sensitivity gradient field has both absolute sensitivity and gradient change trend, which serves as input for the structural response path search.

[0139] It is understandable that the spatial sensitivity gradient field can shield isolated noise and take into account the overall trend of the structure. It not only retains the sensitivity change trend between angles, but also forms a smooth and traceable field, effectively supports multi-directional parallel search, and reduces the inversion deviation caused by local data anomalies.

[0140] S3.3.2: In the spatial sensitivity gradient field, a structural response path is calculated based on the gradient ascent direction, wherein the structural response path is a sequence of directions in the gradient ascent path with the largest response change;

[0141] Specifically, multiple local growth channels may exist along different paths, and local growth rates alone cannot fully capture all structural characteristics. By constructing response paths and selecting the optimal sequence, we ensure that the selected direction is the endpoint of the path with the best overall performance in terms of structural edge clarity and integrity.

[0142] In this embodiment, multiple initial perspectives are randomly selected as starting points, and the process iteratively proceeds along the direction with the maximum gradient, and the next direction with the maximum sensitivity gain is selected within the neighborhood of each step.

[0143] It can be understood that the present application accumulates the total sensitivity gain value along each path and tracks the path coherence; and finally selects the best one in terms of overall sensitivity gain, path coherence and gradient curve smoothness as the structural response path.

[0144] S3.3.3: Perform a rate of change analysis on the structural response path to obtain an extreme point of the rate of change of the structural sensitivity, and use the direction corresponding to the extreme point as the first observation direction of the local visual feature;

[0145] Specifically, while the structural response path shows an overall upward trend, in reality, the rate of improvement in structural clarity and integrity often spikes at certain local angles rather than continuing linearly. Therefore, selecting the path endpoint based solely on the total sensitivity will cause the shooting angle to fall into a slow but smooth growth segment, losing the high-response perspective with clear structural boundaries and significant texture contrast. This application uses rate analysis to accurately discover the locations of sensitivity mutations in the path to reflect the true nonlinear relationship between observation angle and image structural response.

[0146] In this embodiment, the obtained structural response path is divided into several angle-stepping segments, and the ratio of the structural sensitivity increment to the angle increment between adjacent segments is calculated sequentially to form a structural sensitivity change rate curve. Subsequently, a first-order derivative analysis is performed on the curve to extract all local extreme points, and filter out valid extreme points whose rate increase exceeds a preset threshold. It should be noted that, for the sake of structural continuity and feature diversity, a single extreme point is not used as the basis for the first observation direction. Instead, the set of direction vectors corresponding to all valid extreme points is vector-weighted averaged. The weighting process can be based on the sensitivity gain amplitude, structural detail enhancement ratio, and gradient field continuity stability corresponding to each extreme point, thereby generating the final first observation direction.

[0147] Furthermore, if the angles between the direction vectors corresponding to multiple local extreme points deviate significantly, exceeding the set tolerance range, isolated extreme points with inconsistent directions are preferentially removed to avoid interfering with the stability of the directional synthesis. The resulting first observation direction not only incorporates the local advantages of multiple structural response high points, but also preserves the continuity of structural information and the feasibility of the shooting angle, ensuring that the selected direction has optimal recognition conditions and high path robustness during execution.

[0148] S3.4: Calibrate the first observation direction based on illumination information in the second image data to determine a second observation direction, use the second observation direction as the shooting angle, and use the intersection of the second observation direction and the initial electronic map as the shooting position, where the illumination information is the brightness variation range of the second image data at different viewing angles.

[0149] Specifically, the first observation direction is usually the optimal angle for structural response. Under variable outdoor lighting conditions, details may be blurred due to strong light or shadows. The angle needs to be adjusted based on the statistics of brightness changes to improve image quality and stability.

[0150] In this embodiment, with the first observation direction as the center, several neighboring view frames are sampled, and the corresponding brightness gradient sequence is extracted. The fluctuation amplitude is counted and compared with the sequence mean. When the fluctuation is lower than the mean, the original vector is retained; otherwise, a brightness disturbance trend vector field is constructed in these neighboring view angles, and the distribution trend with the smallest fluctuation and the highest consistency with the structural sensitivity direction is identified. The calibration vector is calculated based on the trend.

[0151] Furthermore, the calibration vector and the first observation direction are linearly combined according to a preset offset coefficient, and the result is normalized to obtain the second observation direction; the second observation direction takes into account the strongest structural sensitivity and the smallest illumination disturbance, which not only improves the clarity of details but also avoids visual occlusion caused by light and shadow, thereby improving the stability of subsequent intelligent inspection and maintenance work.

[0152] In one example, calibrating the first observation direction according to illumination information in the second image data to determine the second observation direction includes:

[0153] S3.4.1: Extract brightness gradient values ​​of corresponding illumination information from multiple adjacent viewing angles in the first observation direction to obtain a local brightness change sequence;

[0154] S3.4.2: Calculate the brightness fluctuation amplitude corresponding to the first observation direction. If the brightness fluctuation amplitude is less than the mean amplitude of the local brightness change sequence, use the first observation direction as the second observation direction.

[0155] S3.4.3: If the brightness fluctuation amplitude is greater than or equal to the amplitude mean of the local brightness change sequence, construct a brightness fluctuation trend vector field based on the brightness change sequence;

[0156] S3.4.3: Calculate a directional distribution trend of the minimum brightness disturbance amplitude in the brightness fluctuation trend vector field, and calculate a calibration vector based on the directional distribution trend, wherein the calibration vector represents a minimum illumination disturbance adjustment direction relative to the first observation direction;

[0157] S3.4.4: Perform direction calibration on the first observation direction according to the calibration vector to obtain a second observation direction;

[0158] Specifically, in actual inspection environments, external natural lighting conditions are difficult to maintain stable, and differences in target surface material, angle, and reflective properties may also cause sudden changes in brightness gradients, directly affecting the recognizability of structural features in the image. If there is high reflection interference or shadow superposition between the shooting angle and the lighting direction, it is easy to cause blurred boundaries and texture distortion of local visual features, thereby interfering with subsequent information extraction and recognition accuracy. Therefore, although the first observation direction has the best performance in terms of structural information sensitivity, it still needs to be combined with the illumination distribution for disturbance analysis and direction calibration to avoid perceptual misjudgment caused by illumination deviation and ensure the stability and practicality of the inspection image.

[0159] In this embodiment, a number of adjacent viewing angles, offset by equal angles, are set around a selected first viewing direction. Luminance gradient information for the corresponding image regions is extracted from the second image data, establishing a one-to-one correspondence between viewing angle and brightness variation. Each adjacent viewing angle outputs a sequence of brightness gradient values, and the brightness variation amplitudes statistically obtained from these values ​​are recorded as a local brightness variation sequence. Subsequently, the brightness fluctuation amplitude of the first viewing direction within this sequence is calculated as a quantitative indicator of illumination stability in that direction.

[0160] Furthermore, if the brightness fluctuation amplitude in the first observation direction is less than the mean amplitude of the local brightness variation sequence, this indicates that this direction has a relatively stable brightness response under the current lighting conditions and can be directly used as the final shooting angle without further directional adjustment. If the test result is the opposite, it means that although this direction has excellent structural sensitivity, it has fluctuations in illumination response, which may pose a risk to image clarity. In this case, the illumination interference calibration mechanism needs to be activated.

[0161] To this end, this embodiment constructs a brightness fluctuation trend vector field based on the entire local brightness change sequence. This vector field, with viewing angle change as the coordinate axis, maps brightness gradient fluctuations in all directions into continuous vector states, forming a spatial distribution model of illumination stability with the perturbation amplitude as the amplitude. Based on this, directional regression analysis of the vector field is performed to search for continuous trend regions with minimal directional perturbations. Vector averaging is performed on these main directions, and a calibration vector is output as a reference direction for illumination adjustment.

[0162] The calibration vector represents the direction and degree of adjustment relative to the first viewing direction. In practice, the calibration vector is linearly fused with the original first viewing direction vector. Fusion weights are adaptively assigned based on the perturbation strength to form a fine-tuning vector. The final output direction vector is normalized and used as the second viewing direction.

[0163] In an example, the specific steps of S4 are as follows:

[0164] S4.1: Construct path nodes based on the shooting position and shooting angle of each inspection target, and construct a cost function between the nodes. The cost function includes the spatial movement distance between nodes, the angle of view switching amplitude, and the degree of change in shooting lighting conditions;

[0165] S4.2: Input all path nodes into the multi-objective optimization algorithm, comprehensively minimize the total inspection path length, perspective switching cost and the interference effect of lighting changes, and output an inspection path that meets the shooting continuity constraint, wherein the constraints of the multi-objective optimization algorithm include the kinematic constraints, dynamic constraints of the inspection robot and the time constraints of the inspection task.

[0166] In one example, generating a motion control instruction according to the inspection path includes:

[0167] According to the path nodes of the inspection path and in combination with the kinematic model of the inspection robot, a control parameter set including a speed curve, a steering angle command, and a torque command is generated;

[0168] The control parameter set is converted into an electrical signal and sent to the inspection robot.

[0169] In one example, the present application provides a path planning system for an inspection robot, the system comprising:

[0170] An image acquisition module is used to control the inspection robot to perform a first-stage inspection according to the initial electronic map and obtain first image data and corresponding posture data;

[0171] an image processing module, configured to process the first image data, identify an inspection target, and register the first image data acquired from multiple shooting positions of the same inspection target to obtain second image data;

[0172] a shooting configuration generation module, which performs structural sensitivity analysis and observation direction inversion based on local visual features in the second image data, calibrates the first observation direction in combination with illumination information, and determines the shooting position and shooting angle of each inspection target as the shooting configuration;

[0173] The path planning module is used to construct a cost function using the shooting configuration of each inspection target as a path node and generate an inspection path through a multi-objective optimization algorithm;

[0174] The control execution module is used to send the inspection path to the inspection robot and control it to complete the target information collection in sequence according to the inspection path.

[0175] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A path planning method for an inspection robot, characterized in that: The method comprises: Driving the inspection robot to perform a first phase inspection according to a preset initial electronic map to obtain first image data and corresponding posture data; Processing the first image data to identify inspection targets, and registering the first image data acquired from multiple shooting positions for each inspection target to obtain second image data; Based on the second image data, determining a shooting position and a shooting angle for acquiring inspection target information as a shooting configuration of the corresponding inspection target; Inputting the shooting configurations of all inspection targets as path nodes into a preset path optimization model, and outputting the inspection path of the inspection robot, wherein the path optimization model generates the inspection path through a multi-objective optimization algorithm, and the multi-objective optimization algorithm includes the kinematic constraints of the inspection robot; Motion control instructions are generated according to the inspection path to control the inspection robot to perform inspection tasks according to the inspection path.

2. A path planning method for an inspection robot according to claim 1, characterized in that: The shooting configurations of all inspection targets are input into the preset path optimization model as path nodes, and the inspection path of the inspection robot is output, including: The shooting position and shooting angle of each inspection target are used to form path nodes, and a cost function is constructed between the nodes. The cost function includes the spatial movement distance between nodes, the angle of view switching amplitude, and the degree of change of shooting lighting conditions; All path nodes are input into a multi-objective optimization algorithm, which comprehensively minimizes the total inspection path length, the perspective switching cost, and the interference effect of lighting changes, and outputs an inspection path that meets the shooting continuity constraint. The constraints of the multi-objective optimization algorithm include the kinematic constraints, dynamic constraints of the inspection robot, and the time constraints of the inspection task.

3. A path planning method for an inspection robot according to claim 1, characterized in that: Generating motion control instructions according to the inspection path includes: According to the path nodes of the inspection path and in combination with the kinematic model of the inspection robot, a control parameter set including a speed curve, a steering angle command, and a torque command is generated; The control parameter set is converted into an electrical signal and sent to the inspection robot.

4. A path planning method for an inspection robot according to claim 1, characterized in that: Processing the first image data to identify an inspection target includes: Processing the first image data using a pre-trained target detection model to extract candidate inspection target areas, wherein the target detection model is a convolutional neural network structure trained on a power equipment image dataset; According to the inspection target candidate area, the corresponding contour features and texture features are calculated through edge detection, and the contour features and texture features are matched with a preset template library for similarity to identify the inspection target.

5. A path planning method for an inspection robot according to claim 4, characterized in that: Registering first image data of each inspection target acquired from multiple shooting positions to obtain second image data includes: Determining an image sequence for the same inspection target based on a timestamp and position information when the first image data is collected, wherein each first image data in the image sequence corresponds to a different shooting position; The first image data in the image sequence are registered by feature point matching to obtain second image data, wherein the feature point matching includes extracting corner points of the first image data, constructing local descriptors, and obtaining corresponding point pairs between images by bidirectional matching based on the local descriptors.

6. A path planning method for an inspection robot according to claim 1, characterized in that: Determine the shooting position and angle for obtaining inspection target information, including: For each inspection target, identifying local visual features of the corresponding inspection target in the second image data, wherein the local visual features include nameplate characters, connection terminals, and equipment seams; Performing structural sensitivity analysis on the local visual features to obtain a structural sensitivity distribution; Inverting a first observation direction of the local visual feature according to the structural sensitivity distribution, wherein the shooting angle corresponding to the first observation direction is a shooting direction that makes the comprehensive value of the structural sensitivity of the multiple local visual features reach a maximum value; The first observation direction is calibrated according to lighting information in the second image data to determine a second observation direction, the second observation direction is used as a shooting angle, and the intersection of the second observation direction and the initial electronic map is used as a shooting position, wherein the lighting information is a brightness variation range of the second image data at different viewing angles.

7. A path planning method for an inspection robot according to claim 6, characterized in that: Performing a structural sensitivity analysis on the local visual features to obtain a structural sensitivity distribution includes: Calculating pixel gradients and spatial arrangement features of the local visual features; Comparing the image structure changes of the pixel gradients and spatial arrangement features frame by frame to obtain a structural change amount, wherein the structural change amount includes a clarity change, a structural integrity change, and an image detail preservation degree; According to the variation trend of the structural variation in the viewing angle dimension, a corresponding sensitivity curve is constructed, and according to the slope of the sensitivity curve, a structural sensitivity distribution is generated.

8. A path planning method for an inspection robot according to claim 6, characterized in that: Inverting a first observation direction of the local visual feature according to the structural sensitivity distribution includes: The sensitivity values ​​corresponding to each shooting direction in the structural sensitivity distribution are combined into a spatial sensitivity point set, and the structural sensitivity change rate between each shooting direction is calculated based on the spatial sensitivity point set to generate a spatial sensitivity gradient field; In the spatial sensitivity gradient field, a structural response path is calculated based on the gradient rising direction, wherein the structural response path is a direction sequence with the largest response change in the gradient rising path; A change rate analysis is performed on the structural response path to obtain an extreme point of the structural sensitivity change rate, and a direction corresponding to the extreme point is used as the first observation direction of the local visual feature.

9. A path planning method for an inspection robot according to claim 6, characterized in that: Calibrating the first observation direction according to illumination information in the second image data to determine the second observation direction includes: Extracting brightness gradient values ​​of corresponding illumination information according to a plurality of adjacent viewing angles in the first observation direction to obtain a local brightness change sequence; Calculating the brightness fluctuation amplitude corresponding to the first observation direction, and if the brightness fluctuation amplitude is less than the amplitude mean of the local brightness change sequence, using the first observation direction as the second observation direction; If the brightness fluctuation amplitude is greater than or equal to the amplitude mean of the local brightness change sequence, constructing a brightness fluctuation trend vector field based on the local brightness change sequence; calculating a directional distribution trend of a minimum brightness disturbance amplitude in the brightness fluctuation trend vector field, and calculating a calibration vector based on the directional distribution trend, wherein the calibration vector represents a minimum illumination interference adjustment direction relative to the first observation direction; Direction calibration is performed on the first observation direction according to the calibration vector to obtain a second observation direction.

10. A path planning system for an inspection robot, used to implement a path planning method for an inspection robot according to any one of claims 1 to 9, characterized in that: The system comprises: An image acquisition module is used to control the inspection robot to perform a first-stage inspection according to the initial electronic map and obtain first image data and corresponding posture data; an image processing module, configured to process the first image data, identify an inspection target, and register the first image data acquired from multiple shooting positions of the same inspection target to obtain second image data; a shooting configuration generation module, which performs structural sensitivity analysis and observation direction inversion based on local visual features in the second image data, calibrates the first observation direction in combination with illumination information, and determines the shooting position and shooting angle of each inspection target as the shooting configuration; The path planning module is used to construct a cost function using the shooting configuration of each inspection target as a path node and generate an inspection path through a multi-objective optimization algorithm; The control execution module is used to send the inspection path to the inspection robot and control it to complete the target information collection in sequence according to the inspection path.

Images