Inspection robot complex obstacle avoidance method, control device and inspection robot

Abstract

The embodiment of the invention provides a complex obstacle avoidance method for an inspection robot, a control device and the inspection robot, and the method comprises the steps: S1, obtaining environment data through a multi-source sensor group, the environment data comprising an RGB image of the surrounding view of the inspection robot and millimeter wave radar data; s2, path planning is carried out based on an improved dynamic window method, and improvement measures comprise the steps that an obstacle semantic weight item is added in a track scoring function, and the RGB image is input into an improved BiSeNetV2 lightweight semantic segmentation model to obtain an obstacle type; inputting the millimeter wave radar data into an improved LSTM (Long Short Term Memory) model to obtain a movement track of the obstacle; and S3, performing path planning on the inspection robot based on the obstacle type and the movement track. According to the method, the obstacle semantic weight item is added in the trajectory scoring function, and the LSTM provides dynamic trajectory prediction to improve the dynamic window algorithm, so that the obstacle avoidance capability of the inspection robot to the complex obstacle in the complex scene is improved.

Description

Technical Field

[0001] The present application relates to the field of robot obstacle avoidance, and in particular to a complex obstacle avoidance method for an inspection robot, a control device, and an inspection robot. Background Art

[0002] In the field of industrial automation and intelligent inspection, inspection robots need to achieve reliable obstacle avoidance in complex and dynamic environments, and their technical reliability is directly related to equipment safety and operation and maintenance efficiency. However, traditional obstacle avoidance solutions have significant limitations when dealing with irregular and special obstacles: for slender obstacles, conventional visual algorithms are prone to edge breakage, missed detection, and other problems due to insufficient feature extraction capabilities, causing robots to accidentally touch cables and cause accidents; for dynamic obstacles, trajectory prediction models based on a single sensor are difficult to accurately capture sudden movement trends, and the delayed response of path planning often forces robots to brake urgently, seriously affecting the continuity of operations. In addition, existing methods have bottlenecks in multimodal data fusion and embedded real-time computing, making it difficult to balance detection accuracy and execution efficiency, and their performance degrades sharply in complex scenarios.

[0003] Therefore, there is an urgent need for an obstacle avoidance solution that can accurately identify irregular obstacles, predict dynamic targets in real time, and efficiently utilize resources to support the autonomous and highly reliable operation of inspection robots in complex industrial scenarios. Summary of the Invention

[0004] The present application provides a complex obstacle avoidance method for an inspection robot, a control device, and an inspection robot. The method improves the dynamic window algorithm by adding an obstacle semantic weight term to the trajectory scoring function and providing dynamic trajectory prediction through LSTM, thereby improving the inspection robot's ability to avoid complex obstacles in complex scenarios.

[0005] In a first aspect, a method for avoiding complex obstacles of an inspection robot is provided, the method comprising:

[0006] S1: Acquire environmental data through a multi-source sensor group, wherein the environmental data includes an RGB image of the inspection robot's surrounding field of view and millimeter-wave radar data;

[0007] S2: Path planning is performed based on an improved dynamic window method. The improvement measures include: adding an obstacle semantic weight term to the trajectory scoring function, inputting the RGB image into an improved BiSeNetV2 lightweight semantic segmentation model to obtain the obstacle type; and inputting the millimeter-wave radar data into an improved LSTM model to obtain the movement trajectory of the obstacle.

[0008] S3: Planning a path for the inspection robot based on the obstacle type and the motion trajectory to avoid obstacles.

[0009] It should be understood that by fusing visual images with millimeter-wave radar data, combining it with an improved BiSeNetV2 semantic segmentation model to identify irregular obstacle types, and using an improved LSTM model to predict dynamic obstacle trajectories, a multimodal collaborative decision-making mechanism is formed by introducing semantic weight terms and predicted trajectories into the path planning of the dynamic window algorithm. This method effectively solves the problems of traditional obstacle avoidance algorithms, such as high missed detection rates for complex obstacles and delayed response to moving obstacles. It effectively reduces the collision risk of inspection robots during obstacle avoidance, while ensuring the safe passage of inspection robots through complex areas and improving inspection safety.

[0010] In conjunction with the first aspect, in certain implementations of the first aspect, the improvement measures for the BiSeNetV2 lightweight semantic segmentation model include:

[0011] An anisotropic convolution enhancement module is added at the end of the Detail branch to capture the detailed features of complex obstacles.

[0012] An edge-guided attention module is added to the Context branch to fuse low-level edge features with high-level semantic features.

[0013] It should be understood that the advantage of BiSeNetV2 as a lightweight semantic segmentation model lies in its unique dual-branch architecture. The Detail branch retains high-resolution spatial details through a shallow network, which can accurately capture the detailed features of irregular obstacles such as cables and branches. The Context branch extracts semantic information through a lightweight design, ensuring real-time performance while avoiding the problem of slender target breakage caused by excessive downsampling in traditional models. Compared with other real-time models (such as ICNet and Fast-SCNN), its dual-path feature fusion mechanism is more adapted to the characteristics of irregular obstacles in inspection scenarios, which have both local continuity and global morphological complexity, and is very suitable for the application scenarios of this application.

[0014] It should also be understood that by inserting an anisotropic convolution enhancement module into the Detail branch of BiSeNetV2 to capture the continuous edge features of irregular obstacles, and introducing an edge-guided attention module into the Context branch, the underlying edge information is used to correct high-level semantic features, and the edge integrity of irregular obstacle targets is significantly improved. This can significantly improve the feature extraction capability of the model, thereby improving the inspection robot's resolution accuracy for complex obstacles.

[0015] In combination with the first aspect, in some implementations of the first aspect, the anisotropic convolution enhancement module includes two parallel branches:

[0016] A first branch comprising two convolutional modules connected in series;

[0017] The second branch includes a deformable convolution module, and the offset learning rate of the deformable convolution module is set to 0.1 times the baseline value.

[0018] It should be understood that anisotropic convolution is a convolution operation designed for multidimensional data (such as 3D voxels, image sequences, etc.). Its core lies in enhancing the model's ability to capture complex geometric or semantic features through directionally differentiated kernel design, which is very suitable for the application scenario of this application. The ordinary convolution branches connected in series in the anisotropic convolution enhancement module extract horizontal / vertical continuity features, and the deformable convolution branches adapt to irregular obstacles. The parallel structure of the two enables the model to enhance both geometric sensitivity and deformation robustness to slender targets at a low computing power cost.

[0019] In conjunction with the first aspect, in certain implementations of the first aspect, the implementation of the edge-guided attention module includes:

[0020] Extracting underlying features from the Detail branch and processing them with the Laplacian operator to obtain an edge response feature map;

[0021] The edge response feature map generates a spatial attention weight through convolution and activation function;

[0022] The spatial attention weight is multiplied point by point with the semantic features of the Context branch, and the enhanced feature map is output.

[0023] It should be understood that the edge-guided attention module extracts the underlying edge features through the Laplacian operator, generates spatial attention weights and fuses them with high-level semantic features, so that the model retains the edge details of slender obstacles in the deep network, solves the edge blurring problem caused by multiple downsampling in traditional semantic segmentation, and effectively avoids path planning oscillations caused by edge misjudgment.

[0024] In combination with the first aspect, in certain implementations of the first aspect, the BiSeNetV2 lightweight semantic segmentation model adopts a dynamic morphological perception loss function to increase the loss weight of obstacle samples whose aspect ratio is greater than a first threshold.

[0025] In combination with the first aspect, in certain implementations of the first aspect, the weight calculation method of the dynamic morphological perception loss function is: for the obstacle instance in the training sample, calculate its minimum circumscribed rectangle aspect ratio R; when R is greater than a second threshold, the loss weight ω is dynamically increased.

[0026] It should be understood that the dynamic morphological perception loss function dynamically adjusts training weights based on the aspect ratio of obstacles, forcing the model to focus on easily overlooked targets. A dynamic weight calculation mechanism based on the minimum bounding rectangle aspect ratio accurately quantifies the morphological characteristics of obstacles. When the aspect ratio exceeds a threshold, the loss weight is automatically amplified, allowing the model to prioritize learning the discriminative features of slender targets. This design directs network resources toward difficult samples during the training phase, effectively improving the model's ability to detect irregular obstacles without affecting the detection performance of conventional obstacles.

[0027] In combination with the first aspect, in certain implementations of the first aspect, the radar data includes: information on the distance between the inspection robot and the obstacle and the speed of the obstacle.

[0028] In conjunction with the first aspect, in some implementations of the first aspect, the improved LSTM model includes:

[0029] A radar feature compression layer is added before the LSTM layer to convert the millimeter-wave radar data into a time-series dense feature vector;

[0030] Add the spatiotemporal attention module to the LSTM hidden layer, and the calculation formula is:

[0031] α t =Softmax(W q h t-1 W k H),

[0032] Among them, H is the historical state data matrix of the obstacle, h t-1 is the state of the obstacle at the previous moment, W q , W k is the learnable parameter matrix, α t Used to weight key historical moment information.

[0033] It should be understood that the improved LSTM model converts sparse point clouds into dense time series features through the radar feature compression layer, and introduces the spatiotemporal attention module to weight key historical frame information, thereby solving the problem of traditional LSTM's lag in predicting sudden motions and providing a reasonable motion trajectory prediction for dynamic obstacles, thereby providing better support for inspection robots to avoid obstacles.

[0034] In a second aspect, a control device is provided, which includes a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions in the memory, so that the method described in any implementation method of the first aspect is executed.

[0035] In a third aspect, a patrol robot is provided, wherein the patrol robot includes the control device as described in the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart for implementing a complex obstacle avoidance method for an inspection robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to be limiting of the present application. As used in the specification of the present application and the appended claims, the singular expressions "a", "an", "", "above", "the" and "this" are intended to also include expressions such as "one or more", unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B may represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B may be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship.

[0038] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0039] In the fields of industrial automation and intelligent inspection, inspection robots face the technical challenge of obstacle avoidance in complex dynamic environments. Traditional vision algorithms, due to their insufficient feature representation capabilities, are prone to edge breakage and missed detections in irregular obstacles, leading to collision risks. In dynamic obstacle scenarios, trajectory prediction models based on single-modal sensors struggle to capture sudden motion trends, and delayed path planning responses force frequent emergency braking, disrupting operational continuity. Furthermore, existing solutions suffer from feature mismatches when fusing multi-source heterogeneous data, and the computing power constraints of embedded platforms lead to insufficient real-time performance. The difficulty in balancing accuracy and efficiency is particularly pronounced in complex scenarios, causing exponential degradation in system performance.

[0040] The embodiments of the present application provide a complex obstacle avoidance method for an inspection robot, a control device, and an inspection robot to overcome the above-mentioned problems.

[0041] The technical solutions of the embodiments of the present application will be described below with reference to the accompanying drawings.

[0042] Figure 1 A flowchart of a method for implementing complex obstacle avoidance for an inspection robot provided in an embodiment of the present application. In some examples, the method includes:

[0043] S1: Acquire environmental data through a multi-source sensor group, wherein the environmental data includes an RGB image of the inspection robot's surrounding field of view and millimeter-wave radar data;

[0044] S2: Path planning is performed based on an improved dynamic window method. The improvement measures include: adding an obstacle semantic weight term to the trajectory scoring function, inputting the RGB image into an improved BiSeNetV2 lightweight semantic segmentation model to obtain the obstacle type; and inputting the millimeter-wave radar data into an improved LSTM model to obtain the movement trajectory of the obstacle.

[0045] S3: Planning a path for the inspection robot based on the obstacle type and the motion trajectory to avoid obstacles.

[0046] In some examples, improvements to the BiSeNetV2 lightweight semantic segmentation model include:

[0047] An anisotropic convolution enhancement module is added at the end of the Detail branch to capture the detailed features of complex obstacles.

[0048] An edge-guided attention module is added to the Context branch to fuse low-level edge features with high-level semantic features.

[0049] In some examples, the anisotropic convolution enhancement module includes two parallel branches:

[0050] A first branch comprising two convolutional modules connected in series;

[0051] The second branch includes a deformable convolution module, and the offset learning rate of the deformable convolution module is set to 0.1 times the baseline value.

[0052] In some examples, the implementation of the edge-guided attention module includes:

[0053] Extracting underlying features from the Detail branch and processing them with the Laplacian operator to obtain an edge response feature map;

[0054] The edge response feature map generates a spatial attention weight through convolution and activation function;

[0055] The spatial attention weight is multiplied point by point with the semantic features of the Context branch, and the enhanced feature map is output.

[0056] In one possible implementation, the underlying features are extracted from stage 1 of the Detail branch and processed by the Laplacian operator to obtain an edge response map; the edge map is subjected to 3×3 convolution and Sigmoid to generate spatial attention weights; the weight matrix is ​​multiplied point by point with the semantic features of stage 3 of the Context branch, and the enhanced feature map is output.

[0057] In some examples, the BiSeNetV2 lightweight semantic segmentation model adopts a dynamic morphological perception loss function to increase the loss weight of obstacle samples with an aspect ratio greater than a first threshold.

[0058] In some examples, the weight calculation method of the dynamic morphological perception loss function is: for obstacle instances in the training sample, the minimum circumscribed rectangle aspect ratio R is calculated, and when R is greater than a second threshold, the loss weight ω is dynamically increased.

[0059] In one possible implementation, the second threshold is 3, and the loss weight ω is dynamically increased as follows:

[0060] ω=1+0.4log2(R).

[0061] In some examples, the radar data includes: information about the distance between the inspection robot and the obstacle and the speed of the obstacle.

[0062] In one possible implementation, radar data processing includes: performing density clustering (DBSCAN algorithm) on the original point cloud to remove noise points; extracting cluster centroid coordinates, velocity vectors, and bounding box aspect ratio features; and organizing these features into a time-series input sequence according to time windows.

[0063] In some examples, the improved LSTM model includes:

[0064] A radar feature compression layer is added before the LSTM layer to convert the millimeter-wave radar data into a time-series dense feature vector;

[0065] Add the spatiotemporal attention module to the LSTM hidden layer, and the calculation formula is:

[0066] α t =Softmax(W q h t-1 W k H),

[0067] Among them, H is the historical state data matrix of the obstacle, h t-1 is the state of the obstacle at the previous moment, W q , W k is the learnable parameter matrix, α t Used to weight key historical moment information.

[0068] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed in the present invention should be included in the protection scope recorded in the claims.

Claims

1. A complex obstacle avoidance method for an inspection robot, characterized in that: The method comprises: S1: Acquire environmental data through a multi-source sensor group, wherein the environmental data includes an RGB image of the inspection robot's surrounding field of view and millimeter-wave radar data; S2: Path planning is performed based on an improved dynamic window method. The improvement measures include: adding an obstacle semantic weight term to the trajectory scoring function, inputting the RGB image into an improved BiSeNetV2 lightweight semantic segmentation model to obtain the obstacle type; and inputting the millimeter-wave radar data into an improved LSTM model to obtain the movement trajectory of the obstacle. S3: Planning a path for the inspection robot based on the obstacle type and the motion trajectory to avoid obstacles.

2. The method according to claim 1, characterized in that Improvements to the BiSeNetV2 lightweight semantic segmentation model include: An anisotropic convolution enhancement module is added at the end of the Detail branch to capture the detailed features of complex obstacles. An edge-guided attention module is added to the Context branch to fuse low-level edge features with high-level semantic features.

3. The method according to claim 2, characterized in that The anisotropic convolution enhancement module consists of two parallel branches: A first branch comprising two convolutional modules connected in series; The second branch includes a deformable convolution module, and the offset learning rate of the deformable convolution module is set to 0.1 times the baseline value.

4. The method according to claim 2, characterized in that The implementation of the edge-guided attention module includes: Extracting underlying features from the Detail branch and processing them with the Laplacian operator to obtain an edge response feature map; The edge response feature map generates a spatial attention weight through convolution and activation function; The spatial attention weight is multiplied point by point with the semantic features of the Context branch, and the enhanced feature map is output.

5. The method according to claim 2, characterized in that The BiSeNetV2 lightweight semantic segmentation model adopts a dynamic morphological perception loss function to increase the loss weight of obstacle samples with an aspect ratio greater than a first threshold.

6. The method according to claim 5, characterized in that The weight calculation method of the dynamic morphological perception loss function is as follows: for the obstacle instance in the training sample, the minimum circumscribed rectangle aspect ratio R is calculated; when R is greater than a second threshold, the loss weight ω is dynamically increased.

7. The method according to claim 1, characterized in that The radar data includes: the distance between the inspection robot and the obstacle and the speed information of the obstacle.

8. The method according to claim 1, characterized in that The improved LSTM model includes: A radar feature compression layer is added before the LSTM layer to convert the millimeter-wave radar data into a time-series dense feature vector; Add the spatiotemporal attention module to the LSTM hidden layer, and the calculation formula is: α t =Softmax(W q h t-1 ·W k H), Among them, H is the historical state data matrix of the obstacle, h t-1 is the state of the obstacle at the previous moment, W q , W k is the learnable parameter matrix, α t Used to weight key historical moment information.

9. A control device, characterized in that: The method comprises a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions in the memory, so that the method according to any one of claims 1 to 8 is performed.

10. A patrol robot, characterized in that: The inspection robot includes the control device according to claim 9.

Images