Path planning method and system for dredging robot in harsh environment based on sonar images

Abstract

The invention relates to the technical field of robot path planning, and discloses a path planning method and system for a dredging robot in a harsh environment based on sonar images, comprising obtaining and preprocessing a sonar image of a low-visibility environment in a harsh environment; segmenting the preprocessed image and extracting key path points; planning a robot motion path based on the key path points to obtain a candidate path; calculating a comprehensive path complexity index, an environmental adaptability index and a time energy consumption index of the candidate path, and evaluating the path; if the evaluation result is a path to be optimized, adjusting a local path segment and re-evaluating; the invention realizes autonomous navigation of a dredging robot in a low-visibility environment in a complex and harsh environment by using sonar imaging technology in combination with a path planning algorithm, thereby improving the efficiency and safety of dredging operations.

Description

Technical Field

[0001] The present invention relates to the technical field of robot path planning, and more specifically, to a path planning method and system for a dredging robot in a harsh environment based on sonar images. Background Art

[0002] With the continuous advancement of urbanization, drainage system maintenance and dredging in low-visibility environments are becoming increasingly important as an important part of urban drainage management. Traditional low-visibility dredging operations in harsh environments are inefficient, labor-intensive, and have certain safety hazards. To solve the above problems, more and more studies have begun to explore the use of intelligent dredging robots to automate low-visibility dredging operations in harsh environments.

[0003] In terms of low-visibility environmental perception in complex and harsh environments, sonar imaging technology has attracted widespread attention due to its imaging advantages in harsh environments such as low visibility and strong background noise. Sonar can obtain high-resolution internal images of low-visibility structures in harsh environments by emitting sound waves and receiving the reflections of sound waves on low-visibility structures in harsh environments, providing rich environmental information for robot navigation. Compared with the limitations of optical images that are easily affected by lighting and turbidity, sonar imaging can overcome the complexity of low-visibility environments in harsh environments and provide stable and reliable perception data.

[0004] The Chinese patent with publication number CN118464027A discloses a dredging path planning system and method for a dredging robot. The system analyzes the dredging path through a path planning screening module to determine the optimal dredging path, and can realize the judgment and correction of path deviation during the dredging process. However, the patent mainly focuses on dredging path planning based on path screening, and does not consider the problems of robot autonomous navigation and path optimization in low-visibility environments under complex and harsh environments, especially the lack of effective use of environmental perception data such as sonar images.

[0005] The existing path planning methods for dredging robots lack the application of sonar imaging and other technologies in low-visibility environmental perception in complex and harsh environments, and lack effective image processing methods to extract navigation paths from sonar images. Path planning and optimization lack comprehensive evaluation of multiple indicators, and do not fully consider the trade-offs of factors such as path complexity, environmental adaptability, and energy efficiency. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art, the present invention provides a path planning method and system for a dredging robot in harsh environments based on sonar images. The method and system extract key path points by acquiring sonar images of low-visibility environments in harsh environments and performing image segmentation processing. The path is evaluated and optimized by comprehensively considering multiple indicators such as path complexity, environmental adaptability, time and energy consumption, and a dynamic adjustment strategy for local segments of the path is introduced to achieve autonomous navigation of the dredging robot in low-visibility environments in complex and harsh environments.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] Acquire first image data of a low-visibility environment in a harsh environment, and preprocess the first image data to obtain second image data; segment the second image data to obtain third image data; the first image data is sonar image data;

[0009] Extract key path points in a low-visibility environment under a harsh environment according to the third image data; plan the robot motion path according to the key path points to obtain a path to be selected and first path data of the path to be selected; calculate a comprehensive path complexity index, a comprehensive environmental adaptability index and a time energy consumption index according to the first path data of the path to be selected, evaluate the path to be selected, and obtain an evaluation result of the path to be selected, wherein the evaluation result of the path to be selected includes an executable path and a path to be optimized;

[0010] If the evaluation result of the path to be selected is the path to be optimized, a local path segment adjustment is performed on the path to be optimized to obtain an adjusted path and second path data of the adjusted path, and the adjusted path is evaluated.

[0011] Further, segmenting the second image data includes:

[0012] Performing local statistical analysis on the second image data, calculating the mean and variance in each pixel neighborhood, and obtaining first mean data and first variance data;

[0013] Generate a first threshold for image segmentation according to the first mean data and the first variance data;

[0014] A preset multi-scale and multi-directional texture feature extraction operator is obtained, and combined with the first threshold, segmentation is performed on the second image data to obtain binary image data of the region of interest, which is marked as third image data, wherein the texture feature extraction operator includes a Gabor filter.

[0015] Furthermore, the extracting of key path points in a low-visibility environment under a harsh environment includes:

[0016] Performing morphological processing on the third image data to obtain fourth image data;

[0017] Extracting skeleton lines of connected domains in the fourth image data to obtain skeleton line image data;

[0018] Search endpoints, intersections and inflection points in the skeleton line image data, mark them as key path points, obtain a set of key path points, and extract coordinate information of each key path point;

[0019] Planning the robot motion path according to the key path points includes:

[0020] Search for the closest key path point pair in the key path point set, connect them to form a local path segment, and construct a local path set;

[0021] Topologically sort the local path set to obtain the global path sequence;

[0022] The global path sequence is smoothly optimized to obtain the candidate path and the first path data of the candidate path.

[0023] Further, searching for endpoints, intersections and inflection points in the skeleton line image data includes:

[0024] Traverse each pixel of the skeleton line image data ,right of The 8 neighboring pixels in the neighborhood arrive Analyze and count the number of skeleton pixels ; (x,y) is pixel The coordinates of , then Marked as an endpoint; if , then Mark as intersection;

[0025] exist Neighborhood according to Two connected skeleton pixels and The coordinates of To skeleton pixels Direction vector and from To skeleton pixels Direction vector ; Calculate the direction vector and Angle , set the inflection point threshold angle ,if , then Marked as inflection point.

[0026] Further, the calculating of the comprehensive path complexity index, the comprehensive environment adaptability index and the time energy consumption index according to the first path data of the to-be-selected path includes:

[0027] Calculate the path length, curvature, and number of turns according to the first path data to obtain path complexity data; calculate corresponding complexity indicators according to the path length, curvature, and number of turns, including length complexity index, curvature complexity index, and turn complexity index; perform weighted summation on the length complexity index, curvature complexity index, and turn complexity index to obtain a comprehensive path complexity index L1;

[0028] Analyze the distance between the path and the low-visibility boundary in the harsh environment and the distance between the path and the obstacle according to the first path data to obtain environmental adaptability data; calculate the boundary fitness index and the obstacle fitness index according to the distance between the path and the low-visibility boundary in the harsh environment and the distance between the path and the obstacle; perform weighted summation of the boundary fitness index and the obstacle fitness index to obtain the comprehensive environmental adaptability index L2;

[0029] The path passing time is predicted according to the first path data to obtain energy consumption data; and the time energy consumption index L3 is calculated according to the path passing time.

[0030] Further, the adjusting of the local path segments of the path to be optimized includes:

[0031] Searching for local extreme value points in the path to be optimized, wherein the local extreme value points include local highest points, local lowest points and curvature maximum points;

[0032] Taking the local extreme point as the center, a local path search is performed in its neighborhood to find a feasible path that meets the constraint conditions; the constraint conditions include that the deviation distance from the original path is less than the distance threshold, the line length increment is less than the increment threshold, and the curvature change is less than the change threshold;

[0033] The feasible path that meets the constraints replaces the corresponding local path segment in the original path to obtain the adjusted path.

[0034] Furthermore, the evaluating the path to be selected includes:

[0035] The path complexity threshold Q1, the environmental adaptability threshold Q2 and the time energy consumption threshold Q3 are preset. If L1≤Q1, L2≥Q2, and L3≤Q3, the evaluation result of the selected path is an executable path; otherwise, the evaluation result of the selected path is a path to be optimized;

[0036] The evaluation of the adjusted path includes:

[0037] According to the second path data of the adjusted path, the second comprehensive path complexity index L4, the second comprehensive environmental adaptability index L5 and the second time energy consumption index L6 are calculated. If L4≤Q1, and L5≥Q2, and L6≤Q3, the adjusted path is an executable path. Otherwise, the distance threshold, increment threshold and change threshold are adjusted, and the local path fragments of the adjusted path are adjusted until an executable path is obtained.

[0038] A path planning system for a dredging robot in a harsh environment based on sonar images, which is used to implement the path planning method for a dredging robot in a harsh environment based on sonar images, and the system comprises:

[0039] Image processing module: used to obtain first image data of a low-visibility environment in a harsh environment, and pre-process the first image data to obtain second image data; segment the second image data to obtain third image data;

[0040] A path evaluation module: used for extracting key path points in a low-visibility environment under a harsh environment according to the third image data; planning the robot motion path according to the key path points, and obtaining a path to be selected and first path data of the path to be selected; calculating a comprehensive path complexity index, a comprehensive environmental adaptability index, and a time energy consumption index according to the first path data of the path to be selected, evaluating the path to be selected, and obtaining an evaluation result of the path to be selected, wherein the evaluation result of the path to be selected includes an executable path and a path to be optimized;

[0041] Path optimization module: if the evaluation result of the path to be selected is the path to be optimized, then adjust the local path segment of the path to be optimized, obtain the adjusted path and the second path data of the adjusted path, and evaluate the adjusted path.

[0042] An electronic device comprises a memory, a central processing unit, and a computer program stored in the memory and executable on the central processing unit. When the central processing unit executes the computer program, the path planning method for a dredging robot in a harsh environment based on sonar images is implemented.

[0043] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the above-mentioned path planning method for a dredging robot in a harsh environment based on sonar images.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] Aiming at the problems of low quality and low resolution of sonar images, the present invention adopts a segmentation strategy of local adaptive threshold and texture feature fusion, automatically generates an optimal segmentation threshold, combines Gabor texture to supplement grayscale information from the frequency and direction angles, enhances the environmental adaptability and detail depiction ability of the segmentation algorithm, and realizes high-precision automatic extraction of low-visibility areas of interest in harsh environments; through skeleton line extraction and key path point search, the main direction, branch structure and key position of the area of ​​interest are intuitively reflected, providing an important basis for planning the motion path; by introducing evaluation indicators such as comprehensive path complexity, environmental adaptability and time energy consumption, the feasibility and advantages and disadvantages of the selected path can be evaluated from multiple angles; the local path fragments of the path evaluated as to be optimized are adjusted and optimized, while ensuring the continuous reachability of the path, the path complexity is further reduced, the environmental adaptability is improved, time and energy consumption are saved, and a better motion path solution is obtained; the method can autonomously plan and optimize the motion path of the dredging robot in a low-visibility environment in a complex and harsh environment according to sensor data such as sonar images, reduce manual intervention, and improve the automation and intelligence level of low-visibility dredging operations in harsh environments. The optimized path takes into account accessibility, safety and economy, and can provide high-quality decision-making basis for the robot's motion control and task planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 It is a principle flow chart of the path planning method of the harsh environment dredging robot based on sonar images in the present invention;

[0048] Figure 2 A flow chart of a method for segmenting second image data in a path planning method for a dredging robot in a harsh environment based on sonar images of the present invention;

[0049] Figure 3 A flowchart of a method for extracting key path points in a low-visibility environment under a harsh environment according to third image data in a path planning method for a dredging robot under a harsh environment based on sonar images of the present invention;

[0050] Figure 4 A flow chart of a method for obtaining a candidate path and first path data of the candidate path in a path planning method for a dredging robot in a harsh environment based on sonar images of the present invention;

[0051] Figure 5 A flow chart of a method for calculating a comprehensive path complexity index according to first path data in a path planning method for a dredging robot in a harsh environment based on sonar images of the present invention;

[0052] Figure 6 A flow chart of a method for calculating a comprehensive environmental adaptability index according to first path data in a path planning method for a dredging robot in a harsh environment based on sonar images of the present invention;

[0053] Figure 7 A flow chart of a method for calculating a time energy consumption index according to first path data in a path planning method for a dredging robot in a harsh environment based on sonar images of the present invention;

[0054] Figure 8 A flowchart of a method for adjusting a local path segment of a path to be optimized to obtain an adjusted path in a path planning method for a dredging robot in a harsh environment based on sonar images of the present invention;

[0055] Fig. 9 This is a functional module diagram of the path planning system of the harsh environment dredging robot based on sonar images in the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] Example 1

[0058] See also Figure 1 As shown, this embodiment provides a path planning method for a dredging robot in a harsh environment based on sonar images, including:

[0059] Step S1000, obtaining first image data of a low-visibility environment in a harsh environment, and preprocessing the first image data to obtain second image data; segmenting the second image data to obtain third image data;

[0060] Furthermore, step S1000 includes:

[0061] Step S1100, acquiring first image data of a low-visibility environment in a harsh environment, and preprocessing the first image data to obtain second image data; the first image data is sonar image data;

[0062] Specifically, the first image data is sonar image data of a low-visibility environment in a harsh environment obtained by sonar imaging technology. Sonar imaging is an imaging technology that uses the propagation characteristics of sound waves in underwater media to transmit sound waves and receive their reflected echoes, measure the flight time of the sound waves from transmission to reception, thereby obtaining information such as the spatial position and shape of the target object and reconstructing it into an image. Unlike visible light imaging, sonar imaging does not rely on external lighting conditions and can obtain clearer image data in complex underwater environments such as darkness and turbidity.

[0063] The first image data is preprocessed to obtain the second image data. The purpose of preprocessing is to improve the quality of sonar images, remove noise interference in the image, enhance the contrast and clarity of the image, and prepare for subsequent image segmentation and feature extraction. Preprocessing mainly includes the following steps:

[0064] (1) Image denoising: Sonar images are susceptible to various noise interferences, such as speckle noise and electronic noise, which can lead to image quality degradation. An adaptive median filter algorithm is used to denoise the image. The algorithm adaptively adjusts the filter window size and filter strength based on the statistical characteristics of the local area of ​​the image, which can effectively suppress noise while maintaining image edge and detail information.

[0065] (2) Pseudo-color enhancement: Sonar images are usually grayscale images and lack color information. Through pseudo-color enhancement processing, the grayscale value is mapped to the color space, giving the image rich color information. Common pseudo-color mapping methods include heat map mapping, rainbow map mapping, etc. Pseudo-color enhancement not only improves the visual effect of the image, but also helps to highlight the area of ​​interest.

[0066] (3) Histogram equalization: The grayscale distribution of sonar images may be relatively concentrated, resulting in low image contrast. Through histogram equalization processing, the grayscale dynamic range of the image is stretched and the overall contrast of the image is enhanced. Histogram equalization redistributes the grayscale values ​​of image pixels so that the grayscale histogram after equalization presents a uniform distribution, making the image details clearer and more discernible.

[0067] Through the above preprocessing steps, a second image data with higher quality and more prominent features can be obtained. Compared with the original first image data, the second image data has the following advantages: (1) Image noise is effectively suppressed, and the image is smoother and cleaner, which is conducive to extracting stable and reliable features. (2) The image after pseudo-color enhancement is more colorful, and the region of interest is more eye-catching, which is convenient for image understanding and analysis. (3) After histogram equalization, the image contrast is enhanced, and the foreground object and background are more clearly distinguished, which is conducive to image segmentation.

[0068] High-quality second image data is the basis for low-visibility dredging robots in harsh environments to perform environmental perception and path planning. Noise removal and contrast enhancement make it more accurate and efficient to extract low-visibility environmental information in harsh environments from images. Pseudo-color processing improves the visual expression of the image and provides the robot with more intuitive and rich environmental information. The pre-processed high-quality images will provide more reliable data support for subsequent tasks such as positioning, obstacle avoidance, and path planning of low-visibility dredging robots in harsh environments, which will help improve their autonomous operation capabilities and achieve more intelligent and efficient low-visibility dredging operations in harsh environments.

[0069] Step S1200, segmenting the second image data to obtain third image data.

[0070] Furthermore, if Figure 2 As shown, step S1200 includes:

[0071] Step S1210, performing local statistical analysis on the second image data, calculating the mean and variance in each pixel neighborhood, and obtaining first mean data and first variance data;

[0072] Step S1220, dynamically generating a first threshold for image segmentation according to the first mean data and the first variance data; the generation algorithm of the first threshold includes an Otsu threshold method and a maximum entropy threshold method;

[0073] Step S1230, obtaining a preset multi-scale and multi-directional texture feature extraction operator, and combining it with the first threshold, performing segmentation on the second image data to obtain binary image data of the region of interest, marked as third image data; the texture feature extraction operator includes a Gabor filter.

[0074] Specifically, local statistical analysis refers to the calculation of mean and variance in the local neighborhood of the image. Commonly used local neighborhoods include square windows of odd sizes such as 3×3, 5×5, and 7×7. The mean reflects the average level of pixel grayscale in the neighborhood, representing the average brightness of a local area in the image; the variance reflects the discrete degree of pixel grayscale in the neighborhood and the texture complexity of the image, representing the drastic degree of brightness change in the local area. These two statistics can effectively characterize the grayscale distribution characteristics of the local area of ​​the image. In order to speed up the calculation, the integral graph method can be used to obtain the mean and variance of the neighborhood of any size by scanning the image once. Applying local statistical analysis to sonar images can obtain the mean and variance of the echo intensity of the local area of ​​the sonar image. Compared with optical images, sonar images have the characteristics of low signal-to-noise ratio and low resolution. The local mean can reflect the average echo intensity of a certain area and represent the material properties of the area. For example, the echo mean of the low-visibility sediment area in the harsh environment is low, while the echo mean of the low-visibility wall area in the harsh environment is high. The local variance reflects the fluctuation of echo intensity and represents the roughness of the regional surface. The bottom textures such as gravel and silt are relatively rough and have large variances, while the low-visibility concrete surface in smooth harsh environments has a small variance. Using the two statistical features of mean and variance, low-visibility areas in harsh environments with different materials and different texture complexities can be distinguished, providing a basis for subsequent image segmentation. Local statistical analysis can overcome the defects of low sonar image quality, extract robust and reliable regional features, and enhance the environmental adaptability of the segmentation algorithm.

[0075] The Otsu threshold method and the maximum entropy threshold method can effectively deal with the problems of low image quality and low contrast. Sonar images are easily disturbed by speckle noise, electronic noise, etc., and the histogram shows multi-peak and uneven characteristics. The Otsu threshold method can adaptively divide the histogram into foreground and background by maximizing the inter-class variance, and can obtain a relatively reliable segmentation threshold even under noise interference. The maximum entropy threshold rule makes full use of the uncertainty of the image grayscale distribution, captures the amount of information contained in the image by maximizing the sum of foreground and background entropy, and can better separate the region of interest under low contrast conditions. Compared with the fixed empirical threshold, the dynamic threshold can automatically adjust the threshold size according to the statistical characteristics of the sonar image, avoid the blindness of manually setting the threshold, and improve the intelligence level of the segmentation method. At the same time, by choosing between the Otsu threshold method and the maximum entropy threshold method, the optimal threshold generation strategy can be adopted for sonar images of different qualities and different scenes, further enhancing the adaptability and robustness of the segmentation results. The automatic and dynamic generation of thresholds liberates manpower, improves segmentation efficiency, and reduces the risk of mis-segmentation. It is an important sign of intelligent sonar image segmentation.

[0076] For sonar images, the introduction of texture features is particularly important. Limited by the imaging mechanism, the resolution of sonar images is low, and the edge and contour information is not clear enough. The texture feature supplements the deficiency of the spatial domain from the frequency domain perspective, and can better reflect the structural pattern of the low-visibility environment in harsh environments. The texture of the low-visibility bottom mud area in harsh environments is relatively uniform and has no obvious directionality; while the texture of the four-wall area shows a certain directionality, which is consistent with the horizontal or vertical direction. Encoding these texture differences in Gabor features can more accurately distinguish the bottom mud area and the four-wall area. In addition, sediments of different particle sizes also have different texture coarseness and frequency characteristics. Sediment components such as gravel, silt, and humus will form a multi-scale representation of texture in sonar images. Through multi-frequency and multi-directional Gabor filter groups, this multi-scale texture change can be described, providing a new criterion for the classification of low-visibility bottom sediments in harsh environments. The combination of texture analysis and threshold segmentation expands the applicability of sonar image segmentation methods, overcomes the limitations of single threshold segmentation, and realizes the refined extraction of low-visibility key areas in harsh environments. Texture-guided sonar image segmentation is an innovative method for low-visibility environmental perception in harsh environments. It is of great significance for sonar image understanding and low-visibility structure analysis in harsh environments. It also provides high-quality basic data for subsequent robot path planning, target positioning and other tasks.

[0077] In summary, step S1200 adopts the strategy of local adaptive threshold and texture feature fusion, which has achieved innovation and breakthrough in the field of sonar image segmentation. In view of the problems of low sonar imaging quality and low image resolution, the local adaptive threshold makes full use of the statistical characteristics of the local area of ​​the image to automatically generate the optimal segmentation threshold of the foreground and background, which can effectively improve the segmentation robustness. The texture feature supplements the deficiency of grayscale information from the perspective of frequency and direction, and enhances the sensitivity of the segmentation algorithm to the properties of low-visibility materials and sediment particle size in harsh environments. At the same time, the dynamic selection of the Otsu threshold method and the maximum entropy threshold method further expands the scope of application of the segmentation method, and can flexibly deal with sonar images of different quality and noise interference. The Gabor texture finely depicts the low-visibility environment in harsh environments at multiple scales and directions, making the segmentation results closer to the real structure of low visibility in harsh environments. The perfect combination of multiple segmentation technologies realizes the high-precision and automatic extraction of low-visibility areas of interest in harsh environments in sonar images, providing a reliable perception basis for the intelligent decision-making of robots in complex environments. This method has important theoretical significance and practical application value for improving the environmental adaptability of low-visibility detection equipment in harsh environments and realizing fully autonomous and refined low-visibility dredging operations in harsh environments. With the continuous advancement of underwater sonar and image processing technology, the method of integrating threshold segmentation and texture features will continue to improve and play a greater role in the fields of smart pipe networks and underwater robots, safeguarding the intelligent operation and maintenance of urban underground pipelines.

[0078] Step S2000, based on the third image data, extract key path points in a low-visibility environment under a harsh environment; plan the robot's motion path based on the key path points to obtain a to-be-selected path and first path data of the to-be-selected path; calculate a comprehensive path complexity index, a comprehensive environmental adaptability index and a time energy consumption index based on the first path data of the to-be-selected path, evaluate the to-be-selected path, and obtain an evaluation result of the to-be-selected path, wherein the evaluation result of the to-be-selected path includes an executable path and a to-be-optimized path; if the evaluation result of the to-be-selected path is a to-be-optimized path, perform a local path segment adjustment on the to-be-optimized path to obtain an adjusted path and second path data of the adjusted path, and evaluate the adjusted path.

[0079] Furthermore, step S2000 includes:

[0080] Step S2100, extracting key path points in a low-visibility environment under a harsh environment according to the third image data;

[0081] Furthermore, if Figure 3 As shown, step S2100 includes:

[0082] Step S2110, performing morphological processing on the third image data to eliminate noise and interference areas in the binary image to obtain fourth image data; the morphological processing includes an erosion operation and a dilation operation;

[0083] Step S2120, using a skeleton line extraction algorithm to extract skeleton lines of connected domains in the fourth image data to obtain skeleton line image data;

[0084] Step S2130, searching for endpoints, intersections and inflection points in the skeleton line image data, marking them as key path points, obtaining a set of key path points, and extracting coordinate information of each key path point.

[0085] Further, step S2130 includes:

[0086] Step S2131, traverse each pixel of the skeleton line image data ,right of The 8 neighboring pixels in the neighborhood arrive Perform analysis and count the number of skeleton pixels (value 1) ;

[0087]

[0088] in, Represents pixels Is it a skeleton pixel? For skeleton pixels, ,otherwise , express of The neighborhood neighboring pixels, where ;

[0089] S2132, if , then Mark as endpoint;

[0090] S2133, if , then Mark as intersection;

[0091] S2134, in Neighborhood according to Two connected skeleton pixels and The coordinates of To skeleton pixels Direction vector and from To skeleton pixels Direction vector ;

[0092]

[0093] Among them, (x, y) is the current skeleton pixel The coordinates of, (i,j) are skeleton pixels The coordinates of (m,n) are the skeleton pixels The coordinates of

[0094] Calculate the direction vector and Angle :

[0095]

[0096] in, , is the dot product of the vectors, For vector The module length, is a vector The module length;

[0097] Set the inflection point threshold angle ,if , then Marked as inflection point.

[0098] Specifically, for step S2110, the segmented binary image is post-processed using a mathematical morphology method. Due to the noise that may be introduced during data transmission and the limitations of the segmentation algorithm itself, there are some small isolated areas or broken areas in the binary result. In order to eliminate these interferences, the image is first eroded. The erosion operation can remove isolated pixels at the boundary of the object, disconnect small connections, and eliminate small objects. Then, an expansion operation is performed. The expansion can fill the holes inside the object, connect adjacent objects, and increase the area of ​​the object. The structural elements of the erosion and expansion can select 3×3 or 5×5 rectangular, circular, and other templates. Morphological processing can not only effectively suppress noise, but also smooth the contour of the object, creating favorable conditions for subsequent skeleton extraction. The fourth image data obtained after processing has a more complete and regular binary area, and noise points and small areas are effectively eliminated, laying a foundation for the accurate extraction of key path points.

[0099] For step S2120, on the basis of morphological processing, the skeleton line of the fourth image data is further extracted. The skeleton is an important tool in shape analysis. It is a refined version of the object, which is composed of the center line of the object and has topological preservation. The skeleton can not only represent the overall shape characteristics of the object, but also reveal the local structural information of the object, such as endpoints, intersections, inflection points, etc. Common skeleton extraction algorithms include a refinement algorithm and an algorithm based on distance transformation. The refinement algorithm continuously erodes the boundary of the object until a skeleton line with a single pixel width is obtained. Its advantage is that it can maintain the connectivity of the skeleton, but its disadvantage is that the calculation amount is large and it is easily disturbed by noise. The distance transformation algorithm extracts the skeleton based on the shortest distance from each point inside the object to the boundary. It first calculates the distance from each foreground pixel to the nearest background pixel, and then thresholds the distance map to obtain the skeleton line. The distance transformation method is fast and insensitive to noise, but may cause discontinuity of the skeleton. According to the image quality and algorithm performance, the refinement method or the distance transformation method is selected to extract the skeleton line to obtain the skeleton line image data. The skeleton line intuitively reflects the main direction and branch structure of the area of ​​interest and is an important basis for planning the motion path.

[0100] For step S2130, after obtaining the skeleton line image data, further search for key points on the skeleton line, including endpoints, intersections and inflection points. The endpoint corresponds to the starting point or end point of the skeleton line and is only connected to one skeleton pixel. The intersection is the intersection of the skeleton lines and is connected to three or more skeleton pixels. The inflection point reflects the sudden change in the direction of the skeleton line and is a point where the curvature changes greatly. These key points characterize the topological structure and direction changes of the area of ​​interest, and are key positions for planning the robot's motion path. In order to search for key points, template matching, connectivity analysis and other methods can be used. For each pixel of the skeleton line image, analyze the distribution of skeleton pixels in its 3×3 neighborhood. If the number of skeleton pixels of the 8 neighboring pixels in the 3×3 neighborhood is , that is, there is only one skeleton pixel in the 3×3 neighborhood connected, then Mark as endpoints, record intersections Coordinates to the intersection list. If , that is, there are three or more skeleton pixels in the 3×3 neighborhood connected, then Mark as intersection, record intersection Coordinates To the intersection list. Set the inflection point threshold angle (For example or ),if , it is considered that the connection direction has changed significantly. Mark as inflection point, record inflection point Coordinates to the inflection point list. After the search is completed, the pixel coordinates of all key path points are extracted and recorded. These coordinate information reflects the key position and spatial distribution of the skeleton line of the region of interest, which is an important input for subsequent path planning. At the same time, key points can also be used to describe the complexity of the region of interest. The more key points there are, the more branches and bends there are in the region, and a more complex path planning strategy may be required.

[0101] In summary, step S2100 uses computer vision technologies such as morphological processing, skeleton extraction, and key point search to achieve the extraction and positioning of key path points from the segmented binary image. Among them, morphological processing effectively eliminates image noise and segmentation errors, ensuring the accuracy of skeleton extraction. The skeleton line not only reflects the overall direction of the area of ​​interest, but also reveals local structural details. The key path points further refine the topological and geometric features of the skeleton line, providing a key basis for robot motion path planning. Through the reasonable connection and smoothing of key points, a safe and efficient motion path can be obtained. This path extraction method makes full use of the structural and spatial information of the image, which is of great significance for guiding the planning decision-making of mobile robots in narrow and tortuous pipes or tunnel environments. At the same time, this method is universal and can be extended to other motion planning tasks in indoor or outdoor environments.

[0102] Step S2200, planning the robot motion path according to the key path points, obtaining a candidate path and first path data of the candidate path;

[0103] Furthermore, if Figure 4 As shown, step S2200 includes:

[0104] Step S2210, searching for the closest key path point pairs in the key path point set, connecting them to form local path segments, and constructing a local path set;

[0105] Step S2220, topologically sorting the local path set to obtain a global path sequence;

[0106] Step S2230, smoothing and optimizing the global path sequence to obtain a candidate path and first path data of the candidate path.

[0107] Specifically, for step S2210, in order to generate the robot motion path, it is necessary to reasonably connect the key path points extracted in step S2130. Since there may be multiple branches and forks in the actual environment, local path segments are first constructed based on the distance criterion. Specifically, the Euclidean distance between each pair of key path points is calculated, and the nearest point pair is searched, and they are connected to form a local path segment. In order to avoid loops and self-intersections, it is necessary to add non-intersecting and non-repeating constraints. Repeat the above process until all key path points are connected to form a local path set. This set reflects the local connectivity of the skeleton line of the region of interest and is the basis for constructing the global path. It is worth noting that if the key points are sparsely distributed, it may be necessary to interpolate and generate auxiliary points to improve the path accuracy and smoothness. At the same time, in order to adapt to the path planning requirements of different environments, other criteria such as path length, energy consumption, safety margin, etc. can be introduced to perform multi-objective optimization on the local path generation process.

[0108] For step S2220, based on the local path set, further splicing is performed to form a global path. Since the local path fragments are generated based on the shortest distance criterion, there may be breaks or discontinuities between them. In order to obtain a complete path from the starting point to the end point, the local paths need to be topologically sorted. Specifically, the path fragment closest to the starting point is used as the starting path, and then the next path fragment closest to its endpoint is searched, and so on, until the path fragment closest to the end point is reached. Topological sorting can be implemented using graph theory algorithms such as depth-first search and minimum spanning tree. After sorting, a global path sequence is obtained, which represents the sequence and connection relationship of the robot's movement. In order to avoid deadlocks and loops, a visited mark can be set to prune repeated paths. At the same time, in order to take into account the coverage and repetition rate of the path, backtracking and branch-and-bound strategies can be introduced to generate multiple candidate paths, and the optimal path is selected according to task requirements.

[0109] For step S2230, since the global path sequence obtained by splicing is composed of straight line segments, there may be sharp changes at the turning point, which is not conducive to the smooth movement of the robot. In order to improve the smoothness and continuity of the path, it is necessary to perform curve fitting on the global path. Common smoothing algorithms include B-spline curves, Bezier curves, moving averages, etc. Among them, the B-spline curve is composed of multiple low-order polynomial curves, has local controllability and shape retention, and can generate smooth and continuous curves. The Bezier curve is a parametric curve determined by the starting point, the end point and the control point. The shape of the curve is affected by the position of the control point and can be adjusted locally. The moving average is a sliding window average of the path point sequence, which can effectively eliminate short-term noise and spike interference. The path obtained after smoothing is the path to be selected. The coordinates of each key point of the path to be selected, the path length, the local path fragment and other information are extracted to form the first path data of the path to be selected. The first path data not only connects the key path points, but also has good geometric characteristics, which is conducive to the motion control and trajectory tracking of the robot. During the smooth optimization process, parameters such as curvature, speed, and acceleration can be constrained according to the robot's kinematic and dynamic models to ensure that the generated path meets the robot's physical limitations. In addition, the dynamic performance and execution efficiency of the path can be further improved through methods such as preview control and trajectory quadratic programming.

[0110] In summary, step S2200 realizes the conversion from discrete point set to continuous path through local connection, global splicing and smooth optimization of key path points. Among them, local path planning utilizes the greedy strategy to preferentially connect the closest point pairs, which reflects the local optimality of the path. Global path planning establishes the topological order between local paths through the topological sorting algorithm, ensuring the connectivity and accessibility of the path. Smooth optimization utilizes curve fitting technology to improve the geometric smoothness and continuity of the path while satisfying the path connectivity constraints. The generated first path data provides an intuitive and accurate reference trajectory for the robot's motion control, which is conducive to improving the stability, accuracy and efficiency of the motion. The computational complexity of this path planning method is low, and it can realize real-time and online path generation, laying the foundation for the robot's autonomous navigation and obstacle avoidance decision-making.

[0111] Step S2300, calculating a comprehensive path complexity index, a comprehensive environmental adaptability index, and a time energy consumption index according to the first path data of the path to be selected, evaluating the path to be selected, and obtaining an evaluation result of the path to be selected, wherein the evaluation result of the path to be selected includes an executable path and a path to be optimized;

[0112] Further, step S2300 includes:

[0113] Step S2310, calculating the comprehensive path complexity index L1 according to the first path data;

[0114] Furthermore, if Figure 5 As shown, step S2310 includes:

[0115] Step S2311, calculating the path length, curvature, and number of turns according to the first path data to obtain path complexity data;

[0116] Step S2312, according to the path length, curvature, and number of turns, respectively calculate corresponding complexity indexes, including length complexity index, curvature complexity index, and turn complexity index;

[0117] Step S2313, performing weighted summation on the length complexity index, the curvature complexity index and the turning complexity index to obtain a comprehensive path complexity index L1.

[0118] The calculation method of the turning complexity index includes:

[0119]

[0120] in:

[0121] : Turn complexity index;

[0122] : Turn complexity weight factor, controls the contribution of the number of turns to the complexity, the recommended range is [0.5, 1.5], and is fitted through experimental data;

[0123] : No. The angle change of the turn is calculated by the angle between two consecutive line segments;

[0124] : Turn count, which refers to the number of times the path direction changes beyond the set threshold;

[0125] : Maximum allowed turning angle, set as the maximum turning angle constraint in the task.

[0126] The turning complexity index takes into account the number and amplitude of turns in the path. Frequent turns will cause the robot to face more directional adjustments in actual movement, increasing the difficulty of movement. and turning angle The increase in the turn complexity index will also increase rapidly, especially when the turning angle approaches the maximum allowed angle The complexity will increase exponentially.

[0127] Specifically, for step S2310, path complexity is an important indicator to measure the difficulty of robot movement, which is directly related to the quality of path planning and decision-making. Generally speaking, the longer the path, the more bends, and the more frequent the turns, the higher the complexity, and the higher the requirements for the robot's motion control and environmental adaptability. Therefore, it is necessary to quantitatively evaluate the complexity from the aspects of path length, curvature change, and number of turns. Among them, the path length can be represented by the cumulative length of each line segment on the path; the curvature change can be estimated by the derivative and second-order derivative of the path at each point; the number of turns can be characterized by the number of times the path direction changes greater than a certain threshold. By calculating these geometric parameters, the shape characteristics of the path can be reflected from the local and overall perspectives. In order to facilitate comprehensive evaluation, the geometric parameters can be further converted into complexity indexes, and the parameters can be mapped to the [0,1] interval using methods such as min-max normalization or Z-score normalization to obtain length complexity index, curvature complexity index, and turn complexity index. The larger the complexity index, the higher the complexity of the corresponding parameter. This quantitative complexity description provides an intuitive and explainable basis for robot path selection.

[0128] Step S2320, calculating the comprehensive environmental adaptability index L2 according to the first path data;

[0129] Furthermore, if Figure 6 As shown, step S2320 includes:

[0130] Step S2321, analyzing the distance between the path and the low visibility boundary in the harsh environment and the distance between the path and the obstacle according to the first path data to obtain environmental adaptability data;

[0131] Step S2322, calculating a boundary fitness index and an obstacle fitness index according to the distance between the path and the low visibility boundary in the harsh environment and the distance between the path and the obstacle;

[0132] Step S2323, performing weighted summation on the boundary fitness index and the obstacle fitness index to obtain a comprehensive environment fitness index L2.

[0133] Specifically, environmental adaptability is another key factor that needs to be considered in robot path planning. Due to the narrowness, tortuosity, and terrain complexity of the low-visibility environment in the harsh environment, the robot faces the risk of collision with the channel boundary and obstacles during the passage. In order to plan a safe and smooth motion path, it is necessary to examine the geometric compatibility of the path and the environment. Generally, the safety margin of the path is judged by calculating the distance between the path and the low-visibility boundary in the harsh environment, and between the path and the obstacle. The distance can be measured by point-to-line, line-to-line, etc., and the minimum value or average value is selected as a representative indicator. The larger the distance, the farther the path is from the boundary and obstacles, the smaller the collision risk, and the better the environmental adaptability. Similar to the complexity evaluation, it is also necessary to design a normalization function to convert the distance index into a fitness index, including the boundary fitness index and the obstacle fitness index. The higher the index, the better the adaptability. Based on the comprehensive evaluation of multiple fitness indices, an overall environmental fitness index can be obtained, which intuitively reflects the suitability of the path to the low-visibility environment in the harsh environment. This quantitative, multi-dimensional environmental adaptability analysis greatly improves the environmental perception capability of path planning, and is conducive to achieving stable and reliable navigation control in complex and dynamic environments.

[0134] Step S2330, calculating the time energy consumption index L3 according to the first path data;

[0135] Furthermore, if Figure 7 As shown, step S2330 includes:

[0136] Step S2331, predicting the path passing time according to the first path data to obtain energy consumption data;

[0137] Step S2332, calculating the time energy consumption index L3 according to the path passing time.

[0138] Specifically, energy consumption is a performance indicator that needs to be optimized for mobile robots. Robots work in small spaces with low visibility in harsh environments, and the energy they can carry is limited. Therefore, they must save energy as much as possible and extend their working time. Path planning affects the energy consumption of robots as a whole. The shorter the path and the smoother the speed, the shorter the passing time and the lower the energy consumption. Therefore, the path passing time is used as the main parameter for energy consumption evaluation. Generally, it is assumed that the robot moves at a constant speed, and the path passing time can be estimated by dividing the path length by the speed. Similar to the first two indicators, the passing time can be further calculated to obtain the time energy consumption index. The lower the energy consumption index, the better the energy saving effect. Quantitative evaluation of the path from the perspective of time and energy consumption makes the path planning more global and strategic. It not only pursues the accessibility and safety of the path in space, but also takes into account the efficiency and economy of the path in time. It is of great value for improving the energy utilization rate and continuous operation capability of mobile robots.

[0139] Step S2340, preset the path complexity threshold Q1, the environmental adaptability threshold Q2 and the time energy consumption threshold Q3, if L1≤Q1, and L2≥Q2, and L3≤Q3, then the evaluation result of the path to be selected is an executable path; otherwise, the evaluation result of the path to be selected is a path to be optimized.

[0140] Specifically, for step S2340, in order to evaluate the feasibility and advantages of the selected path, it is necessary to consider multiple dimensions such as path complexity, environmental adaptability, time and energy consumption. Among them, the comprehensive path complexity index L1 reflects the geometric characteristics of the path, such as path length, curvature, number of turns, etc. The larger the L1 value, the more tortuous and complex the path is, and the greater the difficulty and uncertainty of the robot's movement. Therefore, the path complexity threshold Q1 is set. When L1 exceeds Q1, it is considered that the path is too complex and should not be executed directly. Further simplification and optimization are required. The comprehensive environmental adaptability index L2 measures the degree of fit between the selected path and the environment, such as the distance from obstacles, the width of the passing area, etc. The larger the L2 value, the better the adaptability of the path to the environment, and the more conducive to the safe movement of the robot. Therefore, the environmental adaptability threshold Q2 is set. When L2 is lower than Q2, it is considered that the environmental adaptability of the path is insufficient and local adjustment or replanning is required. The time and energy consumption index L3 comprehensively considers the time overhead and energy consumption of the path. The larger the L3 value, the higher the time and energy cost of the path, and the lower the robot execution efficiency. Therefore, the time and energy consumption threshold Q3 is set. When L3 exceeds Q3, the path is considered to be not economical and efficient, and speed and power optimization is required. Only when L1≤Q1, L2≥Q2 and L3≤Q3 are satisfied at the same time, the candidate path is considered to be an executable path and can be directly applied to robot motion control. Otherwise, the candidate path needs to be further optimized to improve its quality and performance.

[0141] The selection of thresholds Q1, Q2, and Q3 needs to comprehensively consider factors such as robot performance, environmental characteristics, and task requirements. Methods such as empirical settings, data statistics, and parameter searches can be used. Reasonable threshold selection can balance indicators such as path safety, efficiency, and robustness, and improve the effect and reliability of path planning. At the same time, the setting of thresholds also has a certain degree of flexibility and can be adaptively adjusted according to actual needs. For example, in complex environments, the Q1 and Q2 thresholds can be appropriately increased to cope with environmental uncertainties. In pursuit of high efficiency, the Q3 threshold can be relaxed to allow for higher time and energy consumption. Adaptive threshold selection can make the path planning algorithm closer to engineering practice.

[0142] In addition, the feasibility assessment result of the path is an executable path or a path to be optimized, which can further guide subsequent decision-making and processing. For the executable path, it can directly generate control instructions such as speed and acceleration to achieve accurate tracking of the path. For the path to be optimized, it can be further improved to obtain a better candidate path. The third is human-computer interaction. The generated result path is visualized, allowing the operator to check and modify key points, and providing a mechanism for manual intervention and guided optimization. The optimized path can be evaluated for feasibility again until all constraints are met, and finally a high-quality, executable motion path is obtained.

[0143] In summary, step S2340 introduces quantitative indicators such as path complexity, environmental adaptability, time and energy consumption, and constructs a scientific framework for path feasibility evaluation. Based on reasonable threshold selection and flexible optimization strategies, the automatic screening and improvement of candidate paths are realized, providing a complete solution from path generation to path optimization. This method fully considers multiple factors such as robots, environments, and tasks, and improves the adaptability and practicality of path planning. At the same time, this method also provides an explainable and optimizable entry point for path evaluation, which is convenient for engineering implementation and performance improvement. With the advancement of technologies such as intelligent computing and optimization search, threshold selection and path optimization methods will continue to innovate and inject new vitality into robot motion planning. The proposal of multi-criteria path feasibility analysis method has important enlightenment and promotion for subsequent research on environmental perception, multi-objective planning, and autonomous decision-making. Robot path planning technology for complex environments will surely develop further and contribute more wisdom and power to smart cities, smart pipe networks and other fields.

[0144] Step S2400, if the evaluation result of the path to be selected is the path to be optimized, adjusting the local path segments of the path to be optimized to obtain an adjusted path and second path data of the adjusted path;

[0145] Furthermore, if Figure 8 As shown, step S2400 includes:

[0146] Step S2410, searching for local extreme value points in the path to be optimized, wherein the local extreme value points include local highest points, local lowest points and curvature maximum value points;

[0147] Step S2420, taking the local extreme point as the center, performing a local path search in its neighborhood to find a feasible path that meets the constraint conditions; the constraint conditions include that the deviation distance from the original path is less than a distance threshold, the line length increment is less than an increment threshold, and the curvature change is less than a change threshold;

[0148] Step S2430: Replace the corresponding local path segment in the original path with the feasible path that meets the constraint conditions to obtain an adjusted path.

[0149] Specifically, first search for local extreme points on the path to be optimized, including local highest points, local lowest points and curvature maximum points. These points usually correspond to the peaks, valleys and sharp turns of the path, and are key factors affecting the smoothness and safety of the path. The local highest and lowest points can be obtained by analyzing the trend of path elevation changes, and the curvature maximum point can be obtained by calculating the curvature of the path at each point and setting a threshold. After searching for the local extreme point, take it as the center and perform local path exploration in its neighborhood. The so-called neighborhood refers to the area within a certain radius with the extreme point as the center. Search for alternative paths in the neighborhood to find feasible paths that meet the constraints. The constraints here mainly include: the deviation distance between the new path and the original path cannot be too large to avoid causing a significant change in the path; the line length increment of the new path should be as small as possible to avoid increasing the driving distance of the robot; the curvature change of the new path should be as small as possible to improve the smoothness of the path. The deviation distance and length increment can be obtained by calculating the Euclidean distance of the path point coordinates, and the curvature change can be obtained by calculating the curvature change rate of the path at the extreme point. The purpose of setting the distance threshold is to ensure that the optimized path does not deviate too much from the original planned path and maintain the overall consistency. The increment threshold is set to avoid excessive detours in the path and ensure efficiency. It can be considered to be set to a certain percentage of the original path length, such as 5% or 10%, or the maximum acceptable increment can be determined based on task requirements and energy constraints. The change threshold limits the maximum change in path curvature in order to ensure the smoothness of the path and avoid sharp turns. When setting it, the robot's steering ability and minimum turning radius can be considered, or it can be set based on the structural characteristics of low visibility in harsh environments, such as the typical curvature of the curve. Under the premise of meeting the constraints, the alternative path is preferred and replaced with the corresponding local path segment in the original path to obtain a locally adjusted path.

[0150] Through the above local adjustments, the peaks, steep slopes and sharp turns in the path can be effectively eliminated, making the path smoother and safer. At the same time, since the adjustment range is limited to the local neighborhood, it will not cause a significant change in the path, ensuring the overall direction and topological structure of the path. It is worth noting that due to the complexity of the actual environment, multiple iterative adjustments may be required until a satisfactory path is obtained. In addition, in order to further improve the quality of the path, other optimization criteria can be introduced, such as minimizing the path length, maximizing the safety margin, etc., and the local path adjustment problem can be modeled as a multi-objective optimization problem, and solved using heuristic search, dynamic programming and other algorithms.

[0151] Step S2500, calculate the second comprehensive path complexity index L4, the second comprehensive environmental adaptability index L5 and the second time energy consumption index L6 according to the second path data of the adjusted path. If L4≤Q1, and L5≥Q2, and L6≤Q3, the adjusted path is an executable path. Otherwise, adjust the distance threshold, increment threshold and change threshold, and execute step S2400 until an executable path is obtained.

[0152] Specifically, if any of the three conditions L4≤Q1, L5≥Q2, and L6≤Q3 is not met, the path needs to be further adjusted; the adjustment here is achieved by modifying the distance threshold, the increment threshold, and the change threshold. The distance threshold constrains the optimized path from deviating too much from the original planned path, the increment threshold limits the distance between adjacent path points, and the change threshold constrains the curvature change rate of the path point coordinates. By increasing the distance threshold, the safety margin of the path can be increased, but it may cause the path length to increase; by reducing the increment threshold, the accuracy and smoothness of the path can be improved, but the calculation overhead may increase; by adjusting the change threshold, the curvature and turning characteristics of the path can be changed, but the continuity of the path may be affected. Therefore, it is necessary to balance factors such as safety, accuracy, smoothness, and energy consumption to obtain the optimal threshold combination. After adjusting the threshold, re-execute step S2400 to optimize the path until an executable path that meets the conditions is obtained.

[0153] Step S2500 establishes a quantitative evaluation index system, which can comprehensively characterize the complexity, environmental adaptability, time and energy consumption and other characteristics of the path, and provides a clear direction and quantitative standards for path optimization. Secondly, through the threshold comparison and feedback adjustment mechanism, the adaptive control and closed-loop optimization of the path quality are realized, and the optimal path can be automatically searched to reduce manual intervention. Thirdly, the method introduces environmental constraint information, such as obstacle distribution, safety distance, etc., which can generate a feasible path that meets the actual working conditions and has strong environmental adaptability. At the same time, it takes into account the time and energy consumption performance of the robot, and can take into account the efficiency and economy of the path. In addition, the threshold adjustment strategy of this method is flexible and changeable, and the path can be locally fine-tuned or overall reconstructed according to different needs, with good optimization space.

[0154] In summary, the path evaluation and optimization method proposed in step S2500 makes full use of strategies such as multi-index fusion, threshold control, closed-loop iteration, etc., and realizes adaptive optimization of the path while ensuring the feasibility of the path. In particular, through quantitative indicators and threshold constraints, this method can flexibly weigh the complexity, safety, energy consumption, adaptability and other performance of the path, automatically evaluate, screen and fine-tune the path, and greatly improve the efficiency and quality of path planning. Therefore, this method is of great significance for realizing the intelligent behavior decision-making of robots and improving their autonomous operation level, and is expected to be widely used in the fields of intelligent transportation, industrial robots, service robots, etc.

[0155] Example 2

[0156] This embodiment provides a path planning method for a dredging robot in a harsh environment based on sonar images on the basis of Embodiment 1, including:

[0157] Step S1220, dynamically generating a first threshold for image segmentation according to the first mean data and the first variance data;

[0158] The algorithm for generating the first threshold value also includes:

[0159]

[0160] : The first threshold value is used to distinguish the foreground and background pixels in the image, that is, to determine which pixels belong to the target area;

[0161] Mean: The first mean value data, reflecting the average echo intensity of the local area;

[0162] Variance: The first variance data reflects the fluctuation of echo intensity in the local area;

[0163] : The first control factor, which controls the influence weight of the mean in the threshold, the preferred range is: ,Through experimental data adjustment, it is expected that the threshold is larger in the ,high echo intensity region, and the first control factor can be moderately ,increased;

[0164] : The second control factor adjusts the influence of variance on the contribution of mean value in threshold, the preferred range is: , adjusted according to the different texture complexity of the image, and fitted through experimental data;

[0165] : The third control factor controls the weight of the inverse of the variance in the threshold, the preferred range is: Through experiments on different image datasets, smaller Can help adapt to low-contrast areas;

[0166] : Fixed constant used to prevent division by zero errors, usually set to 10 -6 .

[0167] The meaning of the parameters:

[0168] First mean data: The average value of the echo intensity in a local area of ​​the image can reflect the material properties of the area. For example, in the bottom mud deposition area, since sound waves are easily absorbed by the soft mud, the echo intensity is usually weak and the mean is low. For low-visibility walls in harsh environments, since the material is harder, sound waves are more easily reflected, the echo intensity should be stronger and the local mean should be higher.

[0169] First, variance data: The variance reflects the fluctuation of echo intensity in the local area of ​​the sonar image, representing the roughness of the regional surface. If the variance is large, it means that the area may be a rough bottom such as gravel or silt. Areas with small variance are often smooth, low-visibility concrete surfaces in harsh environments.

[0170] When the first mean value data increases: As the echo intensity in the local area increases, the threshold Increases so that more pixels are classified as background pixels. This is to avoid misidentifying high echo intensity areas as foreground.

[0171] When the first variance data increases: As the local area variance increases, the second term in the first threshold decreases, and the overall threshold Reduce it, thereby keeping more areas with complex textures as foreground.

[0172] Three control factors a, b, c:

[0173] Determines the influence strength of the mean. The larger the value, the greater the influence of the mean on the threshold.

[0174] Adjusts the contribution of the mean when there is a higher variance. The larger the variance, the smaller the impact of the mean on the threshold.

[0175] Controls the weight of the inverse of the variance in the formula. A larger variance will reduce the contribution of the inverse term and help increase the threshold in areas with complex textures.

[0176] Experimental data or historical data fitting process:

[0177] To determine , , The optimal range of can be obtained by fitting through the following process:

[0178] Data collection: Collect a large amount of sonar images in low-visibility environments under harsh conditions, especially image data with different material properties and complex textures;

[0179] Experimental analysis: Use different combinations of a, b, and c to calculate the corresponding first threshold and apply it to image segmentation; by observing the segmentation effect, select the parameter combination with the best segmentation result;

[0180] Fitting curve: Use experimental data to fit the optimal range of values ​​for each parameter; for example, through curve fitting, we can get The corresponding minimum segmentation error.

[0181] This formula can dynamically adjust the threshold according to the local echo intensity and bottom roughness of the sonar image, and is suitable for scenes with weak underwater lighting, complex and changeable bottom materials in harsh environments with low visibility. Through the three coefficients a, b, and c, the balance between the echo intensity and the complexity of the bottom texture in the sonar image can be flexibly adjusted, thereby improving the accuracy of sonar image segmentation. This formula is particularly suitable for generating high-quality segmentation results in underwater environments with large differences in echo intensity and large changes in material types, and is of great significance for improving the environmental perception and autonomous operation capabilities of low-visibility dredging robots in harsh environments.

[0182] Example 3

[0183] This embodiment provides a path planning system for a dredging robot in a harsh environment based on sonar images on the basis of Embodiment 1. Fig. 9 As shown, including:

[0184] Image processing module: used to obtain first image data of a low-visibility environment in a harsh environment, and pre-process the first image data to obtain second image data; segment the second image data to obtain third image data;

[0185] A path evaluation module: used for extracting key path points in a low-visibility environment under a harsh environment according to the third image data; planning the robot motion path according to the key path points, and obtaining a path to be selected and first path data of the path to be selected; calculating a comprehensive path complexity index, a comprehensive environmental adaptability index, and a time energy consumption index according to the first path data of the path to be selected, evaluating the path to be selected, and obtaining an evaluation result of the path to be selected, wherein the evaluation result of the path to be selected includes an executable path and a path to be optimized;

[0186] Path optimization module: if the evaluation result of the path to be selected is the path to be optimized, then adjust the local path segment of the path to be optimized, obtain the adjusted path and the second path data of the adjusted path, and evaluate the adjusted path.

[0187] In the image processing module, segmenting the second image data to obtain the third image data includes:

[0188] Step S1210, performing local statistical analysis on the second image data, calculating the mean and variance in each pixel neighborhood, and obtaining first mean data and first variance data;

[0189] Step S1220, dynamically generating a first threshold for image segmentation according to the first mean data and the first variance data; the generation algorithm of the first threshold includes an Otsu threshold method and a maximum entropy threshold method;

[0190] Step S1230, obtaining a preset multi-scale and multi-directional texture feature extraction operator, and combining it with the first threshold, performing segmentation on the second image data to obtain binary image data of the region of interest, marked as third image data; the texture feature extraction operator includes a Gabor filter.

[0191] In the path assessment module, extracting key path points in a low-visibility environment under a harsh environment according to the third image data includes:

[0192] Step S2110, performing morphological processing on the third image data to eliminate noise and interference areas in the binary image to obtain fourth image data; the morphological processing includes an erosion operation and a dilation operation;

[0193] Step S2120, using a skeleton line extraction algorithm to extract skeleton lines of connected domains in the fourth image data to obtain skeleton line image data;

[0194] Step S2130, searching for endpoints, intersections and inflection points in the skeleton line image data, marking them as key path points, obtaining a set of key path points, and extracting coordinate information of each key path point.

[0195] The searching for endpoints, intersections and inflection points in the skeleton line image data comprises:

[0196] Step S2131, traverse each pixel of the skeleton line image data ,right of The 8 neighboring pixels in the neighborhood arrive Perform analysis and count the number of skeleton pixels (value 1) ;

[0197]

[0198] in, Represents pixels Is it a skeleton pixel? For skeleton pixels, ,otherwise , express of The neighborhood neighboring pixels, where ;

[0199] S2132, if , then Mark as endpoint;

[0200] S2133, if , then Mark as intersection;

[0201] S2134, in Neighborhood according to Two connected skeleton pixels and The coordinates of To skeleton pixels Direction vector and from To skeleton pixels Direction vector ;

[0202]

[0203] Among them, (x, y) is the current skeleton pixel The coordinates of, (i,j) are skeleton pixels The coordinates of Skeleton pixels The coordinates of

[0204] Calculate the direction vector and Angle :

[0205]

[0206] in, , is the dot product of the vectors, For vector The module length, is a vector The module length;

[0207] Set the inflection point threshold angle ,if , then Marked as inflection point.

[0208] In the path evaluation module, the robot motion path planning according to the key path points includes:

[0209] Step S2210, searching for the closest key path point pairs in the key path point set, connecting them to form local path segments, and constructing a local path set;

[0210] Step S2220, topologically sorting the local path set to obtain a global path sequence;

[0211] Step S2230, smoothing and optimizing the global path sequence to obtain a candidate path and first path data of the candidate path.

[0212] In the path evaluation module, the comprehensive path complexity index, the comprehensive environmental adaptability index and the time energy consumption index are calculated according to the first path data of the path to be selected, and the path to be selected is evaluated to obtain the evaluation result of the path to be selected, including:

[0213] Step S2310, calculating the comprehensive path complexity index L1 according to the first path data;

[0214] Step S2320, calculating the comprehensive environmental adaptability index L2 according to the first path data;

[0215] Step S2330, calculating the time energy consumption index L3 according to the first path data;

[0216] Step S2340, preset the path complexity threshold Q1, the environmental adaptability threshold Q2 and the time energy consumption threshold Q3, if L1≤Q1, and L2≥Q2, and L3≤Q3, then the evaluation result of the path to be selected is an executable path; otherwise, the evaluation result of the path to be selected is a path to be optimized.

[0217] In the path optimization module, the local path segment adjustment of the path to be optimized includes:

[0218] Step S2410, searching for local extreme value points in the path to be optimized, wherein the local extreme value points include local highest points, local lowest points and curvature maximum value points;

[0219] Step S2420, taking the local extreme point as the center, performing a local path search in its neighborhood to find a feasible path that meets the constraint conditions; the constraint conditions include that the deviation distance from the original path is less than a distance threshold, the line length increment is less than an increment threshold, and the curvature change is less than a change threshold;

[0220] Step S2430: Replace the corresponding local path segment in the original path with the feasible path that meets the constraint conditions to obtain an adjusted path.

[0221] In the path optimization module, the evaluation of the adjusted path includes:

[0222] According to the second path data of the adjusted path, calculate the second comprehensive path complexity index L4, the second comprehensive environmental adaptability index L5 and the second time energy consumption index L6. If L4≤Q1, and L5≥Q2, and L6≤Q3, the adjusted path is an executable path. Otherwise, adjust the distance threshold, increment threshold and change threshold, and execute step S2400 until an executable path is obtained.

[0223] Example 4

[0224] This embodiment discloses an electronic device, which may include one or more processors and one or more memories. The memories store computer-readable codes, which, when executed by the one or more processors, may execute the path planning method for a dredging robot in a harsh environment based on sonar images as described above.

[0225] The method or system according to the embodiment of the present application can also be implemented with the help of the architecture of an electronic device. The electronic device may include a bus, one or more CPUs, a read-only memory (ROM), a random access memory (RAM), a communication port connected to a network, an input / output component, a hard disk, etc. A storage device in the electronic device, such as a ROM or a hard disk, can store the path planning method for a dredging robot in a harsh environment based on sonar images provided in the present application. A path planning method for a dredging robot in a harsh environment based on sonar images may, for example, include: acquiring first image data of a low-visibility environment in a harsh environment, and preprocessing the first image data to obtain second image data; segmenting the second image data to obtain third image data; the first image data is a sonar image; based on the third image data, extracting key path points in a low-visibility environment in a harsh environment; planning the robot's motion path based on the key path points to obtain a path to be selected and first path data of the path to be selected; calculating a comprehensive path complexity index, a comprehensive environmental adaptability index and a time energy consumption index based on the first path data of the path to be selected, evaluating the path to be selected, and obtaining an evaluation result of the path to be selected, wherein the evaluation result of the path to be selected includes an executable path and a path to be optimized; if the evaluation result of the path to be selected is a path to be optimized, performing a local path segment adjustment on the path to be optimized to obtain an adjusted path and second path data of the adjusted path, and evaluating the adjusted path.

[0226] Furthermore, the electronic device may also include a user interface. Of course, the architecture disclosed in the present invention is only exemplary, and when implementing different devices, one or more components in the electronic device disclosed in the present invention may be omitted according to actual needs.

[0227] Example 5

[0228] This embodiment discloses a computer-readable storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the path planning method of a dredging robot in a harsh environment based on sonar images in the embodiment of the present application can be executed. The storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache). Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0229] In addition, according to an embodiment of the present application, the process described above with reference to the flowchart may be implemented as a computer software program. For example, the present application provides a non-temporary machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided in the present application, for example: obtaining first image data of a low-visibility environment in a harsh environment, and preprocessing the first image data to obtain second image data; segmenting the second image data to obtain third image data; the first image data is a sonar image; according to the third image data, extracting key path points in a low-visibility environment in a harsh environment; planning the robot motion path according to the key path points to obtain a to-be-selected path and the first path data of the to-be-selected path; calculating the comprehensive path complexity index, the comprehensive environmental adaptability index and the time energy consumption index according to the first path data of the to-be-selected path, evaluating the to-be-selected path, and obtaining an evaluation result of the to-be-selected path, the evaluation result of the to-be-selected path including an executable path and a to-be-optimized path; if the evaluation result of the to-be-selected path is a to-be-optimized path, adjusting the local path segments of the to-be-optimized path to obtain the adjusted path and the second path data of the adjusted path, and evaluating the adjusted path. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are performed.

[0230] The methods, systems, and devices of the present application may be implemented in many ways. For example, the methods, systems, and devices of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers recording media storing programs for executing the method according to the present application.

[0231] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0232] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A path planning method for a dredging robot in a harsh environment based on sonar images, characterized in that: The method comprises: Acquire first image data of a low-visibility environment in a harsh environment, and pre-process the first image data to obtain second image data; segment the second image data to obtain third image data; the first image data is sonar image data; Extract key path points in a low-visibility environment under a harsh environment according to the third image data; plan the robot motion path according to the key path points to obtain a path to be selected and first path data of the path to be selected; calculate a comprehensive path complexity index, a comprehensive environmental adaptability index and a time energy consumption index according to the first path data of the path to be selected, evaluate the path to be selected, and obtain an evaluation result of the path to be selected, wherein the evaluation result of the path to be selected includes an executable path and a path to be optimized; If the evaluation result of the path to be selected is the path to be optimized, the path to be optimized is adjusted locally to obtain an adjusted path and second path data of the adjusted path, and the adjusted path is evaluated; The segmenting of the second image data comprises: Performing local statistical analysis on the second image data, calculating the mean and variance in each pixel neighborhood, and obtaining first mean data and first variance data; Generate a first threshold for image segmentation according to the first mean data and the first variance data; Obtaining a preset multi-scale and multi-directional texture feature extraction operator, and combining it with the first threshold, performing segmentation on the second image data to obtain binary image data of the region of interest, marked as third image data, wherein the texture feature extraction operator includes a Gabor filter; The extraction of key path points in a low-visibility environment under a harsh environment includes: Performing morphological processing on the third image data to obtain fourth image data; Extracting skeleton lines of connected domains in the fourth image data to obtain skeleton line image data; Search endpoints, intersections and inflection points in the skeleton line image data, mark them as key path points, obtain a set of key path points, and extract coordinate information of each key path point; The step of calculating the comprehensive path complexity index, the comprehensive environment adaptability index and the time energy consumption index according to the first path data of the to-be-selected path includes: Calculate the path length, curvature, and number of turns according to the first path data to obtain path complexity data; calculate corresponding complexity indicators according to the path length, curvature, and number of turns, including length complexity index, curvature complexity index, and turn complexity index; perform weighted summation on the length complexity index, curvature complexity index, and turn complexity index to obtain a comprehensive path complexity index L1; Analyze the distance between the path and the low-visibility boundary in the harsh environment and the distance between the path and the obstacle according to the first path data to obtain environmental adaptability data; calculate the boundary fitness index and the obstacle fitness index according to the distance between the path and the low-visibility boundary in the harsh environment and the distance between the path and the obstacle; perform weighted summation of the boundary fitness index and the obstacle fitness index to obtain the comprehensive environmental adaptability index L2; Predicting the path passing time according to the first path data to obtain energy consumption data; calculating the time energy consumption index L3 according to the path passing time; The local path segment adjustment of the path to be optimized includes: Searching for local extreme value points in the path to be optimized, wherein the local extreme value points include local highest points, local lowest points and curvature maximum points; Taking the local extreme point as the center, a local path search is performed in its neighborhood to find a feasible path that meets the constraint conditions; the constraint conditions include that the deviation distance from the original path is less than the distance threshold, the line length increment is less than the increment threshold, and the curvature change is less than the change threshold; The feasible path that meets the constraints replaces the corresponding local path segment in the original path to obtain the adjusted path.

2. The path planning method for a dredging robot in a harsh environment based on sonar images according to claim 1 is characterized in that: Planning the robot motion path according to the key path points includes: Search for the closest key path point pair in the key path point set, connect them to form a local path segment, and construct a local path set; Topologically sort the local path set to obtain the global path sequence; The global path sequence is smoothly optimized to obtain the candidate path and the first path data of the candidate path.

3. The path planning method for a dredging robot in a harsh environment based on sonar images according to claim 2 is characterized in that: The searching for endpoints, intersections and inflection points in the skeleton line image data comprises: Traverse each pixel of the skeleton line image data ,right of The 8 neighboring pixels in the neighborhood arrive Analyze and count the number of skeleton pixels ; (x,y) is pixel The coordinates of , then Marked as an endpoint; if , then Mark as intersection; exist Neighborhood according to Two connected skeleton pixels and The coordinates of To skeleton pixels Direction vector and from To skeleton pixels Direction vector ; Calculate the direction vector and Angle , set the inflection point threshold angle ,if , then Marked as inflection point.

4. The path planning method for a dredging robot in a harsh environment based on sonar images according to claim 1 is characterized in that: The evaluation of the selected path includes: The path complexity threshold Q1, the environmental adaptability threshold Q2 and the time energy consumption threshold Q3 are preset. If L1≤Q1, L2≥Q2, and L3≤Q3, the evaluation result of the selected path is an executable path; otherwise, the evaluation result of the selected path is a path to be optimized; The evaluation of the adjusted path includes: According to the second path data of the adjusted path, the second comprehensive path complexity index L4, the second comprehensive environmental adaptability index L5 and the second time energy consumption index L6 are calculated. If L4≤Q1, and L5≥Q2, and L6≤Q3, the adjusted path is an executable path. Otherwise, the distance threshold, increment threshold and change threshold are adjusted, and the local path fragments of the adjusted path are adjusted until an executable path is obtained.

5. A path planning system for a dredging robot in a harsh environment based on sonar images, which is used to implement the path planning method for a dredging robot in a harsh environment based on sonar images as described in any one of claims 1 to 4, characterized in that: The system comprises: Image processing module: used to obtain first image data of a low-visibility environment in a harsh environment, and pre-process the first image data to obtain second image data; segment the second image data to obtain third image data; A path evaluation module: used for extracting key path points in a low-visibility environment under a harsh environment according to the third image data; planning the robot motion path according to the key path points, and obtaining a path to be selected and first path data of the path to be selected; calculating a comprehensive path complexity index, a comprehensive environmental adaptability index, and a time energy consumption index according to the first path data of the path to be selected, evaluating the path to be selected, and obtaining an evaluation result of the path to be selected, wherein the evaluation result of the path to be selected includes an executable path and a path to be optimized; Path optimization module: if the evaluation result of the path to be selected is the path to be optimized, then adjust the local path segment of the path to be optimized, obtain the adjusted path and the second path data of the adjusted path, and evaluate the adjusted path.

6. An electronic device comprising a memory, a central processing unit, and a computer program stored in the memory and executable on the central processing unit, characterized in that: When the central processing unit executes the computer program, the path planning method for a dredging robot in a harsh environment based on sonar images described in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the path planning method for a dredging robot in a harsh environment based on sonar images according to any one of claims 1 to 4 is implemented.

Images