Obstacle avoidance marking path generation method based on image semantic segmentation

Abstract

The application relates to the technical field of image semantic segmentation, in particular to a barrier avoidance marking path generation method based on image semantic segmentation, which comprises the following steps: inputting collected images to be processed into a pre-trained semantic segmentation neural network to output a semantic category probability matrix; and extracting pixel point clusters belonging to a background category to construct a background surface domain according to the semantic category probability matrix. The application generates a safe and drivable polygon surface domain from the difference set of an initial drivable corridor surface domain and a closed area of weak feature ground obstacles, so that areas such as light-colored protrusions, ground damage and low-contrast occlusions which are not easy to be directly classified into conventional obstacles participate in driving constraints; the boundary distance of the safe and drivable polygon surface domain is used to screen an equal-distance topological skeleton line set, and then the coordinate sequence is converted into marking control points, so that the path result can keep balanced spacing with the boundary, and the situations such as edge marking, local detouring through a narrow area and control point jumping are reduced.

Description

Technical Field

[0001] This invention relates to the field of image semantic segmentation technology, and in particular to a method for generating obstacle avoidance line marking paths based on image semantic segmentation. Background Technology

[0002] The obstacle avoidance path generation technology involves environmental perception, spatial geometry analysis, access area determination, trajectory planning, path control point generation, and automated operation control. It focuses on solving the problem of continuous path generation in complex scenarios with fixed obstacles, temporary obstacles, boundary occlusions, and obstacles with weak ground features.

[0003] Existing obstacle avoidance path generation in practice typically revolves around environmental perception, spatial geometry analysis, accessibility zone determination, trajectory planning, and path control point generation. However, the spatial relationship between accessibility zones and obstacle zones often remains at the level of local geometric boundaries or single obstacle outlines. When fixed obstacles, temporary obstacles, boundary occlusions, and obstacles with weak ground features coexist, the judgment of regional connectivity is easily affected by occlusion boundaries, low-contrast ground textures, and local noise points, leading to passable areas being misjudged as broken areas, or obstructed areas being misjudged as passable areas. In actual marking scenarios, parking lot pillars, construction cones, light-colored ground protrusions, water stains, and remnants of old markings may all appear in the work area simultaneously. If path generation is based solely on conventional obstacle outlines or channel geometric widths, the marked path may approach the obstacle edge, forming sharp turning control points, or even passing through risky areas with weak ground features. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing an obstacle avoidance path generation method based on image semantic segmentation.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: an obstacle avoidance path generation method based on image semantic segmentation, comprising the following steps:

[0006] The acquired image to be processed is input into a pre-trained semantic segmentation neural network, which outputs a semantic category probability matrix. Based on the semantic category probability matrix, pixel clusters belonging to the background category are extracted to construct a background surface region, and pixel clusters belonging to the regular obstacle category are extracted to construct a regular obstacle surface region.

[0007] A gridded connected topology graph is constructed for the background surface region, and the cost value parameter is obtained by combining the distribution pattern of the conventional obstacle surface region; a pathfinding algorithm is executed in the connected topology graph according to the cost value parameter to obtain the lowest cost discrete polyline; the lowest cost discrete polyline is smoothed to generate a basic obstacle avoidance smooth trajectory.

[0008] The basic obstacle avoidance smooth trajectory is extended outward by a set distance to obtain the closed area of ​​weak feature obstacles on the ground; spatial expansion operation is performed on the basic obstacle avoidance smooth trajectory to generate an initial passageway area; the difference between the initial passageway area and the closed area of ​​weak feature obstacles on the ground is calculated to generate a safe and passable polygon area.

[0009] Extract the discrete coordinate points of the boundary of the safe and passable polygonal region, calculate the distance from each point inside the safe and passable polygonal region to the boundary, and filter to obtain an equidistant topological skeleton line set; convert the coordinate sequence of the equidistant topological skeleton line set into line control point positions, and generate obstacle avoidance line drawing paths.

[0010] Preferably, the steps for obtaining the conventional obstacle surface region are as follows:

[0011] The acquired images to be processed are input into a pre-trained semantic segmentation neural network in pixel coordinate order. The color channel values, edge texture values, and scale level response values ​​of the images to be processed are called. The scale level response values ​​are upsampled, aligned, and the category probability is normalized to obtain the semantic category probability matrix.

[0012] Based on the semantic category probability matrix, the probability column corresponding to the background category is read pixel by pixel, the relationship between the background category probability and the probability of other categories is compared, the pixel coordinates with the highest background category probability are marked as background pixels, and the background pixels are merged according to the continuity of adjacent coordinates to form a background region.

[0013] Based on the semantic category probability matrix, the probability columns corresponding to the regular obstacle categories are read pixel by pixel. The numerical order of the regular obstacle category probability, the background category probability, and the other category probabilities is compared. The pixel coordinates with the highest regular obstacle category probability are marked as regular obstacle pixels. Regular obstacle pixels are merged according to the continuity of adjacent coordinates to form a regular obstacle surface region.

[0014] Preferably, the step of obtaining the minimum cost discrete piecewise linear curve is as follows:

[0015] The background surface is divided into continuous grid nodes according to a preset grid side length. The center coordinates, adjacent coordinates and boundary distance of each continuous grid node are read. Combined with the overlapping area, adjacent distance and occlusion direction of the conventional obstacle surface, the grid nodes in the connected topology graph are marked with cost parameters to generate a connected topology graph marked with cost parameters.

[0016] Based on the connected topology graph of the marked cost parameters, the grid nodes where the starting point and the ending point are located are read, a pathfinding algorithm is executed, the cumulative cost parameters of adjacent grid nodes are compared layer by layer, grid nodes that enter the conventional obstacle surface region are removed, and the lowest cost discrete polyline between the starting point and the ending point is extracted.

[0017] Preferably, the step of obtaining the basic obstacle avoidance smooth trajectory is as follows:

[0018] Based on the minimum cost discrete polyline, the coordinate difference, turning angle, and node spacing of adjacent polyline nodes are read. Laplace smoothing calculation is performed on the minimum cost discrete polyline. Piecewise polynomial curvature parameters are extracted from the smoothed nodes. Curvature is continuously corrected for polyline nodes whose turning angle exceeds a preset sharp corner threshold to generate a basic obstacle avoidance smooth trajectory.

[0019] Preferably, the step of obtaining the closed region of the weak feature obstacle on the ground is as follows:

[0020] The basic obstacle avoidance smooth trajectory is extended outward by a set distance along the trajectory normal direction. The node coordinates, tangential direction and normal direction of the basic obstacle avoidance smooth trajectory are read point by point. The extension points on the same side are connected in node order, and the extension points at both ends are closed and connected to construct the initial outline of the zero level set.

[0021] Based on the initial contour of the zero level set, the pixel state of the image surrounding the initial contour of the zero level set is read, and the gray value, texture response value and gradient magnitude are extracted pixel by pixel. The global statistical mean parameters of the region inside and outside the contour are calculated respectively. The deviation values ​​inside and outside the evolution curve are calculated and the region energy term is constructed. The level set function is solved iteratively and partial differential dimension reduction is performed. When the change of the region energy term in continuous iteration is less than the preset convergence threshold, the zero level set coordinates are extracted to obtain the closed region of weak feature obstacles on the ground.

[0022] Preferably, the step of obtaining the safe and passable polygonal region is as follows:

[0023] Based on the basic obstacle avoidance smooth trajectory and the closed region of weak feature obstacles on the ground, a spatial expansion operation is performed on the basic obstacle avoidance smooth trajectory to generate an initial passageway region. The difference between the initial passageway region and the closed region of weak feature obstacles on the ground is calculated. The starting coordinates and ending coordinates in the difference result are checked to see if they belong to the same connected region. If the difference result maintains the connection between the starting and ending points, the largest connected region is extracted. If the connection is broken, the radius of the spatial expansion operation is expanded and the difference is recalculated until the difference result maintains the connection between the starting and ending points, generating a topologically continuous safe and passable polygon region.

[0024] Preferably, the step of obtaining the equidistant topological skeleton line set is as follows:

[0025] Extract the boundary discrete coordinate points of the safe and passable polygonal region, record the boundary turning positions and interpolation positions in coordinate order along the boundary contour of the safe and passable polygonal region to form a sequence of boundary discrete coordinate points, read the coordinates of internal points point by point inside the safe and passable polygonal region, calculate the shortest distance from each internal point coordinate to all boundary discrete coordinate points, compare the shortest distance values ​​corresponding to each internal point coordinate, and filter the internal point coordinates with equal shortest distance values ​​and the shortest distance values ​​reaching the maximum value to form the set of coordinates of the center of the largest inscribed circle.

[0026] Read the spatial position, adjacency relationship and boundary equidistant relationship of each center coordinate in the set of coordinates of the largest inscribed circle. Based on the consistency of the corresponding distance from each center coordinate to the discrete coordinate point of the boundary, connect the center coordinates that satisfy the continuous extension relationship segment by segment. Remove the connecting segments that fall outside the boundary and the connecting segments that cross and fold back. Perform topological skeleton extraction operation on the safe and passable polygonal region to obtain the equidistant topological skeleton line set.

[0027] Preferably, the step of obtaining the obstacle avoidance marked path is as follows:

[0028] Read the coordinate sequence, branch connection position, starting position and ending position of each skeleton line in the equidistant topological skeleton line set. Arrange the coordinate sequence of each skeleton line in the extension order from the starting position to the ending position. Convert each coordinate point in the coordinate sequence into a line drawing control point. Verify the connection continuity and transition order of adjacent line drawing control points. Connect the line drawing control points that meet the continuous line drawing requirements in sequence to generate an obstacle avoidance line drawing path.

[0029] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0030] This invention transforms the image to be processed into a semantic category probability matrix, and then forms background and regular obstacle regions by pixel clusters of background category and regular obstacle category, respectively. This allows the passable and obstructed areas to be directly expressed at the spatial level from the pixel level, reducing the risk of region fragmentation caused by relying solely on edge contours or manual annotation. By constructing a gridded connected topology graph on the background region and combining it with the distribution pattern of the regular obstacle region to obtain cost parameters, obstacle position, obstacle occupancy range, and obstacle proximity relationships jointly participate in path selection. The lowest-cost discrete polyline can avoid areas with concentrated regular obstacles, improving the accessibility and stability of the path between the start and end points. By smoothing the lowest-cost discrete polyline... The basic obstacle avoidance smooth trajectory can reduce sharp turns in the polyline path and improve the continuity of the marking equipment. By extending the basic obstacle avoidance smooth trajectory outward by a set distance, the closed area of ​​weak feature obstacles on the ground is obtained. The difference between the initial passage corridor area and the closed area of ​​weak feature obstacles on the ground is used to generate a safe and passable polygon area. This allows areas that are not easily classified as conventional obstacles, such as light-colored protrusions, ground damage, and low-contrast occlusion, to participate in the passage constraints. The boundary distance of the safe and passable polygon area is used to filter the equidistant topological skeleton line set, and then the coordinate sequence is converted into the marking control point position. The path result can maintain a balanced interval with the boundary, reduce the situation of marking along the edge, excessively narrow local detours, and control point jumps, and improve the safety of obstacle avoidance marking path. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the steps of the present invention;

[0032] Figure 2 This is a graph showing the damping force-velocity characteristic.

[0033] Figure 3 This is a temperature rise curve of the motor windings;

[0034] Figure 4 This is a curve showing the curvature change of the smooth trajectory for obstacle avoidance. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0036] Please see Figure 1-4 This invention provides a technical solution: a method for generating obstacle avoidance line marking paths based on image semantic segmentation, comprising the following steps:

[0037] The acquired image to be processed is input into a pre-trained semantic segmentation neural network, which outputs a semantic category probability matrix. Based on the semantic category probability matrix, pixel clusters belonging to the background category are extracted to construct a background surface region, and pixel clusters belonging to the regular obstacle category are extracted to construct a regular obstacle surface region.

[0038] A gridded connected topology graph is constructed for the background surface region, and the cost value parameter is obtained by combining the distribution pattern of the conventional obstacle surface region; a pathfinding algorithm is executed in the connected topology graph according to the cost value parameter to obtain the minimum cost discrete polyline; the minimum cost discrete polyline is smoothed to generate a basic obstacle avoidance smooth trajectory.

[0039] Extend the basic obstacle avoidance smooth trajectory outward by a set distance to obtain the closed area of ​​weak feature obstacles on the ground; perform spatial expansion calculation on the basic obstacle avoidance smooth trajectory to generate the initial passage corridor area; calculate the difference between the initial passage corridor area and the closed area of ​​weak feature obstacles on the ground to generate a safe and passable polygon area.

[0040] Extract the discrete coordinate points of the boundary of the safe and passable polygonal region, calculate the distance from each point inside the safe and passable polygonal region to the boundary, and filter to obtain the equidistant topological skeleton line set; convert the coordinate sequence of the equidistant topological skeleton line set into the line drawing control point position, and generate the obstacle avoidance line drawing path.

[0041] The steps for obtaining a standard obstacle surface region are as follows:

[0042] The acquired images to be processed are input into a pre-trained semantic segmentation neural network in pixel coordinate order. The color channel values, edge texture values, and scale level response values ​​of the images to be processed are called. The scale level response values ​​are upsampled, aligned, and the category probability is normalized to obtain the semantic category probability matrix.

[0043] Based on the semantic category probability matrix, the probability column corresponding to the background category is read pixel by pixel. The probability of the background category is compared with the probability of other categories. The pixel coordinates with the highest background category probability are marked as background pixels. Background pixels are merged according to the continuity of adjacent coordinates to form a background region.

[0044] Based on the semantic category probability matrix, the probability columns corresponding to the regular obstacle categories are read pixel by pixel. The probability values ​​of the regular obstacle category, the background category, and other categories are compared and sorted. The pixel coordinates with the highest regular obstacle category probability are marked as regular obstacle pixels. Regular obstacle pixels are merged according to the continuity of adjacent coordinates to form a regular obstacle surface region.

[0045] Specifically, an RGB three-channel image of size 1920×1080 pixels is input into a pre-trained semantic segmentation neural network according to the pixel coordinates from left to right and from top to bottom. This network adopts the DeepLabV3+ architecture, and its encoder uses ResNet-101 as the backbone network. It extracts the color channel values, edge texture values, and multi-scale hierarchical response values ​​of the image through cascaded residual modules and dilated convolutions with different dilation rates. The dilated spatial pyramid pooling module (ASPP) at the end of the encoder captures multi-scale contextual information through parallel dilated convolutions. The decoder then processes the encoded data. The low-level feature map output by the generator is reduced in dimensionality by a 1×1 convolution, then concatenated with the ASPP feature map that has undergone bilinear interpolation and upsampling by a factor of 4. A 3×3 convolution is then used for feature fusion, followed by another bilinear interpolation upsampling to restore the original image resolution. The network is trained on the Cityscapes dataset, using the cross-entropy loss function and the Adam optimizer for parameter updates. The initial learning rate is set to 0.001, the batch size is 8, and the network is trained for 100 epochs. During the inference phase, the final Softmax layer normalizes the probability of the class prediction vector for each pixel. The calculation method is as follows: ,in, Is the pixel a category? The probability, It is a network to category The original logical value output. It is the total number of predefined categories, such as 30 categories, including roads, sidewalks, buildings, vehicles, pedestrians, etc. The final output is a semantic category probability matrix with dimensions of 1920×1080×30.

[0046] Based on the semantic category probability matrix generated in the previous step, a pixel-by-pixel classification and merging process is initiated. This process first traverses the coordinates of each pixel in the matrix. ,in The range is from 0 to 1919. The range is 0 to 1079. For each coordinate point, read its corresponding category probability vector of length 30, and locate the probability value corresponding to the background category. For example, the index of the background category in the category definition is 0, which means the probability is extracted. Then, the probability of this background category is... The probability values ​​of this pixel are compared with the probability values ​​of the other 29 categories. The maximum value in the probability vector is found by calculation. If the category index corresponding to the maximum value is exactly 0, then this pixel is... In a newly created binary mask image, the value at the corresponding position is set to 1, indicating that it is a background pixel, and otherwise it is set to 0. After traversing and marking all pixels, a two-pass connected component labeling algorithm based on 8-neighborhood connectivity is executed on the generated binary mask image. In the first pass of the algorithm, the image is traversed from top to bottom and from left to right. For each pixel with a value of 1, a temporary label is assigned according to the labels of its upper and left neighbors, and the equivalence relationship between the labels is recorded. In the second pass, the image is traversed again, and each temporary label is replaced with the smallest label in its equivalence class. Thus, all spatially adjacent background pixels are merged into the same connected region and assigned a unique integer label. Finally, the set of all these connected regions with unique labels together constitutes the background region.

[0047] Based on the semantic category probability matrix, all predefined common obstacle categories are identified and regions are constructed. First, it is clarified that common obstacle categories include "vehicles," "pedestrians," "bicycles," "traffic cones," and "pillars," with their indices in the category definition set to 15, 16, 17, 18, and 19, respectively. The processing flow also involves each pixel coordinate... Perform the following steps: read the class probability vector and find the index of the class with the highest probability value. Then, determine the index. Does it exist in the predefined set of regular obstacle category indices {15, 16, 17, 18, 19}? Additionally, to filter out low-confidence fuzzy predictions, a probability threshold is set. Only when the maximum probability value itself is also greater than Only then is the pixel initially identified as a regular obstacle pixel; this threshold... It is determined based on the model's performance on the validation set. For example, by plotting the precision-recall curve, the probability value corresponding to the peak point of the F1 score is selected and set to 0.8. Pixels that meet these two conditions are marked as 1 in a new binary mask image, and the rest are marked as 0. After all pixels have been processed, the binary mask image is also processed using the connected component labeling algorithm based on 8 neighborhoods. All spatially connected regular obstacle pixels are merged to form one or more independent connected regions. Each region is assigned a unique identifier. The overall set of these connected regions constitutes the final regular obstacle surface region.

[0048] The steps to obtain the minimum cost discrete piecewise linear curve are as follows:

[0049] The background surface is divided into continuous grid nodes according to the preset grid side length. The center coordinates, adjacent coordinates and boundary distance of each continuous grid node are read. Combined with the overlapping area, adjacent distance and occlusion direction of the conventional obstacle surface, the grid nodes in the connected topology graph are marked with cost parameters, and a connected topology graph marked with cost parameters is generated.

[0050] Based on the connected topology graph with labeled cost parameters, the grid nodes of the starting point and the ending point are read, the pathfinding algorithm is executed, the cumulative cost parameters of adjacent grid nodes are compared layer by layer, grid nodes that enter the regular obstacle area are removed, and the lowest cost discrete polyline between the starting point and the ending point is extracted.

[0051] Specifically, the background region is meshed. First, a preset mesh edge length is set, for example, 20 pixels. A virtual mesh is constructed within the two-dimensional coordinate range of the background region using this edge length. All mesh cells are traversed. If the center coordinates of a mesh cell are located inside the background region, the mesh cell is defined as a continuous mesh node, and its center coordinates and the coordinates of its eight adjacent nodes sharing an edge or vertex are recorded. Simultaneously, the boundary distance is obtained by calculating the shortest Euclidean distance from the node's center to the background region boundary. Then, for each continuous mesh node, a comprehensive cost parameter is calculated based on the distribution pattern of typical obstacle regions. This parameter is obtained by a weighted sum of multiple components, and its calculation formula is as follows: ,in, It is the ultimate generation value of the node. It is the cost of nearby obstacles, calculated by taking the distance from the node center to the boundary of the nearest regular obstacle region. Obtain, specifically ,in It is an attenuation coefficient, for example, 0.05. It is the cost of the adjacent background boundary, and the calculation method is the same as... Similarly, paths that are close to the edge of passable areas are penalized. This is the occlusion cost, calculated by analyzing the positional relationship between nodes and obstacles relative to a preset travel direction (e.g., from bottom to top of the image). The cost is higher when an obstacle is in front of the node's travel direction than when it is behind it. (The weighting coefficient is...) , and Based on experience, values ​​are set, for example, to 10.0, 2.0, and 3.0 respectively, to reflect the high priority of obstacle avoidance. For any mesh node whose area overlaps with a regular obstacle's surface, its cost is... It is directly set to a maximum value, such as 99999. After traversing all nodes and completing the cost value calculation, a connected topology graph with labeled cost value parameters is generated.

[0052] Based on the connected topology graph with labeled cost parameters, the A* pathfinding algorithm is executed. First, based on the start and end positions of the line-drawing task, the grid node containing the start point and the grid node containing the end point are determined in the gridded connected topology graph. The A* algorithm then evaluates the function... To find the optimal path, where for any grid node , From the starting point to the node The actual path cost is obtained by summing the cost parameters of all nodes on the path. From node The estimated cost to the destination is calculated using the Manhattan distance, i.e. ,in It is a node coordinates These are the coordinates of the endpoint node. The algorithm maintains an open list to store nodes to be evaluated and a closed list to store nodes that have already been evaluated. Starting from the starting point, nodes are added to the open list. In each iteration, nodes are selected from the open list. The node with the smallest value is selected as the current node and moved to the closed list. Then, all its adjacent nodes are checked. For each adjacent node, it is first determined whether its cost parameter is a maximum value (99999). If so, it means that the node is located within the normal obstacle region and is removed from consideration. Otherwise, a new path is calculated from the starting point through the current node to the adjacent node. Value, if this new If the value is smaller, or the adjacent node is not in the open list, then update it. Value and The value is set and its parent node is set as the current node. Then it is added to the open list. This process is repeated until the endpoint node is selected as the current node. At this point, by backtracking the parent node chain from the endpoint, a series of grid nodes connecting the endpoint are finally extracted to form the lowest cost discrete polyline.

[0053] The steps to obtain the basic obstacle avoidance smooth trajectory are as follows:

[0054] Based on the minimum cost discrete polyline, the coordinate difference, turning angle, and node spacing of adjacent polyline nodes are read. Laplace smoothing calculation is performed on the minimum cost discrete polyline. Piecewise polynomial curvature parameters are extracted from the smoothed nodes. Curvature is continuously corrected for polyline nodes whose turning angle exceeds the preset sharp corner threshold, and a basic obstacle avoidance smooth trajectory is generated.

[0055] Specifically, based on the minimum-cost discrete piecewise linear curve, Laplace smoothing is first performed on it. This calculation is achieved through an iterative process. In each iteration, every node of the piecewise linear curve, except for the start and end points, is smoothed. new coordinates Based on its preceding and following adjacent nodes and The position is adjusted using the following formula: Among them, smoothing factor The value of is between 0 and 1, and is set to 0.5 here. The number of iterations is determined based on the initial tortuosity of the path, for example, 15 iterations. After smoothing, the new node sequence is traversed, and for each non-endpoint node... Calculate the vector sum vector If the resulting turning angle is less than a preset sharp corner threshold, such as 90 degrees, then the node... Sharp corners are marked, with the 90-degree threshold being an empirical value set based on the maximum steering capability of the vehicle or mobile device. For each marked sharp corner, continuous curvature correction is performed. Specifically, two nodes before and two nodes after the corner are selected, for a total of five nodes. These five nodes are used as control points, and a cubic B-spline curve interpolation algorithm is used to regenerate the path of the local segment. B-spline curves have local correction and... Continuity allows for the generation of smooth curves with continuously changing curvature, replacing the original sharp corners. After correcting all sharp corners with B-spline curves, the smooth paths are seamlessly connected to generate the final basic obstacle avoidance smooth trajectory.

[0056] The steps for obtaining the closed region of weak feature obstacles on the ground are as follows:

[0057] Extend the basic obstacle avoidance smooth trajectory outward by a set distance along the trajectory normal direction, read the node coordinates, tangential direction and normal direction of the basic obstacle avoidance smooth trajectory point by point, connect the extended points on the same side in the node order, and connect the extended points at both ends to construct the initial outline of the zero level set.

[0058] Based on the initial contour of the zero level set, the pixel state of the image surrounding the initial contour of the zero level set is read, and the gray value, texture response value and gradient magnitude are extracted pixel by pixel. The global statistical mean parameters of the region inside and outside the contour are calculated respectively. The deviation values ​​inside and outside the evolution curve are calculated and the region energy term is constructed. The level set function is solved iteratively and partial differential dimension reduction is performed. When the change of the region energy term in continuous iteration is less than the preset convergence threshold, the zero level set coordinates are extracted to obtain the closed region of weak feature obstacles on the ground.

[0059] Specifically, the basic obstacle avoidance smoothing trajectory is extended outward by a predetermined distance along the trajectory normal direction. This distance is determined based on the physical width of the moving entity (e.g., a small sweeper). If the vehicle width is 0.8 meters and the image resolution is 40 pixels per meter, then the predetermined distance is... Pixels, reading the discrete node coordinates of the basic obstacle avoidance smooth trajectory point by point. For each non-endpoint node Through its preceding and following adjacent nodes and Calculate the tangential direction vector Specifically, it means converting vectors Perform normalization to obtain the unit tangential vector. Normal direction vector Then by The result is obtained by rotating 90 degrees counterclockwise. Then, extension points are generated along the normal directions on both sides of the trajectory. The extension point on the left is... The right extension point is After traversing all nodes, two sequences of extended points are obtained, namely the left boundary point sequence. and the right boundary point sequence Connect the left extension points sequentially according to node order to form the left outline. Similarly, connect the right extension points sequentially to form the right outline. Finally, connect the extension point pairs on the starting side by adding two straight line segments. and the extension point on the end side The two open contour lines are closed to form a closed polygon, which is the initial contour of the zero level set used for subsequent processing.

[0060] Based on the initial contour of the zero-level set, a region-based active contour model (specifically the Chan-Vese model) is applied to segment weakly featured obstacles on the ground. First, the initial contour is used as a zero-level set function. In its initial state, the function is initialized as a signed distance function, with negative values ​​for points inside the contour and positive values ​​for points outside. Then, within a 50-pixel-wide strip region surrounding the initial contour, the feature vector of each pixel is extracted. This vector consists of three components: the grayscale value after converting the RGB image, the maximum texture response value obtained by passing through a set of Gabor filters in four directions (0, 45, 90, 135 degrees), and the gradient magnitude calculated using a 3x3 Sobor operator. Finally, based on the current level set function... The sign of the value is used to calculate the average of the feature vectors of all pixels in the region inside and outside the contour, respectively, resulting in two mean vectors. and Then, the level set function is iteratively evolved by minimizing the energy functional, and its evolution equation is: ,in, It is the feature vector of the pixel. It's a Dirac function. It is a divergence operator. , , These are weighting coefficients, empirically set to 0.2, 1.0, and 1.0 respectively. This partial differential equation is solved using the finite difference method at time steps... The solution is obtained through iterative steps, and the calculation is recalculated after each iteration. and When the absolute value of the change in the region energy term (i.e., the integral of the last two terms in the equation over the entire image domain) calculated in two consecutive iterations is less than the preset convergence threshold of 0.01, the iteration stops. This threshold is an empirical value obtained from a large number of experiments that can balance computational efficiency and segmentation accuracy. Finally, the pixel coordinates corresponding to the contour lines with zero values ​​in the final converged level set function are extracted. The closed curve formed by these coordinates is the closed region of the weak feature obstacle on the ground.

[0061] The steps to obtain a safe and passable polygon region are as follows:

[0062] Based on the basic obstacle avoidance smooth trajectory and the closed region of weak feature obstacles on the ground, a spatial expansion operation is performed on the basic obstacle avoidance smooth trajectory to generate an initial passageway region. The difference between the initial passageway region and the closed region of weak feature obstacles on the ground is calculated. The starting coordinates and ending coordinates in the difference result are checked to see if they belong to the same connected region. If the difference result maintains the connection between the starting and ending points, the largest connected region is extracted. If the connection is broken, the radius of the spatial expansion operation is expanded and the difference is recalculated until the difference result maintains the connection between the starting and ending points, generating a topologically continuous safe and passable polygonal region.

[0063] Specifically, based on the basic obstacle avoidance smooth trajectory and the closed region of weak-feature obstacles on the ground, a spatial dilation operation is first performed on the basic obstacle avoidance smooth trajectory. A circular structuring element with a radius of 30 pixels is used, and this radius value is set based on the vehicle width and safety redundancy, for example, corresponding to a vehicle half-width of 20 pixels plus a safety distance of 10 pixels. The dilation operation replaces each point on the trajectory line with a circular region centered on that point. The union of all circular regions constitutes the initial passageway region. Next, the closed region of weak-feature obstacles on the ground obtained in the previous step is subtracted from this initial passageway region through a pixel-by-pixel Boolean difference operation, resulting in a result region that may contain multiple disconnected parts. Subsequently, connectivity detection is performed on this result region to extract the start and end coordinates of the basic obstacle avoidance smooth trajectory, and a two-pass scan connected component labeling algorithm is used to define the result region. Each independent connected region in the algorithm is assigned a unique label. The labels of the pixels containing the start and end coordinates are checked to see if they are the same and not background labels (label 0). If the labels are the same, it indicates that there is a path between the start and end points located within a safe area. In this case, the connected region containing the start and end points is extracted as the final result. If the connected region is not the largest, the largest connected region containing the start and end points is selected. If the start and end point labels are different, it indicates that the path is blocked by a newly discovered weak feature obstacle. In this case, the radius of the spatial dilation operation is increased by a fixed step, such as 5 pixels. Then, the spatial dilation and difference calculation are performed again using the new radius (35 pixels), and the connectivity detection is performed again. This process is repeated until the start and end points in the difference result are restored to a connected state. The largest connected region containing the start and end points is then determined as a topologically continuous safe and passable polygon region.

[0064] The steps for obtaining the equidistant topological skeleton wire set are as follows:

[0065] Extract the discrete coordinate points of the boundary of the safe and passable polygonal region. Record the turning positions and interpolation positions of each boundary in coordinate order along the boundary contour of the safe and passable polygonal region to form a sequence of discrete coordinate points of the boundary. Read the coordinates of the internal points point by point inside the safe and passable polygonal region. Calculate the shortest distance from each internal point coordinate to all the discrete coordinate points of the boundary. Compare the shortest distance values ​​corresponding to each internal point coordinate. Filter the internal point coordinates with equal shortest distance values ​​and the shortest distance values ​​reaching the maximum value to form the set of coordinates of the center of the largest inscribed circle.

[0066] Read the spatial position, adjacency relationship and boundary equidistant relationship of each circle center coordinate in the set of coordinates of the largest inscribed circle center. Based on the consistency of the corresponding distance from each circle center coordinate to the discrete coordinate point of the boundary, connect the circle center coordinates that satisfy the continuous extension relationship segment by segment. Remove the connecting segments that fall outside the boundary and the connecting segments that cross and fold back. Perform topological skeleton extraction operation on the safe and passable polygonal region to obtain the equidistant topological skeleton line set.

[0067] Specifically, the discrete coordinate points of the boundary of the safe and passable polygonal region are extracted. First, the Suzuki-Abe algorithm is used to traverse the binarized region image, tracking and recording the contour pixel coordinates of its outer boundary and all inner hole boundaries to obtain a series of initial boundary turning positions. To increase the accuracy of the boundary representation, linear interpolation is performed along the straight line segment between every two adjacent turning positions, with an interpolation step size set to 1 pixel, thereby generating dense boundary interpolation positions. All turning positions and interpolation positions are arranged in a clockwise or counterclockwise order along the contour, forming a sequence of discrete coordinate points of the boundary containing thousands of points. Next, a distance transformation map with the same size as the region is created. A fast distance transformation algorithm based on chamfer distance is used to calculate the Euclidean distance from each pixel inside the safe and passable polygonal region to all points in the sequence of discrete coordinate points of the boundary, and the shortest distance value is used as the value of that pixel. After completing the distance transformation, the entire distance transformation map is traversed to find the global maximum value. Then, filter out all distance values ​​equal to The pixels are the centers of the largest inscribed circle. Due to the complexity of the shape, there may be multiple largest inscribed circles. Therefore, the set of these points constitutes the set of coordinates of the centers of the largest inscribed circle.

[0068] The system reads the coordinates of each circle's center from the set of coordinates of the largest inscribed circle and, combined with the distance transformation graph generated in the previous step, performs a distance transformation-based topological skeleton extraction operation. This method identifies skeleton points by finding local maxima in the distance transformation graph. An interior point is considered a local maximum if its distance value is greater than or equal to the distance values ​​of all its eight neighbors. All these local maxima constitute the initial set of the skeleton. Subsequently, to connect these scattered skeleton points and form continuous skeleton lines, a path is traced from each local maximum along the gradient ascent direction of the distance transformation graph until another local maximum or a region boundary is reached. Points on the traced path are also added to the skeleton set. During the connection process, newly generated connection segments are checked, and connection calculations are performed. The actual shortest distance from each point on the segment to the boundary of the safe and passable polygon region is calculated. If this distance is less than the value of the point in the distance transformation graph multiplied by a tolerance factor (e.g., 0.9), the segment is considered to fall outside the boundary and is discarded. At the same time, it is checked whether the segment intersects with the existing skeleton line. This is identified by judging whether there are non-endpoint intersections between the new segment and the existing skeleton line. If there is an intersection, the intersection is trimmed, and only the part with the same extension direction is retained. After connecting and trimming all local maxima points, the entire skeleton is finally refined. For example, the Zhang-Suen refinement algorithm is used to remove redundant burrs and isolated points. Finally, a network structure composed of curves and intersections with a width of one pixel is obtained, which is the equidistant topological skeleton line set.

[0069] The steps to obtain the obstacle avoidance path are as follows:

[0070] Read the coordinate sequence, branch connection position, start position and end position of each skeleton line in the equidistant topology skeleton line set. Arrange the coordinate sequence of each skeleton line in the extension order from the start position to the end position. Convert each coordinate point in the coordinate sequence into a line drawing control point. Verify the connection continuity and transition order of adjacent line drawing control points. Connect the line drawing control points that meet the continuous line drawing requirements in sequence to generate an obstacle avoidance line drawing path.

[0071] Specifically, the equidistant topological skeleton line set is read. This set is represented as a graph in the data structure, where nodes are the endpoints or branch connections of the skeleton lines, and edges are the skeleton line segments connecting the nodes. Each edge is accompanied by a coordinate sequence. First, the two skeleton points closest to the starting and ending points of the original drawing task are found in the skeleton line set and marked as the starting point and ending point corresponding positions, respectively. Then, Dijkstra's algorithm is applied to find the shortest path from the starting point to the ending point on the skeleton graph. The edge weights of this algorithm can be simply set as the geometric lengths of the edges. The resulting path is the skeleton line sequence of the main drawing path. For other branches in the skeleton, if one end connects to the main path, it is considered a candidate detour path, and the detour path is determined according to the coordinate sequence from the starting point. The extension sequence corresponding to the endpoint is set, and the coordinate sequences of each skeleton line on the main path are arranged and spliced ​​in sequence to form a continuous list of coordinate points. Each coordinate point in this list is converted into a control point of the line-marking robot or system. After conversion, the continuity of adjacent line-marking control points is checked, and the distance between every two consecutive points is calculated. If the distance exceeds the preset maximum step size threshold, such as 5 pixels, linear interpolation is performed between the two points to increase the point density. This threshold is set according to the motion smoothness requirements of the line-marking device. At the same time, the transition sequence is checked to ensure that the path does not self-intersect or swing sharply back and forth. For line-marking control points that meet all the continuous line-marking requirements, they are connected in sequence to finally generate an obstacle avoidance line-marking path that can be directly used by the device.

[0072] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for generating obstacle avoidance line-marking paths based on image semantic segmentation, characterized in that, Includes the following steps: The acquired image to be processed is input into a pre-trained semantic segmentation neural network, which outputs a semantic category probability matrix. Based on the semantic category probability matrix, pixel clusters belonging to the background category are extracted to construct a background surface region, and pixel clusters belonging to the regular obstacle category are extracted to construct a regular obstacle surface region. A gridded connected topology graph is constructed for the background surface region, and the cost value parameter is obtained by combining the distribution pattern of the conventional obstacle surface region; a pathfinding algorithm is executed in the connected topology graph according to the cost value parameter to obtain the lowest cost discrete polyline; the lowest cost discrete polyline is smoothed to generate a basic obstacle avoidance smooth trajectory. Extend the basic obstacle avoidance smooth trajectory outward by a set distance to obtain the closed area of ​​weak feature obstacles on the ground; A spatial expansion operation is performed on the basic obstacle avoidance smooth trajectory to generate an initial passageway region. The difference between the initial passageway region and the closed region of the weak feature obstacle on the ground is calculated to generate a safe and passable polygon region. Extract the discrete coordinate points of the boundary of the safe and passable polygonal region, calculate the distance from each point inside the safe and passable polygonal region to the boundary, and filter to obtain an equidistant topological skeleton line set; convert the coordinate sequence of the equidistant topological skeleton line set into line control point positions, and generate obstacle avoidance line drawing paths.

2. The obstacle avoidance path generation method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the conventional obstacle surface region are as follows: The acquired images to be processed are input into a pre-trained semantic segmentation neural network in pixel coordinate order. The color channel values, edge texture values, and scale level response values ​​of the images to be processed are called. The scale level response values ​​are upsampled, aligned, and the category probability is normalized to obtain the semantic category probability matrix. Based on the semantic category probability matrix, the probability column corresponding to the background category is read pixel by pixel, the relationship between the background category probability and the probability of other categories is compared, the pixel coordinates with the highest background category probability are marked as background pixels, and the background pixels are merged according to the continuity of adjacent coordinates to form a background region. Based on the semantic category probability matrix, the probability columns corresponding to the regular obstacle categories are read pixel by pixel. The numerical order of the regular obstacle category probability, the background category probability, and the other category probabilities is compared. The pixel coordinates with the highest regular obstacle category probability are marked as regular obstacle pixels. Regular obstacle pixels are merged according to the continuity of adjacent coordinates to form a regular obstacle surface region.

3. The obstacle avoidance path generation method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the minimum cost discrete piecewise linear curve are as follows: The background surface is divided into continuous grid nodes according to a preset grid side length. The center coordinates, adjacent coordinates and boundary distance of each continuous grid node are read. Combined with the overlapping area, adjacent distance and occlusion direction of the conventional obstacle surface, the grid nodes in the connected topology graph are marked with cost parameters to generate a connected topology graph marked with cost parameters. Based on the connected topology graph of the marked cost parameters, the grid nodes where the starting point and the ending point are located are read, a pathfinding algorithm is executed, the cumulative cost parameters of adjacent grid nodes are compared layer by layer, grid nodes that enter the conventional obstacle surface region are removed, and the lowest cost discrete polyline between the starting point and the ending point is extracted.

4. The obstacle avoidance path generation method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the basic obstacle avoidance smooth trajectory are as follows: Based on the minimum cost discrete polyline, the coordinate difference, turning angle, and node spacing of adjacent polyline nodes are read. Laplace smoothing calculation is performed on the minimum cost discrete polyline. Piecewise polynomial curvature parameters are extracted from the smoothed nodes. Curvature is continuously corrected for polyline nodes whose turning angle exceeds a preset sharp corner threshold to generate a basic obstacle avoidance smooth trajectory.

5. The obstacle avoidance path generation method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the closed region of the weakly featured ground obstacle are as follows: The basic obstacle avoidance smooth trajectory is extended outward by a set distance along the trajectory normal direction. The node coordinates, tangential direction and normal direction of the basic obstacle avoidance smooth trajectory are read point by point. The extension points on the same side are connected in node order, and the extension points at both ends are closed and connected to construct the initial outline of the zero level set. Based on the initial contour of the zero level set, the pixel state of the image surrounding the initial contour of the zero level set is read, and the gray value, texture response value and gradient magnitude are extracted pixel by pixel. The global statistical mean parameters of the region inside and outside the contour are calculated respectively. The deviation values ​​inside and outside the evolution curve are calculated and the region energy term is constructed. The level set function is solved iteratively and partial differential dimension reduction is performed. When the change of the region energy term in continuous iteration is less than the preset convergence threshold, the zero level set coordinates are extracted to obtain the closed region of weak feature obstacles on the ground.

6. The obstacle avoidance path generation method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the safe and passable polygonal region are as follows: Based on the basic obstacle avoidance smooth trajectory and the closed region of weak feature obstacles on the ground, a spatial expansion operation is performed on the basic obstacle avoidance smooth trajectory to generate an initial passageway region. The difference between the initial passageway region and the closed region of weak feature obstacles on the ground is calculated. The starting coordinates and ending coordinates in the difference result are checked to see if they belong to the same connected region. If the difference result maintains the connection between the starting and ending points, the largest connected region is extracted. If the connection is broken, the radius of the spatial expansion operation is expanded and the difference is recalculated until the difference result maintains the connection between the starting and ending points, generating a topologically continuous safe and passable polygon region.

7. The obstacle avoidance path generation method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the equidistant topological skeleton wire set are as follows: Extract the boundary discrete coordinate points of the safe and passable polygonal region, record the boundary turning positions and interpolation positions in coordinate order along the boundary contour of the safe and passable polygonal region to form a sequence of boundary discrete coordinate points, read the coordinates of internal points point by point inside the safe and passable polygonal region, calculate the shortest distance from each internal point coordinate to all boundary discrete coordinate points, compare the shortest distance values ​​corresponding to each internal point coordinate, and filter the internal point coordinates with equal shortest distance values ​​and the shortest distance values ​​reaching the maximum value to form the set of coordinates of the center of the largest inscribed circle. Read the spatial position, adjacency relationship and boundary equidistant relationship of each center coordinate in the set of coordinates of the largest inscribed circle. Based on the consistency of the corresponding distance from each center coordinate to the discrete coordinate point of the boundary, connect the center coordinates that satisfy the continuous extension relationship segment by segment. Remove the connecting segments that fall outside the boundary and the connecting segments that cross and fold back. Perform topological skeleton extraction operation on the safe and passable polygonal region to obtain the equidistant topological skeleton line set.

8. The obstacle avoidance path generation method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the obstacle avoidance path are as follows: Read the coordinate sequence, branch connection position, starting position and ending position of each skeleton line in the equidistant topological skeleton line set. Arrange the coordinate sequence of each skeleton line in the extension order from the starting position to the ending position. Convert each coordinate point in the coordinate sequence into a line drawing control point. Verify the connection continuity and transition order of adjacent line drawing control points. Connect the line drawing control points that meet the continuous line drawing requirements in sequence to generate an obstacle avoidance line drawing path.

Images