Edge following profile acquisition method for robot walking

Abstract

The application discloses a method for acquiring a contour along an edge for robot walking, comprising the following steps: A, processing an image collected by a robot through a target detection model to obtain a target frame and category information of an obstacle surrounded by the target frame; B, performing image shape processing on the image collected by the robot based on the target frame to obtain a binary image; C, extracting contour points of the obstacle of a corresponding category based on the binary image, and excluding invalid points from the extracted contour points based on an area threshold; D, extracting valid contour points from the contour points after the invalid points are excluded in a candidate region closest to a robot walking surface and framed by the target frame; E, performing regression fitting processing on the valid contour points, and introducing R into scoring; and selecting a boundary of the candidate region closest to the robot walking surface and occupying the target frame or a fitting line generated by the regression fitting processing as the contour along the edge according to a score value.

Description

Technical Field

[0001] This application relates to the technical field of visual inspection, specifically to a method for obtaining edge contours for robot walking. Background Technology

[0002] Existing mobile robots use image recognition technology with rectangular frames for object localization and obstacle avoidance. The contour extraction operation often focuses resources on distinguishing the inner and outer edges of the rectangle and processing the fitting accuracy of the rectangle's corners to straight lines. However, it does not adapt to the actual shape and size of ground obstacles. If there are obstacles such as power lines or long objects on the ground, the mobile robot cannot correctly locate these obstacles using the aforementioned image recognition technology with rectangular frames. This causes the mobile robot to fail to avoid obstacles correctly during actual navigation, resulting in missed areas or frequent collisions with ground obstacles, thus reducing the mobile robot's obstacle avoidance capability in the ground environment. Summary of the Invention

[0003] This application discloses a method for obtaining edge contours for robot walking, and the specific technical solution is as follows: A method for obtaining edge contours for robot walking includes the following steps: Step A: Processing the image acquired by the robot using a target detection model to obtain the target box and the category information of the obstacles enclosed by the target box, and then executing Step B; Step B: Performing image morphology processing on the image acquired by the robot based on the target box to obtain a binarized image, and then executing Step C; Step C: Extracting contour points of obstacles of the corresponding category based on the binarized image, and then excluding invalid points from the extracted contour points based on an area threshold, and then executing Step D; wherein, the area threshold is preset according to the category information of the obstacles enclosed by the target box; Step D: Extracting valid contour points from the contour points after excluding invalid points in the candidate region closest to the robot's walking surface defined by the target box, and then executing Step E; Step E: Performing regression fitting processing on the valid contour points, and introducing R-squared for scoring to obtain a score value; then selecting the boundary occupied by the candidate region closest to the robot's walking surface in the target box or the fitted line generated by the regression fitting processing as the edge contour based on the score value, so that the robot walks along the edge contour. In summary, by executing steps A to E, the robot performs image morphology processing on the images acquired by the robot based on the target bounding box, extracts the contour points of obstacles of the corresponding category, and excludes invalid points from the extracted contour points based on an area threshold. Then, valid contour points are extracted from the candidate region closest to the robot's walking surface within the target bounding box. Therefore, by optimizing the contour points within the target bounding box, obstacle contours with less noise and close to the robot's walking surface are extracted, suppressing the influence of interference regions on subsequent contour fitting operations. Then, in the regression fitting process of the valid contour points, R-squared is introduced for scoring to evaluate the fitting quality of the obstacle contours of the corresponding category. Based on the score value, the boundary occupied by the candidate region closest to the robot's walking surface within the target bounding box or the fitting line generated by the regression fitting process is selected as the edge contour. The robot automatically selects a suitable edge contour based on the fitting calculation of the valid contour points extracted within the candidate region closest to the robot's walking surface, allowing the robot to walk along the edge contour, thus improving the robustness of the robot's navigation and obstacle avoidance actions in the ground environment.

[0004] Compared with existing technologies, the edge contour acquisition method disclosed in this application accurately acquires obstacle contours and eliminates noise contour information based on the candidate region closest to the robot's walking surface defined by the target box. It also introduces a regression method to design a new evaluation index. Based on the evaluation score, the boundary occupied by the candidate region closest to the robot's walking surface in the target box or the fitted line generated by regression fitting is selected as the edge contour. When applied to ground obstacle avoidance scenarios with complex obstacles, especially wires and long objects, it can solve the problem of difficult evaluation of complex contours and improve the robustness of contour extraction. Therefore, during the obstacle avoidance operation, the robot reduces the omission of ground obstacle distribution areas and edge-accessible areas due to obstacle contour extraction not being biased towards fitting the contours of obstacles distributed on the walking surface or not specifically defining the contours of obstacles distributed on the walking surface.

[0005] Further, in step E, the method of selecting the boundary occupied by the candidate region closest to the robot's walking surface within the target box or the fitted line generated by regression fitting as the edge profile based on the score value includes: determining whether the score value is lower than a preset fit threshold; when it is determined that the score value is lower than the preset fit threshold, selecting the boundary occupied by the candidate region closest to the robot's walking surface within the target box as the edge profile; wherein, the boundary occupied by the candidate region closest to the robot's walking surface within the target box includes the boundary closest to the robot's walking surface within the target box; and dividing a relatively large obstacle avoidance field of view within the candidate region closest to the robot's walking surface.

[0006] When a score value is determined to be higher than or equal to a pre-set fit threshold, the fitted line generated by the regression fitting process is selected as the edge contour. This is sufficient to fit a continuous and reliable contour line, and the robot's pose can be adjusted using the parameters of the fitted line to correct the robot's movement along the obstacle contour enclosed by the target box.

[0007] Further, in step E, the method of performing regression fitting on the effective contour points and introducing R-squared for scoring includes: performing regression fitting on the effective contour points to obtain a contour fitting regression model; controlling the value 1 minus the ratio between the residual sum of squares and the total sum of squares, and then recording the difference as the score value; wherein, the residual sum of squares is the sum of squares of the differences between the model predictions of the contour fitting regression model and the actual observed values; the total sum of squares is the sum of squares of the differences between the mean of the dependent variable of the contour fitting regression model and the actual observed values; wherein, the contour fitting regression model is a polynomial fitting regression equation, making the contour fitting regression model represent the fitting line generated by the regression fitting process. For the same effective contour point set, a higher score value indicates a smaller difference between the observed data and the fitted values. Thus, when the score value is higher than or equal to the preset fit threshold, the fitting line generated by the regression fitting process is selected as the edge contour, which also proves that all the effective contour points extracted in step D become the optimal contour point set of the obstacle contour enclosed by the fitted target box.

[0008] Further, in step D, the method for extracting valid contour points from the contour points after excluding invalid points within the candidate region closest to the robot's walking surface defined by the target box includes: designating the candidate region closest to the robot's walking surface defined by the target box as the image region to be extracted; wherein, the image region to be extracted is the region within the area defined by the target box, located below the reference function line of the target box; the reference function line is located between the upper boundary and the lower boundary of the target box, and the lower boundary of the target box is the boundary within the target box closest to the robot's walking surface; dividing the image region to be extracted into a predetermined number of target intervals along a direction parallel to the robot's walking surface; among the contour points after excluding invalid points, extracting one contour point closest to the lower boundary of the target interval within each target interval, and designating the contour point extracted from the target interval as a valid contour point, accumulating a predetermined number of valid contour points; wherein, the lower boundary of a target interval is the boundary within a target interval closest to the robot's walking surface. Therefore, this application sets a corresponding preset number of target intervals based on the category information of the obstacles surrounded by the target box obtained in step A. Thus, the robot can divide the part of the obstacle surrounded by the target box in the image area to be extracted into a reasonable number of regions according to the preset number of target intervals. This enables the division of a preset number of target intervals in the region where the obstacle is close to the robot's walking surface. The robot also extracts the contour points closest to the robot's walking surface from each target interval to form the contour point set that needs to be fitted later. This provides contour points that are close to the robot's walking surface and can characterize the edge of the ground obstacle for regression fitting.

[0009] Further, in step B, the method for image morphology processing of the image acquired by the robot based on the target bounding box specifically includes: expanding and shrinking the target bounding box, and performing grayscale conversion, edge detection, and morphological denoising on the region defined by the expanded target bounding box; then shrinking the expanded target bounding box, and then setting the image containing the shrunken target bounding box as the binarized image; wherein, the target bounding box before expansion is a circumscribed polygon enclosing the contour of an obstacle of a certain type. Thus, during the expansion and shrinking of the target bounding box, grayscale conversion, edge detection, and morphological denoising are used to detect complete and continuous contours in the obstacle.

[0010] Further, in step B, the process of expanding and shrinking the target box includes: setting the boundary extension direction of the target box perpendicular to the robot's walking surface as a first preset expansion direction, and setting the opposite direction of the first preset expansion direction and the perpendicular direction of the first preset expansion direction as second preset expansion directions; controlling the target box to expand towards the first preset expansion direction by a first preset proportion, and simultaneously controlling the target box to expand towards the second preset expansion direction by a second preset proportion, to obtain a preset expanded box, so as to provide pixel information in the neighborhood of the obstacle contour enclosed by the target box; after the target box has been expanded towards the first preset expansion direction and the second preset expansion direction respectively, the contour points of the obstacle in each direction are obtained by edge detection, and a preprocessed image is obtained; then the morphological denoising is performed on the preprocessed image; then the preset expanded box is shrunk back to the target box to remove the contour points in the area where the target box expands in each direction. Therefore, compared with existing technologies that filter contour points row by row and column by column in the whole frame image, this application adopts an expansion and contraction strategy after obtaining the target box. It makes full use of the pixel information detected by the edge while removing noise pixels in the expanded area, reducing the amount of computation. Specifically, it can reduce thousands of pixels to 100 precise points for obstacle contour recognition and localization, making the calculation more efficient.

[0011] Further, the target frame is a rectangle; the method for expanding the target frame includes: setting the extension direction of the boundary of the rectangle perpendicular to the robot's walking surface as a first preset expansion direction, setting the opposite direction of the first preset expansion direction as a second preset expansion direction, setting the direction of the boundary of the rectangle parallel to the robot's walking surface extending to one side as a second preset expansion direction, and setting the direction of the boundary of the rectangle parallel to the robot's walking surface extending to the other side as a second preset expansion direction; wherein, the second preset expansion direction is the second preset expansion direction, the second preset expansion direction, or the second preset expansion direction; during the process of controlling the target frame to expand towards the first preset expansion direction by a first preset ratio, the boundary of the rectangle parallel to the first preset expansion direction extends along the first preset expansion direction by a first preset ratio; controlling the target frame to expand towards the second preset expansion direction by a second preset ratio. In the example process, the boundary parallel to the first preset expansion direction within the rectangular frame is extended by a second preset proportion along the second preset expansion direction; while controlling the target frame to expand by a second preset proportion along the second preset expansion direction, the boundary parallel to the second preset expansion direction within the rectangular frame is extended by a second preset proportion along the second preset expansion direction; while controlling the target frame to expand by a second preset proportion along the second preset expansion direction, the boundary parallel to the second preset expansion direction within the rectangular frame is extended by a second preset proportion along the second preset expansion direction; when the target frame expands along the first preset expansion direction, the second preset expansion direction, the second preset expansion direction, and the second preset expansion direction until each extended boundary intersects at four boundary points, the preset expansion frame is obtained; wherein, the first preset proportion is greater than the second preset proportion, so as to obtain more contour points in the detection direction close to the ground. In summary, after obtaining the target frame in step A, step B adopts the aforementioned expansion strategy for the target frame, making full use of edge pixel information to calculate the gradient, and obtaining pixel information in the neighborhood of the obstacle contour surrounded by the target frame, especially obtaining more contour points in the detection direction close to the ground.

[0012] Further, in step B, the method for edge detection of the region defined by the expanded target box includes: within the preset expanded box, calculating the pixel gradient in the neighborhood of a pixel point based on a target operator; when an extreme value is obtained during the calculation of the pixel gradient, determining the contour point of the detected obstacle; and determining the position of the detected pixel value with the largest change within the preset expanded box. This achieves the determination of the contour position by calculating gradient changes.

[0013] Further, in step C, the method for extracting the contour points of obstacles of the corresponding category based on the binarized image includes: sequentially traversing each contour point detected in step B in the binarized image, then retaining the contour points used to connect them into a closed contour line, and marking the retained contour points as the contour points of the corresponding category of obstacles extracted in step C; wherein, the line connecting each contour point detected in step B is used to represent the contour of obstacles of the same category. The shape of the line connecting each contour point detected in step B is used to describe the shape of obstacles of the same category, providing a path that can be followed along the edge for the robot's obstacle avoidance.

[0014] Furthermore, in step B, when the detected contour points are configured to form multiple closed contour lines in sequence, if one closed contour line is contained within another closed contour line, then that closed contour line is set as a sub-contour of the other closed contour line to form a multi-level contour structure. Within the target box, each closed contour line is used to represent the contour of an obstacle of the same category. The technical advantages of the robot constructing a multi-level contour structure are: 1. It can better extract the outer contours of the same obstacle because, during the construction of the multi-level contour structure, each closed contour line is assigned a marker to indicate its position in the multi-level contour structure; 2. It can distinguish between primary and secondary objects. For example, if there is a striped shape of a flower on a shoe, the primary object will be the shoe contour rather than the striped shape of the flower. It is necessary to exclude the interference of secondary objects. Therefore, this application can more accurately calculate the area and perimeter of the primary object to extract the required contour.

[0015] Further, in step C, the method for excluding invalid points from the extracted contour points based on an area threshold includes: calculating the area of ​​the region enclosed by each closed contour line; determining whether the area of ​​the region enclosed by each closed contour line is greater than or equal to the area threshold; when the area of ​​the region enclosed by a closed contour line is determined to be greater than or equal to the area threshold, controlling all contour points in the currently determined closed contour line to be retained in the binarized image; when the area of ​​the region enclosed by a closed contour line is determined to be less than the area threshold, discarding the currently determined closed contour line, wherein all contour points in the currently determined closed contour line are marked as invalid points. This method, by setting a certain area threshold, excludes excessively small contours and retains contours that are valuable for the robot's obstacle avoidance along edges.

[0016] Furthermore, in step A, after processing the image acquired by the robot using the target detection model, if multiple target boxes are obtained in a single frame image, the target box surrounding the obstacle closest to the robot is set as the target box obtained in step A, and the category information of the obstacle closest to the robot surrounded by the target box is obtained. Therefore, in the single frame image currently acquired by the robot, only the obstacle closest to the robot's frontal field of view is processed, and the category information of the obstacle closest to the robot is obtained for subsequent elimination of excessively small contour lines. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a method for obtaining edge contours for robot walking, as disclosed in this application. Detailed Implementation

[0018] To provide a clearer description of the present invention, specific embodiments are given below for further explanation. In the following description, specific details such as particular system structures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that the present application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0019] Furthermore, in the description of this application, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0020] This application addresses the challenge of assessing complex contours in ground terrain by identifying the actual shape and size of obstacles using bounding boxes and selecting appropriate contour information for obstacle avoidance. The method discloses an edge contour acquisition method for robot movement, where the robot itself is the execution entity, or specifically, the robot's built-in controller (a control device with visual image processing capabilities). This method is implemented in scenarios where the robot avoids obstacles. The obstacles disclosed in this application are ground objects within the robot's captured image that impede its movement. These objects are generally close to the robot's real-time position and fall within the detection field of a visual sensor or laser sensor. The outer contour of these obstacles can be defined by bounding boxes.

[0021] like Figure 1 As shown, the method for obtaining the edge contour includes the following steps: Step A: Process the images acquired by the robot using the object detection model to obtain the target bounding box and the category information of the obstacles enclosed by the target bounding box, and then proceed to Step B; Whenever the robot acquires an image frame through the camera, it uses the object detection model to identify and locate the object to be identified, and uses a target bounding box to surround the object to be identified. If there are multiple objects to be identified, multiple target bounding boxes are used, and each target bounding box surrounds one object to be identified. In the robot navigation and obstacle avoidance scenario applicable to this application, by running the object detection model, in the currently acquired image, an object to be identified is an obstacle of one category, and a target bounding box surrounds one obstacle, thereby obtaining the category information of the target bounding box and the obstacles enclosed by the target bounding box in Step A.

[0022] When multiple bounding boxes are obtained through the object detection model, the distance between the obstacle enclosed by each bounding box and the robot's current position can be calculated separately using a monocular ranging algorithm. Then, the distances between the obstacle enclosed by each bounding box and the robot's current position are compared, and the bounding box containing the obstacle with the smallest distance is output. This output includes the category information of the obstacle closest to the robot's current position and the bounding box containing that obstacle.

[0023] Preferably, the target detection model is the neural network model used by the YOLO (You Only Look Once) algorithm to achieve end-to-end target detection. For example, the image collected by the robot is input into the neural network model, and the position of the target box in the image and its category are obtained in the output layer. The target box is a polygonal box such as a rectangle, triangle, or trapezoid that has been trained by the neural network model. The target box can be understood as the smallest bounding polygon of the obstacle, and the target box can surround the outermost contour of the obstacle.

[0024] Step B: Perform image morphology processing on the image acquired by the robot based on the target bounding box to obtain a binarized image, and then execute step C; In step B, the robot uses the image region defined by the target bounding box to extract the range associated with the robot's walking surface for processing, including expanding the region, capturing contour points, denoising, partitioning, etc., to obtain a binarized image, which facilitates the robot to store and retrieve information in the image.

[0025] To extract obstacle contours from the robot-acquired images with maximum range and precision, the robot expands and shrinks the target box in step B. This expansion and shrinkage of the target box is equivalent to performing an edge expansion and a shrinkage process near the obstacle's contour. During edge expansion, the obstacle's contour is extracted completely and accurately based on pixel gradient information. Then, during shrinkage, noise points are removed, such as pixels truncated in the obstacle's contour. Continuous and reliable contour points are retained within the shrunken target box before proceeding to step C. Existing techniques for contour extraction of entire frames may involve excessive computational cost due to layer-by-layer or point-by-point contour localization and alignment, leading to reduced obstacle localization and recognition speed. This application, however, utilizes the target box for image morphology processing, allowing for the removal and retention of contour points around the target box boundary, thus reducing computational cost compared to existing techniques.

[0026] Step C: Extract contour points of obstacles of the corresponding category based on the binarized image. This extraction operation occurs within the obstacle contours enclosed by the target bounding box that participated in image morphology processing in Step B, using as few pixels as possible to represent or approximate the obstacle contours enclosed by the target bounding box. Then, invalid points are excluded from the extracted contour points based on an area threshold, thus eliminating contour lines with abnormal contour area from the same category of obstacle contours. Then, Step D is executed. In this embodiment, the area threshold is preset according to the category information of the obstacles enclosed by the target bounding box. When the category information of the obstacle changes, the area threshold set for the corresponding category of obstacle changes, and of course, the target bounding box surrounding the corresponding category of obstacle also changes.

[0027] Step D: Within the candidate region closest to the robot's walking surface defined by the target bounding box, extract valid contour points from the contour points after excluding invalid points, thereby optimizing the contour within the candidate region closest to the robot's walking surface, ensuring that the valid contour points extracted within this candidate region are the contour points closest to the robot's walking surface; then execute step E.

[0028] Preferably, in step D, the robot's walking surface is the ground, and the target frame is set perpendicularly or obliquely to the robot's walking surface. The target frame surrounds the obstacle closest to the robot, at least defining the outline of the ground-facing portion of the obstacle closest to the robot, i.e., defining the outline area that is more likely to collide with the walking robot; this is beneficial for the robot to use it for obstacle avoidance. The candidate area closest to the robot's walking surface defined by the target frame is preferably the sub-region closest to the ground among the multiple sub-regions divided within the target frame. The sub-region closest to the ground can be understood as the area below the reference function line (e.g., the horizontal midline) of the target frame. The height of the lower boundary of the target frame relative to the walking surface (horizontal ground line) detected by the robot's visual sensor or laser sensor can be set to no higher than a preset height threshold. The preset height threshold can be set to 8 centimeters, which facilitates the extraction of the outline points of obstacles distributed on the ground in a reasonable direction and excludes situations such as wires hanging above walls or shoes on shoe racks in the robot's walking environment, thereby retaining the defined ground obstacles within the target frame.

[0029] To accelerate obstacle avoidance, the candidate area closest to the robot's walking surface can be divided into intervals. In each interval, only the point closest to the lower boundary of the interval is selected as the effective contour point. This allows for the extraction of the most reasonable number of contour points within a limited interval to describe the contour distribution characteristics of ground obstacles.

[0030] Step E: Perform regression fitting on the effective contour points and introduce R-squared for scoring to obtain a score value; then, based on the score value, select the boundary occupied by the candidate region closest to the robot's walking surface in the target box or the fitted line generated by the regression fitting as the edge contour, so that the robot walks along the edge contour. Specifically, after the robot extracts all effective contour points within a single target box in step D, it performs regression fitting on the effective contour points to obtain a contour fitting regression model; then, it introduces R-squared to score the contour fitting regression model and obtains a score value; where R-squared is one of the statistics used for the goodness of fit of the contour fitting regression model, it represents the proportion of the variance of the dependent variable in the contour fitting regression model that can be explained by the independent variable, and R-squared refers to the score value in this application.

[0031] When the score is greater than a goodness-of-fit threshold, the fitting line generated by the regression fitting process can accurately fit the contour of the obstacle that the robot needs to avoid, and the extraction quality of each effective contour point is good. The robot can use the curve parameters generated by the regression fitting process to adjust its pose to walk along the fitting line. When the score is less than or equal to a goodness-of-fit threshold, the fitting line generated by the regression fitting process cannot accurately fit the contour of the obstacle that the robot needs to avoid. The boundary occupied by the candidate region closest to the robot's walking surface in the target box is used to replace the actual contour of the obstacle, so that the robot walks according to the occupied boundary. Then, the robot walks directly in the local area according to the lower boundary of the target box obtained at the current moment (the smallest outer edge of the contour of the obstacle currently surrounded by the target box). The idea of ​​replacing curves with straight lines is used to walk along the edge of the overall contour of the ground obstacle.

[0032] In summary, by executing steps A to E, the robot performs image morphology processing on the images acquired by the robot based on the target bounding box, extracts the contour points of obstacles of the corresponding category, and excludes invalid points from the extracted contour points based on an area threshold. Then, valid contour points are extracted from the candidate region closest to the robot's walking surface within the target bounding box. Therefore, by optimizing the contour points within the target bounding box, obstacle contours with less noise and close to the robot's walking surface are extracted, suppressing the influence of interference regions on subsequent contour fitting operations. Then, in the regression fitting process of the valid contour points, R-squared is introduced for scoring to evaluate the fitting quality of the obstacle contours of the corresponding category. Based on the score value, the boundary occupied by the candidate region closest to the robot's walking surface within the target bounding box or the fitting line generated by the regression fitting process is selected as the edge contour. The robot automatically selects a suitable edge contour based on the fitting calculation of the valid contour points extracted within the candidate region closest to the robot's walking surface, allowing the robot to walk along the edge contour, thus improving the robustness of the robot's navigation and obstacle avoidance actions in the ground environment.

[0033] Compared with existing technologies, the edge contour acquisition method disclosed in this application accurately acquires obstacle contours and eliminates noise contour information based on the candidate region closest to the robot's walking surface defined by the target box. It also introduces a regression method to design a new evaluation index. Based on the evaluation score, the boundary occupied by the candidate region closest to the robot's walking surface in the target box or the fitted line generated by regression fitting is selected as the edge contour. When applied to ground obstacle avoidance scenarios with complex obstacles, especially wires and long objects, it can solve the problem of difficult evaluation of complex contours and improve the robustness of contour extraction. Therefore, during the obstacle avoidance operation, the robot reduces the omission of ground obstacle distribution areas and edge-accessible areas due to obstacle contour extraction not being biased towards fitting the contours of obstacles distributed on the walking surface or not specifically defining the contours of obstacles distributed on the walking surface.

[0034] As one embodiment, in step E, the method of selecting the boundary occupied by the candidate region closest to the robot's walking surface within the target box or the fitted line generated by regression fitting as the edge profile based on the score value includes: The system determines whether the score is lower than a pre-set goodness-of-fit threshold. This threshold is used to evaluate the difference between the fitted line generated by the regression fitting process and the contour of the obstacle enclosed by the target box. It should be noted that R-squared is a statistic used to evaluate the goodness of fit of a regression model; it represents the proportion of the variance of the dependent variable that can be explained by the independent variable. The range of R-squared (i.e., the range of the score) is between 0 and 1. The closer the score is to 1, the stronger the model's explanatory power for the observed data, and the closer the fitted line generated by the regression fitting process is to the contour of the obstacle enclosed by the target box. In this embodiment, the score specifically represents the difference between the coordinates of the effective contour points and the coordinates of the contour points predicted during the regression fitting process.

[0035] When the score is determined to be lower than the preset fit threshold, the boundary occupied by the candidate region closest to the robot's walking surface within the target box is selected as the edge contour; wherein, the boundary occupied by the candidate region closest to the robot's walking surface within the target box includes the boundary closest to the robot's walking surface within the target box, and a relatively large obstacle avoidance field of view is defined within the candidate region closest to the robot's walking surface.

[0036] Preferably, when the target box is perpendicular to the robot's walking surface and the score is lower than a preset fitting threshold, the target box can use its lower boundary as the edge contour, so that the robot can use the lower bounding box to replace the real contour for edge walking, that is, walk along the boundary of the smallest bounding box of the real contour to avoid the obstacle where the real contour is located.

[0037] When the score is found to be higher than or equal to the preset fitting threshold, the fitting line generated by the regression fitting process is selected as the edge contour. If the fitting quality score of the obstacle contour surrounded by the target box is high, it indicates that the contour point extraction quality is good and sufficient to fit a continuous and reliable contour line. The parameters of the fitting line can be used to adjust the robot's pose and correct the way the robot walks along the obstacle contour surrounded by the target box.

[0038] In the above embodiments, the method of performing regression fitting on the effective contour points and introducing R-squared for scoring in step E includes: The effective contour points are subjected to regression fitting to obtain a contour fitting regression model; wherein, the contour fitting regression model is a polynomial fitting regression equation, so that the contour fitting regression model represents the fitting line generated by the regression fitting process; preferably, the fitting line corresponding to the contour fitting regression model is represented by a fitting curve, and the polynomial based on the least squares method is used to characterize the fitting curve.

[0039] The score is calculated by subtracting the ratio of the residual sum of squares to the total sum of squares from the control value of 1. The residual sum of squares is the sum of squares of the differences between the model predictions and the actual observed values ​​of the profile-fitted regression model; the total sum of squares is the sum of squares of the differences between the mean of the dependent variable and the actual observed values ​​of the profile-fitted regression model. Specifically, the formula for calculating R-squared is as follows: R-squared = 1 - (SSR / SST); where SSR represents the residual sum of squares, representing the sum of squares of the differences between the model predictions and the actual observed values; and SST represents the total sum of squares, representing the sum of squares of the differences between the actual observed values ​​and the mean of the dependent variable.

[0040] The R-squared value ranges from 0 to 1; where 0 indicates that the model cannot explain any variation in the dependent variable, that is, the model cannot explain any variation in the response variable near its mean; 1 indicates that the model can fully explain the variation in the dependent variable, that is, the model can explain all variation in the response variable near its mean.

[0041] R-squared is a measure of fit for a profile-fitted regression model, representing the percentage of variance of the dependent variable that the independent variables jointly explain. R-squared can conveniently measure the strength of the relationship between the model and the dependent variable using a percentage range of 0 to 100%.

[0042] A contour fitting regression model is an equation that minimizes the difference between all actual observations and fitted values ​​(model predictions). If used as a linear regression, it can find the smallest possible sum of squared residuals within the effective contour point set. In this case, R-squared evaluates the distribution of contour points around the fitted line generated by the regression fitting process. The corresponding calculated score is also called the coefficient of determination, or the multivariate coefficient of determination for multivariate regression. For the same effective contour point set, a higher score indicates a smaller difference between the observed data and the fitted values. Therefore, when the score is higher than or equal to a pre-set goodness-of-fit threshold, the fitted line generated by the regression fitting process is selected as the edge contour. This also proves that all the effective contour points extracted in step D become the optimal contour point set for the obstacle contour enclosed by the fitted target box.

[0043] As one embodiment, in step D, the method for extracting valid contour points from the contour points after excluding invalid points within the candidate region closest to the robot's walking surface defined by the target box includes: The candidate region closest to the robot's walking surface, defined by the target bounding box, is denoted as the image region to be extracted. This image region can be the sub-region closest to the ground among multiple sub-regions within the target bounding box. The image region to be extracted is the area below the reference function line of the target bounding box, i.e., the area defined from the reference function line towards the robot's walking surface. The reference function line is located between the upper and lower boundaries of the target bounding box, with the lower boundary being the boundary closest to the robot's walking surface. The image region to be extracted is the planar area defined in the visual space between the reference function line and the lower boundary of the target bounding box. Since the lower boundary of the target bounding box is the boundary closest to the robot's walking surface, the image region to be extracted lies between the reference function line and the boundary closest to the robot's walking surface. The lower boundary of the target bounding box is no more than 8 cm above the robot's walking surface or horizontal ground to define the outline of ground obstacles of relatively low height. The robot's walking surface or horizontal ground can be detected by the robot's vision sensor or laser sensor.

[0044] The reference function line can be set to be parallel to the robot's walking surface. It is understood that the reference function line can be parallel to the lower boundary of the target bounding box. When the reference function line is preferably the midline of the target bounding box, the distance between the reference function line and the lower boundary of the target bounding box is equal to half the height of the target bounding box, and the area of ​​the image region to be extracted is equal to half the area of ​​the target bounding box. For example, when the target bounding box is a rectangle, the reference function line is a line segment that bisects the rectangle along a direction parallel to the robot's walking surface. When the robot's walking surface is horizontal and the target bounding box is vertically or obliquely positioned on the horizontal surface, the reference function line is selected as the horizontal midline of the target bounding box.

[0045] Preferably, the robot's walking surface is a horizontal ground and parallel to the reference function line. The image region to be extracted is vertically set on the robot's walking surface. The contour points distributed in the image region to be extracted are located near the robot's body edge and above the robot's walking surface. The curve formed by connecting the contour points distributed in the image region to be extracted can be used as the contour of obstacles distributed on the horizontal ground. It also supports the regression fitting process of step E in the visual space. The contour of the obstacles distributed on the horizontal ground can be understood as being relatively close to the robot's body edge and relatively close to the horizontal ground.

[0046] Along a direction parallel to the robot's walking surface, a preset number of target intervals are divided from the image region to be extracted. Preferably, the direction parallel to the robot's walking surface is parallel to the reference function line. In order to define the contours of obstacles distributed on the ground and extract as many contour points as possible for regression fitting, this embodiment divides the image region to be extracted into a preset number of integer intervals. This yields a preset number of target intervals. That is, the image region to be extracted, which is used to enclose the contours of obstacles, is divided into equal intervals. Since some intervals may not contain contour points, the contour points are not necessarily uniformly distributed in each interval. When the obstacle is a continuous and regular object, the same number of contour points exist in each equally divided interval.

[0047] Among the contour points after excluding invalid points, the contour point closest to the lower boundary of each target interval is extracted, and the contour point extracted from the target interval is recorded as a valid contour point. A preset number of valid contour points are accumulated. In this embodiment, the lower boundary of a target interval is the boundary closest to the robot's walking surface within the target interval. The lower boundaries of the preset number of target intervals form the boundary closest to the robot's walking surface in the target frame, constituting the maximum field of view required for the robot to avoid obstacles on the ground or walk along obstacles. Therefore, this embodiment sets a corresponding preset number of target intervals based on the category information of the obstacles surrounded by the target frame obtained in step A. Thus, the robot can divide the part of the obstacle surrounded by the target frame within the image area to be extracted into a reasonable number of regions according to the preset number of target intervals. This achieves the division of the preset number of target intervals in the region where the obstacle is close to the robot's walking surface, and extracts the contour point closest to the robot's walking surface from each target interval to form the contour point set to be fitted later. This provides contour points close to the robot's walking surface that can characterize the edge of the ground obstacle for regression fitting processing.

[0048] It should be noted that the preset quantity is related to the category information of the obstacles enclosed by the target box; when the number of pixels of one category of obstacle outlines enclosed by a target box is large, the preset quantity is set to be large; when the number of pixels of another category of obstacle outlines enclosed by another target box is small, the preset quantity is set to be small; for two target boxes of different sizes, if the number of outline points of the wire enclosed by one target box is greater than the number of outline points of the shoe enclosed by the other target box, then different preset quantities need to be set for different target boxes to adapt to dividing the target intervals of the corresponding category of obstacle outlines within different target boxes.

[0049] As one embodiment, step B, the method for image morphology processing of the image acquired by the robot based on the target bounding box, specifically includes: The robot expands and shrinks the target bounding box. In this embodiment, for edge detection, the gradient calculation of the obstacle contour relies on the surrounding pixel values. However, existing technologies ignore gradient changes when extracting contour points within the target bounding box. Therefore, the target bounding box needs to be expanded and then shrunk back to its original shape and size. This utilizes the gradient values ​​of surrounding pixels while also reducing the number of noisy contour points. It should be noted that the target bounding box before expansion is a circumscribed polygon enclosing the contours of one type of obstacle, preferably the smallest rectangle that completely encloses the obstacle contours.

[0050] The process of expanding the target bounding box includes expanding each boundary of the target bounding box, and the expansion length of different boundaries may not be the same. The robot performs grayscale conversion, edge detection, and morphological denoising on the area defined by the expanded target bounding box. In this embodiment, for the purpose of denoising, the morphological denoising process performs closing operations (dilation followed by erosion) and opening operations (erosion followed by dilation) sequentially on the image region. This is equivalent to performing additional processing on the truncated pixels at the boundaries, which can eliminate sharp points in the edge-detected contours, making the contours smoother and preventing incomplete extraction of connected regions or loss of edges. Therefore, this embodiment performs morphological denoising after edge detection (the kernel size can be selected according to the actual situation) to facilitate the extraction of the contour points of the largest connected region of the obstacle within the expanded target bounding box.

[0051] Then, the expanded target bounding box is shrunk back, and the image containing the shrunk target bounding box is set as the binarized image. It should be noted that during the shrinking process of the expanded target bounding box, redundant contour points formed in each expanded region or contour points not close to the robot's walking surface are essentially deleted. When the target bounding box is restored to its original shape and size, it is equivalent to reducing the number of noisy contour points, but retaining the contour points obtained by edge detection performed during the expansion stage for the binarized image. Thus, through grayscale conversion, edge detection, and morphological denoising during the expansion and shrinking of the target bounding box, complete and continuous contours in obstacles can be detected.

[0052] Specifically, in step B, the process of expanding and shrinking the target box includes: In order to obtain more contour points in the detection direction close to the ground, that is, to obtain more contour information from the obstacles to be avoided, step B sets the boundary extension direction of the target box that is perpendicular to the robot's walking surface as the first preset extension direction, and sets the opposite direction of the first preset extension direction and the perpendicular direction of the first preset extension direction as the second preset extension direction. The second preset extension direction is then regarded by the robot as the direction that deviates from the robot's walking surface.

[0053] The target bounding box is expanded by a first preset proportion in a first preset expansion direction, and simultaneously expanded by a second preset proportion in a second preset expansion direction, to obtain a preset expanded bounding box. In this embodiment, the preset expanded bounding box is obtained by expanding each boundary of the target bounding box. After obtaining the intersection points of each boundary, the preset expanded bounding box is obtained, meaning the target bounding box has been expanded. Thus, by expanding the boundaries in the first and second preset expansion directions, the search range for obtaining pixel gradient change information is obtained, providing pixel information in the neighborhood of the obstacle contour enclosed by the target bounding box, overcoming the problem of indistinct pixel gradients within the original target bounding box. Then, by using the pixel values ​​of the object edges and their neighborhoods within the preset expanded bounding box, the gradient near the edges is calculated to extract the complete contour.

[0054] Then, after the target box is expanded in the first preset expansion direction and the second preset expansion direction respectively, the contour points of the obstacle in each direction are obtained by edge detection, and a preprocessed image is obtained; then the preprocessed image is subjected to the morphological denoising mentioned in the previous embodiment to make the contour obtained by edge detection smoother.

[0055] The edge detection is performed by extracting the contour within the area enclosed by the expanded boundary. Since the robot mainly uses points near the boundary that are close to the ground (which are more likely to collide with the walking robot), the robot's expansion operation in the first and second preset expansion directions can be understood as expanding the boundary within the maximum obstacle avoidance field of view. This allows for better utilization of the pixel gradient values ​​around the obstacle contour to extract the contour of the obstacle in various directions, thus enabling the robot to extract a relatively complete contour in the preprocessed image.

[0056] Then, the preset expanded bounding box is controlled to shrink back to the target bounding box to remove contour points in the areas where the target bounding box expands in all directions, thus reducing the number of noisy contour points. Then, step C is executed. In step B, while extracting the complete contour, noise points are introduced into the area where the preset expanded bounding box expands relative to the target bounding box, resulting in information redundancy and affecting the robot's navigation speed. Therefore, by shrinking back to the original target bounding box, contour points in the areas where the target bounding box expands in the second preset expansion direction can be removed, retaining the complete contour points in the detection directions required for the robot to avoid obstacles. This reduces the computational load during robot navigation and obstacle avoidance, thereby improving the target detection speed.

[0057] Therefore, compared with existing technologies that filter contour points row by row and column by column in the whole frame image, this application adopts an expansion and contraction strategy after obtaining the target box. It makes full use of the pixel information detected by the edge while removing noise pixels in the expanded area, reducing the amount of computation. Specifically, it can reduce thousands of pixels to 100 precise points for obstacle contour recognition and localization, making the calculation more efficient.

[0058] As one embodiment, the target frame is a rectangular frame, used to completely surround the outermost contour of ground obstacles. Viewed from the perspective of the robot acquiring the image, the rectangular frame is vertically positioned above the robot's walking surface to surround obstacles along the robot's edges or obstacles it might collide with in its walking direction. When the target frame is a rectangular frame, the method for expanding the target frame includes: setting the direction of the boundary of the rectangular frame extending vertically towards the robot's walking surface as a first preset expansion direction, representing the expansion direction closer to the ground; simultaneously setting the opposite direction of the first preset expansion direction as a second preset expansion direction; setting the direction of the boundary of the rectangular frame parallel to the robot's walking surface extending to one side as a second second preset expansion direction; and setting the direction of the boundary of the rectangular frame parallel to the robot's walking surface extending to the other side as a second third preset expansion direction, with the second first preset expansion direction being opposite to the second third preset expansion direction; wherein, the second preset expansion direction is either the second first preset expansion direction, the second second preset expansion direction, or the second third preset expansion direction.

[0059] In conjunction with the foregoing embodiments, during the process of controlling the target frame to expand in the first preset expansion direction by a first preset proportion, the boundary of the rectangular frame parallel to the first preset expansion direction is extended along the first preset expansion direction by a first preset proportion. For example, the target frame obtained in step A is expanded downward by 1 / 3, including controlling the boundaries on both the left and right sides of the target frame to be extended downward by 1 / 3 of the original boundary length; thereby realizing the expansion of the target frame in the direction perpendicular to the robot walking surface.

[0060] Simultaneously, during the process of controlling the target box to expand by a second preset proportion in the second preset expansion direction, the boundary of the rectangular box parallel to the first preset expansion direction is extended by a second preset proportion along the second preset expansion direction. For example, the target box obtained in step A is expanded upward by 1 / 4, including controlling the boundaries on both the left and right sides of the target box to be extended upward by 1 / 4 of the original boundary length; thus realizing the expansion of the target box in a direction perpendicular to the robot's walking surface.

[0061] Simultaneously, during the process of controlling the target box to expand in the second preset direction by the second preset proportion, the boundaries of the rectangle parallel to the second preset expansion direction are extended along the second preset expansion direction by the second preset proportion. For example, if the target box obtained in step A is expanded to the left by 1 / 4, this includes extending the top and bottom boundaries of the target box to the left by 1 / 4 of their original length. This achieves the expansion of the target box to the left.

[0062] Simultaneously, during the process of controlling the target box to expand by a second preset proportion in the second and third preset expansion directions, the boundaries of the rectangle parallel to the second and third preset expansion directions are extended by a second preset proportion along the second and third preset expansion directions. For example, if the target box obtained in step A is expanded to the right by 1 / 4, this includes extending the top and bottom boundaries of the target box to the right by 1 / 4 of their original length. This achieves the expansion of the target box to the right.

[0063] Note that the target bounding box is a rectangle. When the target bounding box is extended in the first, second, second, and third preset extension directions until the extended boundaries intersect at four boundary points, the preset extension bounding box is obtained. The gradient changes around the contour of the obstacle enclosed by the target bounding box are obtained, so as to use the gradient changes to detect contour points. The reason is that if the original target bounding box is close to the obstacle, the contour extraction will be defective because the gradient information cannot be obtained and the complete contour cannot be extracted. Therefore, the first, second, second, and third preset extension directions are extended to expand the bounding range of the target bounding box for the obstacle contour, which is beneficial for calculating the surrounding gradient.

[0064] Since the first preset ratio is greater than the second preset ratio, for example, the first preset ratio is set to 1 / 3 and the second preset ratio is set to 1 / 4, this embodiment tends to extract contour points near the boundary parallel to the robot's walking surface in the preset expansion box within the preset expansion direction, so as to obtain more contour points in the detection direction close to the ground, which is beneficial for the robot to use more contour points for obstacle avoidance along the edge.

[0065] In summary, after obtaining the target bounding box in step A, step B uses the aforementioned expansion strategy to fully utilize edge pixel information to calculate gradients and obtain pixel information in the neighborhood of the obstacle contour enclosed by the target bounding box, especially obtaining more contour points in the detection direction close to the ground.

[0066] As one embodiment, in step B, the method for edge detection of the region defined by the extended target box includes: within the preset extended box, pixel information of the neighborhood of the obstacle contour enclosed by the target box has already been provided for edge detection; then, pixel gradients in the neighborhood of the pixel points are calculated based on the target operator; when an extreme value is obtained during the calculation of the pixel gradient, the contour points of the detected obstacle are determined, and the position where the pixel value change is the largest detected within the preset extended box is determined, i.e., the edge position where the gradient exists within the preset extended box. The calculation of the pixel gradient uses a function upon which the target operator depends, and the obtained extreme value is the extreme value of the function upon which the target operator depends. In this embodiment, the pixel gradient calculation often uses operators such as Sobel and Canny, resulting in binarized contour points, which are convenient for robot storage. Then, morphological denoising is performed on these points, and the coordinate values ​​of the contour points are extracted. Preferably, the pixel gradient calculation uses the Sobel algorithm. Within the preset expansion box, the weighted difference of pixel values ​​among the top, bottom, left, and right neighboring points (i.e., pixels within the neighborhood) of a single pixel is calculated. When the weighted difference reaches its extreme value at the contour point, the contour point can be detected. This determines that the pixel value change in the neighborhood of the obstacle's contour point is the greatest. This achieves the determination of the contour position by calculating gradient changes.

[0067] It's important to note that the outline of an obstacle enclosed by the bounding box is considered as a curve composed of a series of continuous boundary points, representing the shape and structure of the obstacle within the bounding box. These boundary points connect to form the obstacle's outer contour. In machine vision, extracting image contours is fundamental for object detection and shape analysis, but the completeness and accuracy can vary depending on the bounding box's extent. The bounding box defines the boundary between the target and the background; therefore, target operators such as the Sobel operator and the Canny operator are used to detect gradient changes in the image to determine the obstacle's contour location.

[0068] As one embodiment, in step C, the method for extracting contour points of obstacles of the corresponding category based on the binarized image includes: sequentially traversing each contour point detected in step B in the binarized image. As can be seen from the previous embodiment, after edge detection and morphological denoising, the coordinate values ​​of each contour point are sequentially extracted within the target box along the binarized contour or a preset contour extension direction, and recorded as each contour point traversed sequentially. These contour points can be sequentially connected to form one or more closed contour lines. The contour points used to connect the closed contour lines are then retained, and the retained contour points are marked as contour points of the corresponding category of obstacles extracted in step C, including contour points in the inner and outer contours of the same obstacle and contour points in multiple closed contour lines of combined obstacles. In this embodiment, the connecting line of each contour point detected in step B is used to represent the contour of the same category of obstacles, and the shape of the connecting line of each contour point detected in step B is used to describe the shape of the same category of obstacles, providing a path that can be followed along the edge for the robot to avoid obstacles.

[0069] Preferably, in the process of extracting the contour points of obstacles of the corresponding category, the actual contour shape is used to approximate and indicate whether the contour is closed. It is possible to reduce the number of neighboring points in the horizontal, vertical and diagonal directions by calculating the distance from the point to the contour, and retain the endpoints. In fact, it is to use as few points as possible to represent the contour, saving the storage space and computation of contour points without significantly changing the shape of the contour.

[0070] As one embodiment, when the robot executes step C, if the contour points detected in step B are configured to form multiple closed contour lines in sequence, and if one closed contour line is contained within another closed contour line, then the closed contour line is set as a sub-contour of the other closed contour line to form a multi-level contour structure. Within the target box, each closed contour line is used to represent the contour of the same type of obstacle. That is, the same target box does not allow the contours of two or more types of obstacles, but it can be constructed into closed contour lines with multiple structural levels.

[0071] If multiple closed contour lines exist, a multi-level contour structure can be constructed. For example, the containment relationship between contours can be represented by connecting large and small contours. The technical advantages of robots constructing multi-level contour structures are: 1. It can better extract the outer contours of the same obstacle because, during the construction of the multi-level contour structure, each closed contour line is assigned a marker to indicate its position in the multi-level contour structure; 2. It can distinguish between primary and secondary objects. For example, if there is a striped shape of a flower on a shoe, the primary object will be the shoe contour rather than the striped shape of the flower. It is necessary to exclude the interference of secondary objects. Therefore, this application can more accurately calculate the area and perimeter of the primary object for the required contour extraction.

[0072] Based on the above embodiments, in step C, the method for excluding invalid points from the extracted contour points based on an area threshold includes: In step B, when the detected contour points are configured to form multiple closed contour lines, the area enclosed by each closed contour line is calculated and considered as the contour area of ​​each closed contour line. Furthermore, a pre-set area threshold corresponding to the same type of obstacle is set for the area enclosed by each closed contour line; that is, the area enclosed by each closed contour line is compared with the same area threshold. When the type of obstacle to be enclosed changes, a new target bounding box is created, and the area threshold changes accordingly to adapt to the change in the type of obstacle to be enclosed.

[0073] Determine whether the area enclosed by each closed contour line is greater than or equal to the area threshold; eliminate the influence of smaller closed contour lines by comparing with the area threshold.

[0074] When it is determined that the area of ​​a region enclosed by a closed contour line is greater than or equal to the area threshold, all contour points in the currently determined closed contour line are retained in the binarized image, which can be understood as being marked within the target box for subsequent fitting calculation.

[0075] When it is determined that the area enclosed by a closed contour line is less than the area threshold, the currently determined closed contour line is removed. In fact, all contour points that form the currently determined closed contour line are deleted. Each contour point in the currently determined closed contour line is marked as an invalid point. Deleting these invalid points reduces the amount of subsequent fitting calculations.

[0076] To illustrate, an area threshold of 250 is set, and then closed contour lines with excessively small perimeters or areas are filtered out. The filtered-out closed contour lines are typically the outlines of internal decorations within obstacles. For example, if the target box encloses a shoe and the stripes of a decorative flower, the outline of the flower stripes inside the shoe is not relevant to edge-side obstacle avoidance compared to the shoe's overall outline. That is, the robot cannot avoid the shoe as an outer obstacle by following the flower stripe outline. Therefore, the flower stripe outline needs to be removed from the target box. By setting a certain area threshold, excessively small contours are eliminated, while contours that are valuable for the robot's edge-side obstacle avoidance are retained.

[0077] As one embodiment, after processing the image acquired by the robot using the target detection model in step A, if multiple target boxes are obtained in a single frame image, the target box surrounding the obstacle closest to the robot is set as the target box obtained in step A, and the category information of the obstacle closest to the robot surrounded by the target box is obtained in order to obtain the corresponding area threshold. Therefore, in order to obtain the outline of the obstacle closest to the robot, this embodiment uses a conventional monocular ranging algorithm or a binocular ranging algorithm to perform distance calculations on the obstacles surrounded by each target box in the single frame image, then compares the distance calculation results of each obstacle, and then selects only the target box surrounding the obstacle closest to the robot as the target box to be processed in this application. That is, in the single frame image currently acquired by the robot, only the obstacle closest to the robot's frontal field of view is processed, and the category information of the obstacle closest to the robot is obtained for subsequent elimination of excessively small outlines.

[0078] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. References to memory, storage, databases, or other media used in the embodiments provided in this application can all include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable memory (PROM), electrically programmable memory (DPROM), electrically erasable programmable memory (DDPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0079] The above embodiments are merely illustrative of several implementations of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application.

Claims

1. A method for obtaining edge contours for robot walking, characterized in that, Includes the following steps: Step A: Process the images acquired by the robot using the object detection model to obtain the target bounding box and the category information of the obstacles enclosed by the target bounding box, and then proceed to Step B; Step B: Perform image morphology processing on the image acquired by the robot based on the target bounding box to obtain a binarized image, and then proceed to step C; Step C: Extract the contour points of the corresponding category of obstacles based on the binarized image, then exclude invalid points from the extracted contour points based on the area threshold, and then execute step D; where the area threshold is set in advance according to the category information of the obstacles surrounded by the target box. Step D: Within the candidate region defined by the target box that is closest to the robot's walking surface, extract the valid contour points from the contour points after excluding invalid points, and then execute step E. Step E: Perform regression fitting on the effective contour points and introduce R-squared for scoring to obtain a score value; then select the boundary occupied by the candidate region closest to the robot walking surface in the target box or the fitted line generated by the regression fitting as the edge contour based on the score value. In step E, the method of selecting the boundary occupied by the candidate region closest to the robot's walking surface within the target box or the fitted line generated by regression fitting as the edge profile based on the score value includes: Determine whether the score is lower than a pre-set goodness-of-fit threshold; When the score is determined to be lower than the preset fit threshold, the boundary occupied by the candidate region closest to the robot walking surface within the target box is selected as the edge contour; wherein, the boundary occupied by the candidate region closest to the robot walking surface within the target box includes the boundary closest to the robot walking surface within the target box; When the score value is determined to be higher than or equal to the preset fit threshold, the fitted line generated by the regression fitting process is selected as the edge profile.

2. The method for obtaining the edge contour according to claim 1, characterized in that, In step E, the method of performing regression fitting on the effective contour points and introducing R-squared for scoring includes: The effective contour points are subjected to regression fitting to obtain the contour fitting regression model. Subtract the ratio between the residual sum of squares and the total sum of squares from the control value 1, and record the difference as the score. Wherein, the residual sum of squares is the sum of squares of the differences between the model predictions and the actual observed values ​​of the profile-fitted regression model; the total sum of squares is the sum of squares of the differences between the mean of the dependent variable of the profile-fitted regression model and the actual observed values. Among them, the contour fitting regression model is a multinomial fitting regression equation, which makes the contour fitting regression model represent the fitting line generated by the regression fitting process.

3. The method for obtaining the edge contour according to claim 1, characterized in that, In step D, the method for extracting valid contour points from the contour points after excluding invalid points within the candidate region closest to the robot's walking surface defined by the target box includes: The candidate region closest to the robot's walking surface, defined by the target bounding box, is denoted as the image region to be extracted. The image region to be extracted is the region within the target bounding box that is below the reference function line of the target bounding box. The reference function line is located between the upper and lower boundaries of the target bounding box, and the lower boundary of the target bounding box is the boundary within the target bounding box that is closest to the robot's walking surface. Divide the image region to be extracted into a predetermined number of target intervals along a direction parallel to the robot's walking surface; Among the contour points after excluding invalid points, the contour point closest to the lower boundary of each target interval is extracted, and the contour point extracted from the target interval is recorded as a valid contour point, and a preset number of valid contour points are obtained. The lower boundary of a target interval is the boundary of the target interval that is closest to the robot's walking surface.

4. The method for obtaining the edge contour according to claim 3, characterized in that, In step B, the method for image morphology processing of the image acquired by the robot based on the target bounding box specifically includes: The target bounding box is expanded and shrunk, and grayscale conversion, edge detection and morphological denoising are performed on the region defined by the expanded target bounding box; then the expanded target bounding box is shrunk, and the image containing the shrunk target bounding box is set as the binarized image. The target bounding box before expansion is the circumscribed polygon that encloses the outline of an obstacle of a certain type.

5. The method for obtaining the edge contour according to claim 4, characterized in that, In step B, the process of expanding and shrinking the target box includes: Set the direction of the boundary of the target box that is perpendicular to the robot's walking surface as the first preset extension direction, and set the opposite direction of the first preset extension direction and the perpendicular direction of the first preset extension direction as the second preset extension direction. The target box is expanded by a first preset proportion in a first preset expansion direction, and at the same time, the target box is expanded by a second preset proportion in a second preset expansion direction to obtain a preset expansion box, so as to provide pixel information in the neighborhood of the obstacle outline surrounded by the target box. After the target box expands in the first and second preset expansion directions, the contour points of the obstacle in each direction are obtained by edge detection, and a preprocessed image is obtained. Then, the morphological denoising is performed on the preprocessed image. Then, the preset expansion box is controlled to shrink back to the target box to remove the contour points in the area where the target box expands in each direction.

6. The method for obtaining the edge contour according to claim 5, characterized in that, The target box is a rectangle; Methods for expanding the target bounding box include: The direction in which the boundary of the rectangle perpendicularly points to the robot's walking surface is set as the first preset extension direction; the opposite direction of the first preset extension direction is set as the second preset extension direction; the direction in which the boundary of the rectangle parallel to the robot's walking surface extends to one side is set as the second preset extension direction; and the direction in which the boundary of the rectangle parallel to the robot's walking surface extends to the other side is set as the second preset extension direction; wherein, the second preset extension direction is either the second preset extension direction, the second preset extension direction, or the second preset extension direction. During the process of controlling the target box to expand in the first preset expansion direction by a first preset ratio, the boundary of the rectangle that is parallel to the first preset expansion direction is extended along the first preset expansion direction by a first preset ratio; During the process of controlling the target box to expand by a second preset ratio in the second preset expansion direction, the boundary of the rectangle that is parallel to the first preset expansion direction extends by a second preset ratio along the second preset expansion direction; During the process of controlling the target box to expand in the second preset direction by the second preset ratio, the boundary of the rectangle that is parallel to the second preset expansion direction extends along the second preset expansion direction by the second preset ratio; During the process of controlling the target box to expand by a second preset ratio in the second and third preset expansion directions, the boundary of the rectangle that is parallel to the second and third preset expansion directions extends by a second preset ratio along the second and third preset expansion directions; When the target box extends in the first preset extension direction, the second preset extension direction, the second preset extension direction, and the second preset extension direction until the extended boundaries intersect at four boundary points, the preset extension box is obtained. The first preset ratio is greater than the second preset ratio, so as to obtain more contour points in the detection direction close to the ground.

7. The method for obtaining the edge contour according to claim 5, characterized in that, In step B, the method for edge detection of the region defined by the expanded target box includes: Within the preset extended frame, the pixel gradient in the neighborhood of the pixel point is calculated based on the target operator. When an extreme value is obtained during the calculation of the pixel gradient, the contour point of the detected obstacle is determined, and the position where the pixel value change is the largest is determined within the preset extended frame.

8. The method for obtaining the edge contour according to claim 3, characterized in that, In step C, the method for extracting the contour points of obstacles of the corresponding category based on the binarized image includes: In the binarized image, each contour point detected in step B is traversed sequentially, and the contour points used to form a closed contour line are retained. The retained contour points are marked as the contour points of the corresponding category of obstacles extracted in step C. In step B, the lines connecting the various contour points detected are used to represent the contours of obstacles of the same category.

9. The method for obtaining the edge contour according to claim 8, characterized in that, In step B, if each contour point detected is configured to form multiple closed contour lines in sequence, and if one closed contour line is contained inside another closed contour line, then the one closed contour line is set as a sub-contour of the other closed contour line to form a multi-level contour structure. Within the target frame, each closed contour line is used to represent the contour of an obstacle of the same category.

10. The method for obtaining the edge contour according to claim 9, characterized in that, In step C, the method for excluding invalid points from the extracted contour points based on an area threshold includes: Calculate the area of ​​the region enclosed by each closed outline; Determine whether the area enclosed by each closed contour line is greater than or equal to the area threshold; When it is determined that the area of ​​the region enclosed by a closed contour line is greater than or equal to the area threshold, control that each contour point in the currently determined closed contour line is retained in the binarized image. When it is determined that the area enclosed by a closed contour line is less than the area threshold, the currently determined closed contour line is discarded, wherein each contour point in the currently determined closed contour line is marked as an invalid point.

11. The method for obtaining the edge contour according to claim 1, characterized in that, After processing the images acquired by the robot using the target detection model in step A, if multiple target boxes are obtained in a single frame image, the target box surrounding the obstacle closest to the robot is set as the target box obtained in step A, and the category information of the obstacle closest to the robot surrounded by the target box is obtained.

Images