An obstacle detection method for an AGV intelligent handling robot

Abstract

The present invention relates to the field of obstacle detection, and particularly to an obstacle detection method for an AGV intelligent handling robot. The method includes: obtaining multiple grayscale images of the forward path area of the AGV intelligent handling robot, and calculating the gradient values of the pixel points in each grayscale image; calculating the edge intensity of the pixel points at the same position in the multiple grayscale images; calculating the difference feature values of the pixel points at the same position in the multiple grayscale images; taking the pixel points greater than the edge intensity threshold as edge points, connecting the edge points to obtain a connection line, and calculating the consistency of the connection line; constructing an objective function; taking the edge intensity threshold when the objective function is at its maximum value as the optimal threshold, and connecting the pixel points with an edge intensity greater than the optimal threshold to obtain the contour line of the object for detecting obstacles. The present invention effectively reduces the number of redundant points, improves the accuracy of extracting the contour line, and facilitates the detection of obstacles.

Description

Technical Field

[0001] The present invention relates to the field of obstacle detection, and particularly to an obstacle detection method for an AGV intelligent handling robot. Background Art

[0002] An AGV (Automated Guided Vehicle) intelligent handling robot is a robot that can automatically handle materials and goods in an industrial or warehousing environment. AGV robots usually automatically travel within a certain range through technologies such as preset trajectories, lasers, and visual navigation, and are applied in fields such as production lines, warehouses, and logistics centers. A vision-guided AGV intelligent handling robot uses cameras and computer vision technologies to perceive and recognize the surrounding environment, enabling it to perform path planning based on environmental information without a preset trajectory. Therefore, when an AGV intelligent handling robot performs path planning, it needs to detect obstacles in the surrounding environment so that it can bypass obstacles when planning the path.

[0003] The Chinese patent application document with the publication number CN112581526A discloses an evaluation method, device, equipment, and storage medium for obstacle detection. The method includes: determining the position information of the first obstacle in the image; obtaining the obstacle detection result output by the algorithm to be evaluated, where the obstacle detection result at least includes the position information of the second obstacle; and using the position information of the first obstacle to evaluate the obstacle detection result output by the algorithm to be evaluated.

[0004] In the existing technical solutions for obstacle detection, it is necessary to extract the contours of each object in the image, and then use the contours of each object to determine whether the corresponding object is an obstacle. When extracting the contours of an object, the sobel operator is usually used to perform edge detection on the image. In the process of extracting the edge points of an obstacle by the existing sobel operator, due to noise or inappropriate threshold setting, redundant points of false detection will appear. The existence of redundant points is not conducive to obstacle detection during the operation of an AGV intelligent handling robot, and affects the path planning of the AGV intelligent handling robot. Summary of the Invention

[0005] In order to solve the problem that redundant points appear during the process of extracting obstacle edge points by the sobel operator, affecting obstacle detection, the present invention provides an obstacle detection method for an AGV intelligent handling robot.

[0006] The present invention provides an obstacle detection method for an AGV intelligent handling robot, adopting the following technical solution:

[0007] Obtain multiple grayscale images of the forward path area of the AGV intelligent handling robot, and calculate the gradient values of the pixel points in each grayscale image; calculate the edge intensity of the pixel points at the same position in multiple grayscale images, and the edge intensity represents the possibility that the pixel points at the corresponding positions are edge points;

[0008] Calculate the difference feature values of the pixel points at the same position in multiple grayscale images, and the difference feature values represent the difference of the gradient values of the pixel points at the corresponding positions in multiple grayscale images;

[0009] Set the edge intensity threshold, regard the pixel points greater than the edge intensity threshold as edge points, connect the edge points to obtain connection lines, and calculate the consistency of the connection lines. The consistency is positively correlated with the number of edge points of the connection lines; construct an objective function, and the expression is: ;

[0010] In the formula, represents the objective function with respect to the edge intensity threshold, represents the consistency of the j-th connection line, J represents the total number of connection lines, exp represents the exponential function with e as the base, represents the number of pixel points exceeding the edge intensity threshold, represents the number of isolated edge points, represents the difference feature value of the edge point at the i-th position;

[0011] Take the edge intensity threshold when the objective function is the maximum value as the optimal threshold, and connect the pixel points with edge intensity greater than the optimal threshold to obtain the contour line of the object for obstacle detection.

[0012] By solving the objective function, the edge intensity threshold in the grayscale image is adaptively obtained. Then, according to the edge intensity threshold, edge points are obtained, and further the contour line of the object is obtained, effectively reducing the number of redundant points, improving the accuracy of extracting the contour line, and facilitating obstacle detection.

[0013] Preferably, the method further includes calculating the gradient position ratio of each pixel point in the grayscale image, and the expression is:

[0014]

[0015] In the formula, represents the gradient position ratio of the pixel point at the i-th position in the p-th grayscale image, represents the number of pixel points with a gradient value of a in the p-th grayscale image, represents the gradient value of the pixel point at the i-th position in the p-th grayscale image, is the total number of pixel points in the p-th grayscale image.

[0016] The possibility that the corresponding pixel point is an edge point can be initially judged by the gradient position ratio. The larger the value, the higher the position ratio of the gradient value of the corresponding pixel point in the original image, and the greater the possibility that the pixel point is an edge point.

[0017] Preferably, the method further includes:

[0018] Construct a window area centered on each pixel point in the grayscale image, obtain the gray-level co-occurrence matrix of the window area, and calculate the energy value of the gray-level co-occurrence matrix;

[0019] Calculate the weight of each pixel point in the grayscale image, and the expression is:

[0020]

[0021] In the formula, represents the weight of the i-th pixel point at the p-th grayscale image, represents the gradient value of the i-th pixel point at the p-th grayscale image, represents the average value of the gradient values of all pixel points at the p-th grayscale image, represents the energy value of the gray-level co-occurrence matrix of the window area centered on the i-th pixel point at the p-th grayscale image, represents the average value of the energy values of the gray-level co-occurrence matrices of the window areas centered on the i-th pixel point in multiple grayscale images.

[0022] Calculating the weight of the pixel point through multiple dimensions improves the accuracy of the calculation result and facilitates understanding the possibility that the corresponding pixel point belongs to a noise point through the weight.

[0023] Preferably, the expression of the edge intensity is:

[0024]

[0025] In the formula, represents the edge intensity value of the i-th pixel point, represents the weight of the i-th pixel point at the p-th grayscale image, represents the gradient position ratio of the i-th pixel point at the p-th grayscale image.

[0026] The possibility that the corresponding pixel point is an edge point can be understood through the edge intensity value, thus facilitating the distinction between edge points and noise points.

[0027] Preferably, the expression of the difference eigenvalue is:

[0028]

[0029] In the formula, denotes the difference eigenvalue of the pixel at the $i$-th position, , , respectively denote the gradient position ratios of the pixel at the $i$-th position in the $p$-th, $(p + 1)$-th, and $(p + 2)$-th grayscale images.

[0030] The difference eigenvalue can be used to verify whether a pixel is an edge point or a noise point, so as to evaluate the effect of screening edge points by the edge intensity threshold.

[0031] Preferably, the expression for the consistency of the connection line is:

[0032]

[0033] In the formula, denotes the consistency of the $j$-th connection line, denotes the number of pixels included in the $j$-th connection line, denotes the coefficient of variation of the curvature of the pixels of the $j$-th connection line, and exp represents the exponential function with base $e$.

[0034] Preferably, the expression for the consistency of the connection line is:

[0035]

[0036] In the formula, denotes the consistency of the $j$-th connection line, denotes the number of pixels included in the $j$-th connection line, denotes the coefficient of variation of the curvature of the pixels of the $j$-th connection line.

[0037] The higher the consistency value of the connection line, the better the continuity and smoothness of the edge line, and the higher the possibility that the connection line is an edge line.

[0038] Preferably, the group optimization intelligent algorithm is used to solve the maximum value of the objective function to obtain the optimal threshold.

[0039] The present invention has the following technical effects:

[0040] By constructing the objective function from multiple dimensions, solving the objective function to adaptively obtain the edge intensity threshold in the grayscale image, obtaining edge points according to the edge intensity threshold, and further obtaining the contour line of the object, the number of redundant points is effectively reduced, the accuracy of extracting the contour line is improved, and it is convenient to detect obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flowchart of a method for obstacle detection of an AGV intelligent handling robot according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0043] An obstacle detection method for an AGV intelligent handling robot is disclosed in an embodiment of the present invention. Referring to Figure 1 , the method includes the following steps:

[0044] S1: Obtain multiple grayscale images of the forward path area of the AGV intelligent handling robot.

[0045] Use the high-definition camera installed on the AGV intelligent handling robot to collect the original image of the preset forward path area, perform grayscale processing on the original image to obtain the first grayscale image, perform zero-padding on the first grayscale image, and then perform Gaussian filtering and median filtering on the first grayscale image. Use a 3×3 Gaussian convolution kernel to perform Gaussian filtering on the first grayscale image to obtain the second grayscale image, and use a 3×3 filter kernel to perform median filtering on the first grayscale image to obtain the third grayscale image. Thus, the first grayscale image, the second grayscale image, and the third grayscale image are obtained. Filtering the image with a Gaussian convolution kernel and a median filtering convolution kernel can effectively suppress noise, and the two convolution kernels can suppress different types of noise and better retain edge details at the same time. It can be understood from this that the sizes of the first grayscale image, the second grayscale image, and the third grayscale image are the same.

[0046] S2: Calculate the gradient value of each pixel point in each grayscale image, and calculate the edge intensity of the pixel points at the same position in multiple grayscale images. The edge intensity represents the possibility that the pixel point at the corresponding position is an edge point.

[0047] S21: Use the sobel operator to calculate the gradient value of each pixel point in each grayscale image. For each grayscale image, calculate the gradient position ratio of each pixel point at each position. The expression is:

[0048]

[0049] In the formula, represents the gradient position ratio of the pixel point at the i-th position in the p-th grayscale image, represents the number of pixel points with a gradient value of a in the p-th grayscale image, represents the gradient value of the pixel point at the i-th position in the p-th grayscale image, is the total number of pixel points in the p-th grayscale image. Among them, the pixel point at the i-th position can also be understood as the i-th pixel point in the corresponding grayscale image.

[0050] The larger the gradient position of a pixel point, the larger the gradient value of the corresponding pixel point. If the pixel points in the grayscale image are sorted in descending order of gradient value, then the positions of the corresponding pixel points are relatively forward, and they are more likely to be edge points; conversely, the smaller the gradient position, the lower the gradient value of the corresponding pixel point in the corresponding grayscale image, and the less likely the pixel point is to be an edge point.

[0051] S22: Calculate the weights of the pixel points at each position in the grayscale image.

[0052] Construct a window region centered on the pixel point at each position in the grayscale image, obtain the gray-level co-occurrence matrix of the window region, and calculate the energy value of the gray-level co-occurrence matrix. The energy value is the sum of the squares of the element values of the gray-level co-occurrence matrix, which reflects the uniformity of the image gray-scale distribution and the coarseness of the texture. If all the values of the gray-level co-occurrence matrix are equal, the energy value is small; on the contrary, if some of the values are large and others are small, the energy value is large. When the elements in the co-occurrence matrix are concentratedly distributed, the energy value is large at this time. A large energy value indicates a relatively uniform and regularly changing texture pattern. The calculation method of the energy value is a prior art, and the specific steps are not elaborated here.

[0053] The expression for the weight of the pixel point at each position in the grayscale image is:

[0054]

[0055] In the formula, represents the weight of the pixel point at the i-th position in the p-th grayscale image, represents the gradient value of the pixel point at the i-th position in the p-th grayscale image, represents the average value of the gradient values of all pixel points in the p-th grayscale image, represents the energy value of the gray-level co-occurrence matrix of the window region centered on the pixel point at the i-th position in the p-th grayscale image, represents the average value of the energy values of the gray-level co-occurrence matrices of the window regions centered on the pixel point at the i-th position in multiple grayscale images.

[0056] Among them, The larger it is, the more obvious the edge feature of the pixel point at the i-th position in the corresponding grayscale image; The value closer to 1 indicates that the local texture feature of the pixel point at the i-th position in the corresponding grayscale image is consistent with that of other grayscale images, and the lower the possibility that the pixel point at the i-th position is a noise point, and the higher its weight.

[0057] S23: Calculate the edge intensity of the pixel points at the same position in multiple grayscale images,

[0058] The expression for the edge intensity is:

[0059]

[0060] In the formula, represents the edge intensity value of the pixel at the i-th position, represents the weight of the pixel at the i-th position in the p-th grayscale image, represents the gradient position ratio of the pixel at the i-th position in the p-th grayscale image.

[0061] is the weight of the pixel at the i-th position in the p-th grayscale image. The larger the weight, the greater the ratio of the gradient value of the pixel at the i-th position to the mean value in the p-th grayscale image, and the more consistent its local texture features in the p-th grayscale image with those in other grayscale images, indicating that the edge intensity of the pixel at the i-th position is higher and the probability of it being a noise point is lower.

[0062] is the gradient position ratio of the pixel at the i-th position in the p-th image. The larger its value, the greater the probability that the corresponding pixel is an edge point.

[0063] In summary, the edge intensity value represents the probability that the pixel at the corresponding position is an edge pixel. The larger the value, the greater the probability that the pixel at the corresponding position is an edge pixel.

[0064] S3: Calculate the difference feature value of the pixels at the same position in multiple grayscale images. The difference feature value represents the difference situation of the gradient values of the pixels at the corresponding position in multiple grayscale images.

[0065] The expression of the difference feature value is:

[0066]

[0067] In the formula, represents the difference feature value of the pixel at the i-th position, , , respectively represent the gradient position ratios of the pixel at the i-th position in the p-th, (p + 1)-th, and (p + 2)-th grayscale images.

[0068] Exemplarily, for the first grayscale image, the second grayscale image, and the third grayscale image, the expression of the difference feature value of the pixel at the i-th position is:

[0069]

[0070] In the above formula is the difference between the gradient position ratio of the pixel at the i-th position in the second grayscale image after Gaussian filtering and the first grayscale image, The difference between the gradient position ratio of the pixel at the $i$-th position in the third grayscale image after median filtering and the first grayscale image. The larger the difference eigenvalue of the pixel at the $i$-th position, the greater the difference in the gradient value of the pixel at the $i$-th position in the filtered image compared to the original image, and the greater the likelihood of it being a noise point.

[0071] S4: Set the edge intensity threshold, take the pixels greater than the edge intensity threshold as edge points, connect the edge points to obtain a connection line, and calculate the consistency of the connection line. The consistency is positively correlated with the number of edge points on the connection line.

[0072] In any grayscale image, take the pixels greater than the edge intensity threshold as edge points, and connect the edge points to obtain a connection line.

[0073] In one embodiment, the expression for the consistency of the connection line is:

[0074]

[0075] In the formula, represents the consistency of the $j$-th connection line, represents the number of pixels included in the $j$-th connection line, represents the coefficient of variation of the curvature of the pixels on the $j$-th connection line, and exp represents the exponential function with base $e$. The calculation method of the coefficient of variation is prior art, and the specific calculation steps are not elaborated here.

[0076] In one embodiment, the expression for the consistency of the connection line is:

[0077]

[0078] In the formula, represents the consistency of the $j$-th connection line, represents the number of pixels included in the $j$-th connection line, represents the coefficient of variation of the curvature of the pixels on the $j$-th connection line.

[0079] The larger the value of , the better the continuity of the connection line, and the greater the likelihood that the connection line is the edge line of the object. The smaller the value of

[0080] S5: Construct the objective function.

[0081] The expression of the objective function is: ;

[0082] In the formula, represents the objective function with respect to the edge strength threshold, represents the consistency of the j-th connection line, J represents the total number of connection lines, and exp represents the exponential function with base e, represents the number of pixel points exceeding the edge strength threshold, represents the number of isolated edge points, represents the differential feature value of the edge point at the i-th position; an isolated edge point means that the edge strengths of the pixel points within the eight-neighborhood range of the pixel point exceeding the edge strength threshold are all less than the edge strength threshold. Exemplarily, if the edge strength threshold of the pixel point at the q-th position is greater than the edge strength threshold, and the edge strength thresholds of the pixel points within the eight-neighborhood range of the pixel point at the q-th position are less than the edge strength threshold, then the pixel point at the q-th position is an isolated edge point.

[0083] In the formula is the sum of the differential feature values of all edge points screened by the edge strength threshold T. The larger the differential feature value of the edge point, the greater the difference in the gradient value of the edge point in the filtered grayscale image compared to the original grayscale image, and the greater the possibility of being a noise point and not a real edge point.

[0084] is the total sum of the consistencies of the connection lines formed by the edge points screened by the edge strength threshold T. The higher the consistency of the connection line, the better the continuity and smoothness of the connection line, and the closer it is to the real edge line.

[0085] is the proportion of isolated edge points among all edge points screened by the edge strength threshold T. The more isolated edge points indicate that the threshold selection is inappropriate, resulting in discontinuous edge points.

[0086] S6: Connect the pixel points with edge strength greater than the optimal threshold to obtain the contour line of the object for obstacle detection.

[0087] Use the group-optimal intelligent algorithm to solve the maximum value of the objective function, and take the edge strength threshold when the objective function is at its maximum as the optimal threshold. In any grayscale image, connect the pixel points with edge strength greater than the optimal threshold to obtain the contour line of the object, and input the grayscale image containing the contour line into a preset convolutional neural network model to determine whether there are obstacles in the forward path area.

Claims

1. An obstacle detection method for an AGV intelligent handling robot, characterized in that: The method comprises the following steps: obtaining multiple grayscale images of the forward path area of ​​the AGV intelligent handling robot, and calculating the gradient value of the pixel points in each grayscale image; Calculate the edge strength of the pixel at the same position of multiple grayscale images. The edge strength indicates the possibility that the pixel at the corresponding position is an edge point. Calculate the gradient position ratio of each pixel in the grayscale image. The expression is: In the formula, Indicates the gradient position ratio of the i-th pixel in the p-th grayscale image. Indicates the number of pixels with gradient value a in the pth grayscale image, Represents the gradient value of the pixel at the i-th position in the p-th grayscale image, is the total number of pixels in the pth grayscale image; Calculate the difference eigenvalues ​​of pixels at the same position in multiple grayscale images. The difference eigenvalues ​​represent the difference in the gradient values ​​of pixels at the corresponding positions in multiple grayscale images. The expression is: In the formula, Represents the difference feature value of the pixel at the i-th position, , , Respectively represent the gradient position ratio of the pixel at the i-th position in the p-th, p+1-th, and p+2-th grayscale images; Set the edge strength threshold, take the pixel points greater than the edge strength threshold as edge points, connect the edge points to get the connecting lines, calculate the consistency of the connecting lines, and the consistency is positively correlated with the number of edge points of the connecting lines; construct the objective function, the expression is: ; In the formula, represents the objective function about the edge strength threshold, represents the consistency of the jth connecting line, J represents the total number of connecting lines, and exp represents the exponential function with e as the base. Indicates the number of pixels exceeding the edge strength threshold. represents the number of isolated edge points, Represents the difference eigenvalue of the edge point at the i-th position; The edge strength threshold when the objective function is at its maximum value is taken as the optimal threshold, and the pixel points whose edge strength is greater than the optimal threshold are connected to obtain the contour line of the object for obstacle detection.

2. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The method also includes: A window area is constructed with each pixel point at each position in the grayscale image as the center, the grayscale co-occurrence matrix of the window area is obtained, and the energy value of the grayscale co-occurrence matrix is ​​calculated; Calculate the weight of each pixel in the grayscale image. The expression is: In the formula, Represents the weight of the pixel at the i-th position in the p-th grayscale image, Represents the gradient value of the pixel at the i-th position in the p-th grayscale image, represents the mean value of the gradient values ​​of all pixels in the pth grayscale image, Represents the energy value of the gray-level co-occurrence matrix of the window area centered at the i-th pixel in the p-th grayscale image. Represents the mean energy value of the gray-level co-occurrence matrix of the window area centered at the i-th pixel in multiple gray-level images. represents the natural exponential function.

3. The obstacle detection method for an AGV intelligent handling robot according to claim 2, characterized in that: The expression of edge strength is: In the formula, Represents the edge strength value of the pixel at the i-th position, Represents the weight of the pixel at the i-th position in the p-th grayscale image, Represents the gradient position ratio of the i-th pixel in the p-th grayscale image.

4. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The expression for the consistency of the connecting line is: In the formula, represents the consistency of the jth connection line, Indicates the number of pixels contained in the jth connecting line, It represents the coefficient of variation of the curvature of the pixel point of the j-th connecting line, and exp represents the exponential function with e as the base.

5. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The expression for the consistency of the connecting line is: In the formula, represents the consistency of the jth connection line, Indicates the number of pixels contained in the jth connecting line, Represents the curvature variation coefficient of the pixel point of the jth connecting line.

6. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The group optimization intelligent algorithm is used to solve the maximum value of the objective function and obtain the optimal threshold.

7. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The plurality of grayscale images include a first grayscale image, a second grayscale image and a third grayscale image. The first grayscale image is convolved with a Gaussian convolution kernel to obtain the second grayscale image, and the first grayscale image is convolved with a median filter convolution kernel to obtain the third grayscale image.

8. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The Sobel operator is used to calculate the gradient value of each pixel in the grayscale image.

Images