Obstacle detection method for AGV intelligent transfer robot

Abstract

The invention relates to the field of obstacle detection, in particular to an obstacle detection method for an AGV intelligent transfer robot, and the method comprises the steps: obtaining a plurality of gray-scale maps of an advancing path region of the AGV intelligent transfer robot, and calculating the gradient value of a pixel point in each gray-scale map; calculating the edge intensity of pixel points at the same position of the plurality of grey-scale images; calculating a difference characteristic value of pixel points at the same position of the plurality of grey-scale images; taking the pixel points greater than the edge strength threshold as edge points, connecting the edge points to obtain connecting lines, and calculating the consistency of the connecting lines; constructing a target function; and taking the edge intensity threshold value when the target function is the maximum value as the optimal threshold value, and connecting the pixel points with the edge intensity greater than the optimal threshold value to obtain the contour line of the object so as to detect the obstacle. According to the invention, the number of redundant points is effectively reduced, the accuracy of contour line extraction is improved, and obstacles can be conveniently detected.

Description

Technical Field

[0001] The present invention relates to the field of obstacle detection, and in particular to an obstacle detection method for an AGV intelligent transport robot. Background Art

[0002] AGV (Automated Guided Vehicle) intelligent handling robot is a robot that can automatically carry materials and goods in industrial or warehousing environments. AGV robots usually travel automatically within a certain range through preset tracks, lasers, visual navigation and other technologies, and are used in production lines, warehouses, logistics centers and other fields. The visually guided AGV intelligent handling robot uses cameras and computer vision technology to perceive and identify the surrounding environment, allowing it to plan paths based on environmental information without a preset track. Therefore, the AGV intelligent handling robot needs to detect obstacles in the surrounding environment when planning the path, so that it can bypass obstacles when planning the path.

[0003] A Chinese patent application document with publication number CN112581526A discloses an obstacle detection evaluation method, device, equipment and storage medium. The method includes: determining the position information of a first obstacle in an image; obtaining an obstacle detection result output by an algorithm to be evaluated, wherein the obstacle detection result includes at least the position information of a second obstacle; and evaluating the obstacle detection result output by the algorithm to be evaluated using the position information of the first obstacle.

[0004] When detecting obstacles, the existing technical solutions need to extract the contours of each object in the image, and then use the contours of each object to determine whether the corresponding object is an obstacle. When extracting the contours of the object, the Sobel operator is usually used to perform edge detection on the image. In the process of extracting obstacle edge points, the existing Sobel operator will produce redundant points of false detection due to noise or inappropriate threshold setting. The existence of redundant points is not conducive to obstacle detection during the operation of the AGV intelligent handling robot, and affects the path planning of the AGV intelligent handling robot. Summary of the invention

[0005] In order to solve the problem that redundant points appear in the process of extracting obstacle edge points by the Sobel operator and affect obstacle detection, the present invention provides an obstacle detection method for an AGV intelligent transport robot.

[0006] The present invention provides an obstacle detection method for an AGV intelligent handling robot, which adopts the following technical solution: Obtain multiple grayscale images of the AGV intelligent handling robot's forward path area, and calculate the gradient value of the pixel points in each grayscale image; calculate the edge strength of the pixel points at the same position in multiple grayscale images, and the edge strength indicates the possibility that the pixel point at the corresponding position is an edge point; Calculate the difference feature values ​​of the pixels at the same position in multiple grayscale images, where the difference feature values ​​represent the difference in the gradient values ​​of the pixels at the corresponding positions in the multiple grayscale images; Set the edge strength threshold, take the pixel points greater than the edge strength threshold as edge points, connect the edge points to get the connecting lines, calculate the consistency of the connecting lines, and the consistency is positively correlated with the number of edge points of the connecting lines; construct the objective function, the expression is: ; In the formula, represents the objective function about the edge strength threshold, represents the consistency of the jth connecting line, J represents the total number of connecting lines, and exp represents the exponential function with e as the base. Indicates the number of pixels exceeding the edge strength threshold. represents the number of isolated edge points, Represents the difference eigenvalue of the edge point at the i-th position; The edge strength threshold when the objective function is at its maximum value is taken as the optimal threshold, and the pixel points whose edge strength is greater than the optimal threshold are connected to obtain the contour line of the object for obstacle detection.

[0007] By solving the objective function, the edge intensity threshold in the grayscale image is adaptively obtained. Then, the edge point is obtained according to the edge intensity threshold, and the contour line of the object is further obtained, which effectively reduces the number of redundant points, improves the accuracy of contour extraction, and facilitates the detection of obstacles.

[0008] Preferably, the method further comprises calculating the gradient position ratio of each pixel point at each position in the grayscale image, and the expression is:

[0009] In the formula, Indicates the gradient position ratio of the i-th pixel in the p-th grayscale image. Indicates the number of pixels with gradient value a in the pth grayscale image, Represents the gradient value of the pixel at the i-th position in the p-th grayscale image, is the total number of pixels in the pth grayscale image.

[0010] The gradient position ratio can be used to preliminarily determine the possibility that the corresponding pixel point is an edge point. The larger its value is, the higher the position ratio of the gradient value of the corresponding pixel point in the original image is, and the greater the possibility that the pixel point is an edge point.

[0011] Preferably, the method further comprises: A window area is constructed with each pixel point at each position in the grayscale image as the center, the grayscale co-occurrence matrix of the window area is obtained, and the energy value of the grayscale co-occurrence matrix is ​​calculated; Calculate the weight of each pixel in the grayscale image. The expression is:

[0012] In the formula, Represents the weight of the pixel at the i-th position in the p-th grayscale image, Represents the gradient value of the pixel at the i-th position in the p-th grayscale image, represents the mean value of the gradient values ​​of all pixels in the pth grayscale image, Represents the energy value of the gray-level co-occurrence matrix of the window area centered at the i-th pixel in the p-th grayscale image. Represents the mean energy value of the gray-level co-occurrence matrix of the window area centered at the i-th pixel in multiple grayscale images.

[0013] The weights of the pixels are calculated in multiple dimensions, which improves the accuracy of the calculation results and makes it easier to understand the possibility that the corresponding pixels are noise points through the weights.

[0014] Preferably, the expression of edge strength is:

[0015] In the formula, Represents the edge strength value of the pixel at the i-th position, Represents the weight of the pixel at the i-th position in the p-th grayscale image, Represents the gradient position ratio of the i-th pixel in the p-th grayscale image.

[0016] The edge intensity value can be used to understand the possibility that the corresponding pixel point is an edge point, thereby facilitating the distinction between edge points and noise points.

[0017] Preferably, the expression of the difference characteristic value is:

[0018] In the formula, Represents the difference feature value of the pixel at the i-th position, , , They represent the gradient position ratio of the i-th pixel in the p-th, p+1-th, and p+2-th grayscale images respectively.

[0019] The difference eigenvalue can be used to verify whether a pixel is an edge point or a noise point, and thus to evaluate the effect of edge strength threshold in screening edge points.

[0020] Preferably, the expression for the consistency of the connecting line is:

[0021] In the formula, represents the consistency of the jth connection line, Indicates the number of pixels contained in the jth connecting line, It represents the coefficient of variation of the curvature of the pixel point of the j-th connecting line, and exp represents the exponential function with e as the base.

[0022] Preferably, the expression for the consistency of the connecting line is:

[0023] In the formula, represents the consistency of the jth connection line, Indicates the number of pixels contained in the jth connecting line, Represents the curvature variation coefficient of the pixel point of the jth connecting line.

[0024] The higher the consistency value of the connecting line, the better the continuity and smoothness of the edge line, and the higher the possibility that the connecting line is the edge line.

[0025] Preferably, a group optimization intelligent algorithm is used to solve the maximum value of the objective function to obtain the optimal threshold.

[0026] The present invention has the following technical effects: By constructing the objective function from multiple dimensions and solving the objective function, the edge intensity threshold in the grayscale image is adaptively obtained. The edge point is obtained according to the edge intensity threshold, and the contour line of the object is further obtained, which effectively reduces the number of redundant points, improves the accuracy of contour extraction, and facilitates the detection of obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 The present invention is a flowchart of an obstacle detection method for an AGV intelligent transport robot. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0029] The embodiment of the present invention discloses an obstacle detection method for an AGV intelligent handling robot, referring to Figure 1, including the following steps: S1: Obtain multiple grayscale images of the AGV intelligent handling robot's forward path area.

[0030] The high-definition camera installed on the AGV intelligent handling robot is used to collect the original image of the preset forward path area, and the original image is grayed to obtain the first grayscale image. The first grayscale image is zero-filled, and then the first grayscale image is Gaussian filtered and median filtered. The first grayscale image is Gaussian filtered using a 3×3 Gaussian convolution kernel to obtain a second grayscale image, and the first grayscale image is median filtered using a 3×3 filter kernel to obtain a third grayscale image, thereby obtaining the first grayscale image, the second grayscale image, and the third grayscale image. Using Gaussian convolution kernels and median filter convolution kernels to filter images can effectively suppress noise, and the two convolution kernels can suppress different types of noise while better retaining edge details. It can be understood that the sizes of the first grayscale image, the second grayscale image, and the third grayscale image are all the same.

[0031] S2: Calculate the gradient value of the pixel in each grayscale image, and calculate the edge strength of the pixel at the same position in multiple grayscale images. The edge strength indicates the possibility that the pixel at the corresponding position is an edge point.

[0032] S21: Use the Sobel operator to calculate the gradient value of each pixel in the grayscale image. For each grayscale image, calculate the gradient position ratio of each pixel position. The expression is:

[0033] In the formula, Indicates the gradient position ratio of the i-th pixel in the p-th grayscale image. Indicates the number of pixels with gradient value a in the pth grayscale image, Represents the gradient value of the pixel at the i-th position in the p-th grayscale image, is the total number of pixels in the pth grayscale image. The pixel at the i-th position can also be understood as the i-th pixel in the corresponding grayscale image.

[0034] The larger the gradient position of a pixel, the larger the gradient value of the corresponding pixel. If the pixels in the grayscale image are sorted from large to small according to the gradient value, the corresponding pixel is located at the front and is more likely to be an edge point. Conversely, the smaller the gradient position, the lower the gradient value of the corresponding pixel in the corresponding grayscale image, and the pixel is less likely to be an edge point.

[0035] S22: Calculate the weight of each pixel in the grayscale image.

[0036] A window area is constructed with each pixel point at each position in the grayscale image as the center, the grayscale co-occurrence matrix of the window area is obtained, and the energy value of the grayscale co-occurrence matrix is ​​calculated. The energy value is the sum of the squares of the element values ​​of the grayscale co-occurrence matrix, which reflects the uniformity of the grayscale distribution of the image and the coarseness of the texture. If all the values ​​of the grayscale co-occurrence matrix are equal, the energy value is small; on the contrary, if some of the values ​​are large and other values ​​are small, the energy value is large. When the elements in the co-occurrence matrix are concentrated, the energy value is large at this time. The large energy value indicates a more uniform and regularly changing texture pattern. The method for calculating the energy value is a prior art, and the specific steps are not repeated here.

[0037] The expression of the weight of each pixel in the grayscale image is:

[0038] In the formula, Represents the weight of the pixel at the i-th position in the p-th grayscale image, Represents the gradient value of the pixel at the i-th position in the p-th grayscale image, represents the mean value of the gradient values ​​of all pixels in the pth grayscale image, Represents the energy value of the gray-level co-occurrence matrix of the window area centered at the i-th pixel in the p-th grayscale image. Represents the mean energy value of the gray-level co-occurrence matrix of the window area centered at the i-th pixel in multiple grayscale images.

[0039] in, The larger it is, the more obvious the edge feature of the pixel at the i-th position in the corresponding grayscale image is; The closer the value is to 1, the more the local texture features of the pixel at the ith position in the corresponding grayscale image are consistent with other grayscale images. The lower the possibility that the pixel at the ith position is a noise point, the higher its weight.

[0040] S23: Calculate the edge strength of the pixel at the same position of multiple grayscale images. The expression of edge strength is:

[0041] In the formula, Represents the edge strength value of the pixel at the i-th position, Represents the weight of the pixel at the i-th position in the p-th grayscale image, Indicates the gradient position ratio of the i-th pixel in the p-th grayscale image.

[0042] is the weight of the pixel at the ith position in the pth grayscale image. The larger the weight, the larger the ratio of the gradient value to the mean value of the pixel at the ith position in the pth grayscale image, and its local texture feature in the pth grayscale image is consistent with that in other grayscale images, indicating that the edge strength of the pixel at the ith position is higher and the possibility of being a noise point is lower.

[0043] is the gradient position ratio of the pixel at the ith position in the pth image. The larger its value is, the greater the possibility that the corresponding pixel is an edge point.

[0044] In summary, the edge intensity value indicates the possibility that the pixel point at the corresponding position is an edge pixel point. The larger the value, the greater the possibility that the pixel point at the corresponding position is an edge pixel point.

[0045] S3: Calculate the difference eigenvalues ​​of the pixels at the same position in the multiple grayscale images, where the difference eigenvalues ​​represent the difference in the gradient values ​​of the pixels at the corresponding positions in the multiple grayscale images.

[0046] The expression of the difference eigenvalue is:

[0047] In the formula, Represents the difference feature value of the pixel at the i-th position, , , They represent the gradient position ratio of the i-th pixel in the p-th, p+1-th, and p+2-th grayscale images respectively.

[0048] Exemplarily, for the first grayscale image, the second grayscale image, and the third grayscale image, the expression of the difference characteristic value of the pixel point at the i-th position is:

[0049] In the above formula is the difference between the gradient position ratio of the pixel at the i-th position in the second grayscale image after Gaussian filtering and the first grayscale image, is the difference between the gradient position ratio of the pixel at the ith position in the third grayscale image after median filtering and the first grayscale image. The larger the difference characteristic value of the pixel at the ith position, the greater the difference between the gradient value of the pixel at the ith position in the filtered image and the original image, and the greater the possibility that it is a noise point.

[0050] S4: Setting an edge strength threshold, taking pixel points greater than the edge strength threshold as edge points, connecting the edge points to obtain connecting lines, and calculating the consistency of the connecting lines. The consistency is positively correlated with the number of edge points of the connecting lines.

[0051] In any grayscale image, pixels with a strength greater than the edge intensity threshold are regarded as edge points, and the edge points are connected to obtain connecting lines.

[0052] In one embodiment, the expression for the consistency of the connecting line is:

[0053] In the formula, represents the consistency of the jth connection line, Indicates the number of pixels contained in the jth connecting line, represents the curvature variation coefficient of the pixel point of the jth connecting line, and exp represents an exponential function with e as the base. The calculation method of the variation coefficient is a prior art, and the specific calculation steps are not repeated here.

[0054] In one embodiment, the expression for the consistency of the connecting line is:

[0055] In the formula, represents the consistency of the jth connection line, Indicates the number of pixels contained in the jth connecting line, Represents the curvature variation coefficient of the pixel point of the jth connecting line.

[0056] The larger the value of indicates, the better the continuity of the connecting line is, and the greater the possibility that the connecting line is the edge line of the object. The smaller the value of indicates that the edge line is smoother, the lower its mutation is, and the greater the possibility that the connecting line is the edge line of the object. In summary, the greater the consistency of the connecting line, the greater the possibility that the connecting line is the edge line of the object.

[0057] S5: Construct the objective function.

[0058] The expression of the objective function is: ; In the formula, represents the objective function about the edge strength threshold, represents the consistency of the jth connecting line, J represents the total number of connecting lines, and exp represents the exponential function with e as the base. Indicates the number of pixels exceeding the edge strength threshold. represents the number of isolated edge points, Represents the difference eigenvalue of the edge point at the ith position; an isolated edge point is a pixel point that exceeds the edge strength threshold, and the edge strength of the pixels in the eight neighborhoods of the pixel point are all less than the edge strength threshold. Exemplarily, the edge strength threshold of the pixel point at the qth position is greater than the edge strength threshold, and the edge strength threshold of the pixels in the eight neighborhoods of the pixel point at the qth position is less than the edge strength threshold, then the pixel point at the qth position is an isolated edge point.

[0059] In the formula It is the sum of the difference eigenvalues ​​of all edge points filtered by the edge strength threshold T. The larger the difference eigenvalue of the edge point, the greater the difference in the gradient value of the edge point in the filtered grayscale image compared with the original grayscale image, and the greater the possibility that it is a noise point, not a real edge point.

[0060] It is the sum of the consistencies of the connecting lines formed by the edge points selected by the edge strength threshold T. The higher the consistency of the connecting lines, the better the continuity and smoothness of the connecting lines, and the closer they are to the real edge lines.

[0061] It is the ratio of isolated edge points to all edge points among the edge points screened by the edge strength threshold T. The more isolated edge points there are, the more discontinuous the edge points will be due to inappropriate threshold selection.

[0062] S6: Connect the pixel points whose edge strength is greater than the optimal threshold to obtain the contour line of the object for obstacle detection.

[0063] The group optimization intelligent algorithm is used to solve the maximum value of the objective function, and the edge strength threshold when the objective function is at the maximum value is taken as the optimal threshold. In any grayscale image, the pixel points with edge strength greater than the optimal threshold are connected to obtain the contour line of the object. The grayscale image containing the contour line is input into the preset convolutional neural network model to determine whether there are obstacles in the forward path area.

Claims

1. An obstacle detection method for an AGV intelligent handling robot, characterized in that: The method comprises the following steps: obtaining multiple grayscale images of the forward path area of ​​the AGV intelligent handling robot, and calculating the gradient value of the pixel points in each grayscale image; Calculate the edge strength of the pixel at the same position of multiple grayscale images. The edge strength indicates the possibility that the pixel at the corresponding position is an edge point. Calculate the difference feature values ​​of the pixels at the same position in multiple grayscale images, where the difference feature values ​​represent the difference in the gradient values ​​of the pixels at the corresponding positions in the multiple grayscale images; Set the edge strength threshold, take the pixel points greater than the edge strength threshold as edge points, connect the edge points to get the connecting lines, calculate the consistency of the connecting lines, and the consistency is positively correlated with the number of edge points of the connecting lines; construct the objective function, the expression is: ; In the formula, represents the objective function about the edge strength threshold, represents the consistency of the jth connecting line, J represents the total number of connecting lines, and exp represents the exponential function with e as the base. Indicates the number of pixels exceeding the edge strength threshold. represents the number of isolated edge points, Represents the difference eigenvalue of the edge point at the i-th position; The edge strength threshold when the objective function is at its maximum value is taken as the optimal threshold, and the pixel points whose edge strength is greater than the optimal threshold are connected to obtain the contour line of the object for obstacle detection.

2. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The method also includes calculating the gradient position ratio of each pixel point in the grayscale image, and the expression is: In the formula, Indicates the gradient position ratio of the i-th pixel in the p-th grayscale image. Indicates the number of pixels with gradient value a in the pth grayscale image, Represents the gradient value of the pixel at the i-th position in the p-th grayscale image, is the total number of pixels in the pth grayscale image.

3. The obstacle detection method for an AGV intelligent handling robot according to claim 2, characterized in that: The method also includes: A window area is constructed with each pixel point at each position in the grayscale image as the center, the grayscale co-occurrence matrix of the window area is obtained, and the energy value of the grayscale co-occurrence matrix is ​​calculated; Calculate the weight of each pixel in the grayscale image. The expression is: In the formula, Represents the weight of the pixel at the i-th position in the p-th grayscale image, Represents the gradient value of the pixel at the i-th position in the p-th grayscale image, represents the mean value of the gradient values ​​of all pixels in the pth grayscale image, Represents the energy value of the gray-level co-occurrence matrix of the window area centered at the i-th pixel in the p-th grayscale image. Represents the mean energy value of the gray-level co-occurrence matrix of the window area centered at the i-th pixel in multiple grayscale images.

4. The obstacle detection method for an AGV intelligent handling robot according to claim 3, characterized in that: The expression of edge strength is: In the formula, Represents the edge strength value of the pixel at the i-th position, Represents the weight of the pixel at the i-th position in the p-th grayscale image, Represents the gradient position ratio of the i-th pixel in the p-th grayscale image.

5. The obstacle detection method for an AGV intelligent handling robot according to claim 2, characterized in that: The expression of the difference eigenvalue is: In the formula, Represents the difference feature value of the pixel at the i-th position, , , They represent the gradient position ratio of the i-th pixel in the p-th, p+1-th, and p+2-th grayscale images respectively.

6. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The expression for the consistency of the connecting line is: In the formula, represents the consistency of the jth connection line, Indicates the number of pixels contained in the jth connecting line, It represents the coefficient of variation of the curvature of the pixel point of the j-th connecting line, and exp represents the exponential function with e as the base.

7. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The expression for the consistency of the connecting line is: In the formula, represents the consistency of the jth connection line, Indicates the number of pixels contained in the jth connecting line, Represents the curvature variation coefficient of the pixel point of the jth connecting line.

8. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The group optimization intelligent algorithm is used to solve the maximum value of the objective function and obtain the optimal threshold.

9. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The plurality of grayscale images include a first grayscale image, a second grayscale image and a third grayscale image. The first grayscale image is convolved with a Gaussian convolution kernel to obtain the second grayscale image, and the first grayscale image is convolved with a median filter convolution kernel to obtain the third grayscale image.

10. The obstacle detection method for an AGV intelligent handling robot according to claim 1, characterized in that: The Sobel operator is used to calculate the gradient value of each pixel in the grayscale image.

Images