Ground Detection Method, Device, Electronic Device and Storage Medium

Abstract

This application is applicable to the field of computer technology, and provides a ground detection method, device, electronic device, and storage medium, including: obtaining first point cloud data corresponding to a preset detection area; normalizing the height information of the first point cloud data into an image space to obtain a first grayscale image; performing a filtering process on the first grayscale image to obtain a second grayscale image; determining second point cloud data according to the second grayscale image; determining a target ground image area in the second grayscale image according to the second grayscale image and a preset threshold segmentation algorithm; and positioning a passable ground area according to the target ground image area and the second point cloud data. The embodiments of this application can efficiently and accurately implement ground detection.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and particularly relates to a ground detection method, device, electronic device, and storage medium. Background Art

[0002] With the development of robot technology, robots can achieve autonomous movement functions through computer vision technology and positioning and navigation technology. When a robot realizes its autonomous movement function, it needs to be able to perceive the surrounding environment and autonomously detect passable ground areas for ensuring the safety of the robot's movement. It can be seen that ground detection is a key link for a robot to realize its autonomous movement function. However, the current ground detection methods have the defects of high hardware costs and low accuracy. Summary of the Invention

[0003] In view of this, embodiments of this application provide a ground detection method, device, electronic device, and storage medium to solve the problem of how to accurately implement ground detection in the prior art.

[0004] The first aspect of the embodiments of this application provides a ground detection method, including:

[0005] Obtaining first point cloud data corresponding to a preset detection area;

[0006] Normalizing the height information of the first point cloud data to the image space to obtain a first grayscale image;

[0007] Performing filtering processing on the first grayscale image to obtain a second grayscale image;

[0008] Determining second point cloud data according to the second grayscale image;

[0009] Determining a target ground image area in the second grayscale image according to the second grayscale image and a preset threshold segmentation algorithm;

[0010] Locating a passable ground area according to the target ground image area and the second point cloud data.

[0011] Optionally, the obtaining first point cloud data corresponding to a preset detection area includes:

[0012] Obtaining a depth image corresponding to the preset detection area;

[0013] Determining point cloud data in the camera coordinate system according to the depth image;

[0014] Determining point cloud data in the robot coordinate system as the first point cloud data according to the point cloud data in the camera coordinate system and a preset coordinate conversion relationship.

[0015] Optionally, the normalization of the height information of the first point cloud data to the image space to obtain a first grayscale image includes:

[0016] Converting the height information of the first point cloud data to a preset height interval to obtain target height data;

[0017] Normalizing the target height data to a preset grayscale interval to obtain a first grayscale image.

[0018] Optionally, the filtering process of the first grayscale image to obtain a second grayscale image includes:

[0019] Performing median filtering and dimensionality reduction and cropping on the first grayscale image to obtain the second grayscale image.

[0020] Optionally, the determination of the target ground image area in the second grayscale image according to the second grayscale image and a preset threshold segmentation algorithm includes:

[0021] Calculating the gradient of the second grayscale image to determine the target gradient image corresponding to the second grayscale image;

[0022] Performing threshold segmentation on the target gradient image according to a first grayscale threshold to determine a set of first pixel points in the target gradient image whose grayscale values are less than the first grayscale threshold;

[0023] Performing threshold segmentation on the second grayscale image according to a target threshold interval to determine a set of second pixel points in the second grayscale image whose grayscale values are within the target threshold interval;

[0024] Determining the target ground image area according to the set of first pixel points and the set of second pixel points.

[0025] Optionally, the determination of the target ground image area according to the set of first pixel points and the set of second pixel points includes:

[0026] Determining a target contour according to the set of first pixel points and the set of second pixel points;

[0027] If the area of the target contour is greater than a preset area threshold, determining the target ground image area according to the target contour.

[0028] Optionally, the determination of the target ground image area according to the target contour includes:

[0029] Determining the centroid of the target contour;

[0030] Determining an optimal seed point according to the centroid;

[0031] Based on the optimal seed points, the second grayscale image is processed by the flood filling algorithm to obtain the target ground image area.

[0032] The second aspect of the embodiments of the present application provides a ground detection device, including:

[0033] A first point cloud data acquisition unit, configured to acquire first point cloud data corresponding to a preset detection area;

[0034] A first grayscale image determination unit, configured to normalize the height information of the first point cloud data into the image space to obtain a first grayscale image;

[0035] A second grayscale image determination unit, configured to perform filtering processing on the first grayscale image to obtain a second grayscale image;

[0036] A second point cloud data determination unit, configured to determine second point cloud data according to the second grayscale image;

[0037] A target ground image area determination unit, configured to determine the target ground image area in the second grayscale image according to the second grayscale image and a preset threshold segmentation algorithm;

[0038] A positioning unit, configured to locate the passable ground area according to the target ground image area and the second point cloud data.

[0039] The third aspect of the embodiments of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the steps of the ground detection method as described above.

[0040] The fourth aspect of the embodiments of the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the electronic device implements the steps of the ground detection method as described above.

[0041] The fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the ground detection method described in any item of the first aspect above.

[0042] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: In the embodiments of the present application, after obtaining the first point cloud data corresponding to the preset detection area, the height information of the first point cloud data is normalized to the image space to obtain the first grayscale image. Then, the first grayscale image is filtered to obtain the second grayscale image. According to the second grayscale image, the filtered second point cloud data can be determined, and according to the second grayscale image and the preset threshold segmentation algorithm, the target ground image area can be determined from the second grayscale image. After that, according to the target ground image area and the second point cloud data, the passable ground area can be located from the preset detection area. Since the first grayscale image is an image obtained based on the height information of the first point cloud data, and the second grayscale image is an image obtained by further filtering the first grayscale image, the second grayscale image is an image containing more accurate height information after filtering, so that the subsequent second point cloud data containing more accurate height information can be determined based on the second grayscale image, and the target ground image area can be accurately determined by using the second grayscale image carrying accurate height information and the preset threshold segmentation algorithm, so that according to the target ground image area and the second point cloud data, the passable ground area can be accurately located. That is, the embodiments of the present application can convert the point cloud data to the image space for filtering and threshold segmentation, and then accurately locate the passable ground area in the point cloud data according to the filtering result and the threshold segmentation result; since the filtering and threshold segmentation in the image space are more accurate and the algorithm complexity is lower, the accuracy and efficiency of ground detection can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art.

[0044] Figure 1 It is a schematic flowchart of the implementation of a ground detection method provided by the embodiments of the present application;

[0045] Figure 2 It is an example diagram of a ground detection device provided by the embodiments of the present application;

[0046] Figure 3 It is a schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, those skilled in the art should understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obscuring the description of the present application.

[0048] In order to illustrate the technical solutions described in the present application, the following will be described through specific embodiments.

[0049] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0050] It should also be understood that the terms used in the present specification of the present application are merely for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0051] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0052] As used in this specification and the appended claims, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context.

[0053] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0054] Currently, ground detection is a key link for a robot to achieve autonomous motion functions. However, the existing ground detection methods have the defect of relatively low accuracy.

[0055] To solve this technical problem, the embodiments of the present application provide a ground detection method, device, electronic device, and storage medium, including: obtaining first point cloud data corresponding to a preset detection area; normalizing the height information of the first point cloud data into the image space to obtain a first grayscale image; performing filtering processing on the first grayscale image to obtain a second grayscale image; determining second point cloud data according to the second grayscale image; determining a target ground image area in the second grayscale image according to the second grayscale image and a preset threshold segmentation algorithm; and positioning a passable ground area according to the target ground image area and the second point cloud data.

[0056] Since the first grayscale image is an image obtained based on the height information of the first point cloud data, and the second grayscale image is an image obtained by further performing filtering processing on the first grayscale image, therefore, the second grayscale image is an image containing more accurate height information after filtering, enabling subsequent determination of second point cloud data containing more accurate height information based on the second grayscale image, and accurately determining the target ground image area by using the second grayscale image carrying accurate height information and a preset threshold segmentation algorithm. Thus, according to the target ground image area and the second point cloud data, the passable ground area can be accurately positioned. That is, the embodiments of the present application can convert the point cloud data into the image space for filtering processing and threshold segmentation, and then accurately locate the passable ground area in the point cloud data according to the filtering processing result and the threshold segmentation result; because the filtering processing and threshold segmentation in the image space have high accuracy and low algorithm complexity, the accuracy and efficiency of ground detection can be improved.

[0057] Example 1:

[0058] Figure 1 FIG. shows a schematic flowchart of the first ground detection method provided by the embodiments of the present application. The execution subject of this ground detection method is an electronic device, which can be, for example, a robot. As Figure 1 shown, the ground detection method is described in detail as follows:

[0059] In S101, obtain first point cloud data corresponding to a preset detection area.

[0060] In the embodiments of the present application, the preset detection area can be an area in the forward direction of the robot during the movement of the robot and with a distance less than a preset distance from the robot. In one embodiment, when the robot is running, information of the preset detection area can be collected through a detection device installed on the robot and inclined downward to generate first point cloud data, and the first point cloud data carries three-dimensional information of the preset detection area. The detection device can be a vision device. The preset detection area can specifically be the maximum area that the detection device can detect.

[0061] In S102, the height information of the first point cloud data is normalized to the image space to obtain a first grayscale image.

[0062] The first point cloud data containing the three-dimensional information of the preset detection area obtained in step S101 specifically includes point cloud x-axis data, point cloud y-axis data, and point cloud z-axis data. Among them, the point cloud z-axis data is the data used to represent the height information of the preset detection area in the first point cloud data. Normalizing the point cloud z-axis data to the image space means mapping the numerical value of the point cloud z-axis data to the corresponding grayscale value in the grayscale image, thereby obtaining the first grayscale image. In this first grayscale image, the grayscale value of each pixel point corresponds to the z-axis coordinate value in the first point cloud data. Therefore, this first grayscale image carries the height information of each point in the first point cloud data.

[0063] In S103, the first grayscale image is filtered to obtain a second grayscale image.

[0064] After obtaining the first grayscale image carrying height information, the first grayscale image is filtered to obtain a filtered grayscale image, that is, the second grayscale image. Among them, the filtering process can be a mean filtering process, a median filtering process, a Gaussian filtering process, etc., which are processing methods capable of filtering out image noise. Since the second grayscale image is a grayscale image with image noise filtered out, the height information carried in the second grayscale image is more accurate.

[0065] In S104, second point cloud data is determined according to the second grayscale image.

[0066] After obtaining the second grayscale image carrying more accurate height information, according to the second grayscale image, through an inverse normalization algorithm, that is, mapping the grayscale value in the second grayscale image back to height information, filtered z-axis data can be obtained. Combining the filtered z-axis data, the x-axis data, and the y-axis data in the first point cloud data to form filtered point cloud data, that is, the second point cloud data. This second point cloud data is the point cloud data carrying the z-axis data that can accurately represent the height.

[0067] In S105, a target ground image area in the second grayscale image is determined according to the second grayscale image and a preset threshold segmentation algorithm.

[0068] In addition, since the gray values of the second gray-scale image can represent height information, according to the second gray-scale image and a preset threshold segmentation algorithm, threshold segmentation is performed on the second gray-scale image, and an image region whose gray values meet the preset threshold condition (i.e., the gray value condition corresponding to the ground region) can be segmented therefrom as the target ground image region. For example, after measurement, the image region with gray values in the range of [148, 188] in the gray-scale image corresponds to the ground region with a height close to 0 meters in the actual space. Therefore, a region with gray values between [148, 188] can be segmented from the second gray-scale image as the target ground image region.

[0069] In S106, according to the target ground image region and the second point cloud data, the passable ground region is located.

[0070] In the embodiment of the present application, the passable ground region is a flat and passable ground region in the preset detection region.

[0071] After determining the target ground image region of the second gray-scale image, according to the mapping relationship between the image coordinate system of the second gray-scale image and the point cloud coordinate system of the second point cloud data, the point cloud data corresponding to the target ground image region can be determined from the second point cloud data as the point cloud data of the passable ground region. Finally, the robot can make a motion decision according to the point cloud data of the passable ground region and move on the passable ground region in the actual space.

[0072] In the embodiment of the present application, since the first gray-scale image is an image obtained based on the height information of the first point cloud data, and the second gray-scale image is an image further obtained by filtering the first gray-scale image, therefore, the second gray-scale image is an image containing more accurate height information after filtering, enabling subsequent determination of the second point cloud data containing more accurate height information based on the second gray-scale image, and accurately determining the target ground image region by using the second gray-scale image carrying accurate height information and the preset threshold segmentation algorithm, so that according to the target ground image region and the second point cloud data, the passable ground region can be accurately located. That is, the embodiment of the present application can convert the point cloud data to the image space for filtering processing and threshold segmentation, and then accurately locate the passable ground region in the point cloud data according to the filtering processing result and the threshold segmentation result; since the filtering processing and threshold segmentation in the image space have higher accuracy and lower algorithm complexity, the accuracy and efficiency of ground detection can be improved.

[0073] Optionally, the obtaining of the first point cloud data corresponding to the preset detection region includes:

[0074] Obtain the depth image corresponding to the preset detection region;

[0075] Determine the point cloud data in the camera coordinate system according to the depth image;

[0076] Determine the point cloud data in the robot coordinate system as the first point cloud data according to the point cloud data in the camera coordinate system and a preset coordinate conversion relationship.

[0077] In the embodiments of the present application, the above detection device may specifically be a depth camera. To cope with complex and changeable environments, multiple sensors are usually installed on the robot to enhance the robot's perception ability. These sensors may include infrared sensors, ultrasonic sensors, lidar, etc. Since the working principles of different sensors are different, the usage scenarios will be restricted differently. On the other hand, different sensors are only suitable for some of the tasks. For example, a single-line lidar is suitable for detecting obstacles and performing positioning and navigation, and a multi-line lidar can be used for multiple tasks, including navigation and positioning, object detection, and ground detection. However, due to the high price of multi-line lidar, it will greatly increase the hardware cost of the mobile robot. Therefore, compared with other sensors, the depth camera not only has a large detection range and high spatial resolution of detection, but also has a relatively low price, and is suitable for the application of detecting passable areas on the ground by the robot.

[0078] The depth camera is installed on the robot, and its shooting angle is an angle inclined downward so that the preset detection area includes the ground area. By shooting the preset detection area with the depth camera, a depth image corresponding to the preset detection area can be obtained.

[0079] After obtaining the depth image, data dimensionality reduction processing (such as data sampling processing) can be performed on the depth image to reduce the amount of data for subsequent processing and improve the processing efficiency. Then, according to a preset conversion formula, the data of the depth image is correspondingly converted into the point cloud data in the camera coordinate system of the depth camera.

[0080] After determining the point cloud data in the camera coordinate system, according to the preset coordinate conversion relationship between the camera coordinate system and the robot coordinate system, the point cloud data in the camera coordinate system is correspondingly mapped to the point cloud data in the robot coordinate system to obtain the first point cloud data.

[0081] In the embodiments of the present application, by collecting the depth image with the depth camera, the first point cloud data corresponding to the preset detection area can be determined efficiently and accurately on the premise of controlling the cost. Therefore, while improving the efficiency and accuracy of ground detection, the hardware cost can be reduced.

[0082] Optionally, the normalizing the height information of the first point cloud data to the image space to obtain a first grayscale image includes:

[0083] Convert the height information of the first point cloud data to a preset height interval to obtain target height data;

[0084] Normalize the target height data to a preset grayscale interval to obtain a first grayscale image.

[0085] In the embodiments of the present application, the preset height interval is an area with a height difference less than a preset value from the passable ground area preset according to the conventional obstacle height. Exemplarily, assuming that the height value of the original flat ground without obstacles is 0 meters, the preset height interval can be [-0.31, 0.5] meters, or [-0.32, 0.32] meters, or [-0.42, 0.22] meters. Among them, negative values indicate lower than the original flat ground, and positive values indicate higher than the original flat ground. In the preset detection area, since the height information exceeding this preset height interval cannot be the height information corresponding to the passable ground area, the height information higher than the maximum value of this preset height interval in the first point cloud data can be ignored, and the height information higher than the maximum value of this preset height interval is converted into the maximum value of this preset height interval, and the height information less than the minimum value of this preset height interval is converted into the minimum value of this preset height interval, so as to obtain target height data with all height information falling within this preset height interval. For example, for the preset height interval [-0.31, 0.5] meters, the height information with a height of 1 meter in the first point cloud data can be converted into height information with a height of 0.5 meters.

[0086] After obtaining the target height data, perform normalization processing on the target height data to convert the target height data from the original preset height interval to the [0, 1] interval to obtain normalized height data. For example, for the target height data located in the preset height interval [0.42, 0.22] meters, add the value 0.42 to each data, so as to convert the target height data to the [0, 0.64] interval; then, multiply each number in [0, 0.64] by 1.5625 to obtain normalized height data located in the [0, 1] interval. Then, according to the preset grayscale interval [0, 255], convert the normalized height data to this preset grayscale interval, so that each height information can be accurately represented by the grayscale value in the preset grayscale interval, and a first grayscale image is obtained.

[0087] In the embodiments of the present application, since the height information of the first point cloud data can be first converted to a preset height interval to obtain target height data, and then the target height data can be normalized to a preset grayscale interval, a first grayscale image carrying the height information concerned during ground detection can be accurately obtained, so as to improve the efficiency and accuracy of subsequent ground detection.

[0088] Optionally, the filtering the first grayscale image to obtain a second grayscale image includes:

[0089] Perform median filtering and dimensionality reduction and cropping on the first grayscale image to obtain the second grayscale image.

[0090] In the embodiments of the present application, the filtering process of the first grayscale image may specifically be median filtering. Median filtering can better preserve the image boundary, so that while filtering out inaccurate data scattered in the image, it can retain the boundaries of points with different heights in space, improving the accuracy of ground detection.

[0091] After obtaining the grayscale image after median filtering through median filtering, further perform dimensionality reduction and cropping on the grayscale image to obtain a grayscale image with less data volume as the second grayscale image, thereby further reducing the data calculation complexity and improving the ground detection efficiency. Specifically, the dimensionality reduction and cropping process includes dimensionality reduction processing and cropping processing.

[0092] For the dimensionality reduction processing, since the image has been filtered, a larger-scale dimensionality reduction processing will not affect the final ground detection. Therefore, the dimensionality reduction processing can sample sparser data than the first dimensionality reduction processing, thereby minimizing the data volume and improving the ground detection efficiency.

[0093] For the cropping process, specifically, a nearby range area that the robot focuses on during movement can be determined from the preset detection area, and the area corresponding to this range area in the grayscale image is retained, and other areas outside this range area are cropped and deleted, thereby obtaining the second grayscale image. For example, in some scenarios, it is not necessary to detect the entire camera field of view, but only need to detect a part of the area. For example, when the robot moves at a speed of 1 meter per second, only the data within 10 meters needs to be detected. Therefore, by cropping, only the image within 10 meters in the grayscale image is retained to obtain the second grayscale image. Through the cropping process, while ensuring that the ground detection result meets the movement requirements of the robot, it saves processor resources, accelerates the operation, and improves the ground detection efficiency.

[0094] In the embodiments of the present application, by performing median filtering and dimensionality reduction and cropping on the first grayscale image, a second grayscale image that can accurately retain the height information required for ground detection and reduce the data volume can be obtained, thereby ensuring the accuracy and efficiency of subsequent ground detection.

[0095] Optionally, determining the target ground image area in the second grayscale image according to the second grayscale image and a preset threshold segmentation algorithm includes:

[0096] Perform gradient calculation on the second grayscale image to determine the target gradient image corresponding to the second grayscale image;

[0097] Perform threshold segmentation processing on the target gradient image according to the first gray level threshold to determine a first set of pixel points in the target gradient image whose gray level values are less than the first gray level threshold;

[0098] Perform threshold segmentation processing on the second gray image according to the target threshold interval to determine a second set of pixel points in the second gray image whose gray level values are within the target threshold interval;

[0099] Determine the target ground image area according to the first set of pixel points and the second set of pixel points.

[0100] In the embodiments of the present application, after obtaining the second gray image, gradient calculation can be performed on the second gray image based on the scharr and soble operators to determine the target gradient image corresponding to the second gray image. In one embodiment, gradient calculation can be first performed on the second gray image to obtain a first-order gradient map; then, gradient calculation is performed on the first-order gradient map to obtain a second-order target gradient image. In the target gradient image, the gray level value of each pixel point is used to represent the gradient value of the original second gray image. A gradient of 0 indicates that the position corresponding to the second gray image is the position of a flat area, and the greater the gradient, the more uneven the point at that position. Among them, the flat area can include any area with a relatively flat surface such as a horizontal plane, a gentle slope plane, and a steep slope plane.

[0101] After obtaining the target gradient image, perform threshold segmentation processing on the target gradient image according to the preset first gray level threshold and the gray level values of each pixel point in the target gradient image to determine each pixel point in the target gradient image whose gray level value is less than the first gray level value. The set composed of these pixel points is the first set of pixel points. Among them, the first gray level threshold is the gray level threshold presented in the gradient map corresponding to the maximum fluctuation degree of the passable ground area determined according to the height fluctuation change of the actual ground. Each pixel point in the first set of pixel points obtained by performing threshold segmentation processing according to the first gray level threshold corresponds to a position in the actual ground where the height fluctuation is less than the preset range. Exemplarily, during threshold segmentation, the gray level value of a pixel point in the target gradient image that is less than the first gray level value can be thresholded to 255, and the gray level value of a pixel point that is greater than or equal to the first gray level value can be thresholded to 0. At this time, the set of pixel points with a gray level value of 255 in the target gradient image is the first set of pixel points.

[0102] In addition, threshold segmentation processing is performed on the second grayscale image according to the target threshold interval and the grayscale values of each pixel point in the second grayscale image, and a set composed of each pixel point whose grayscale value in the second grayscale image is within the target threshold interval is determined as the second pixel point set. Among them, the target threshold interval is the grayscale value interval in the grayscale image corresponding to the ground height range determined according to the actual passable ground height range and the mapping relationship between the height information and the grayscale value. In the second grayscale image, the second pixel point set with grayscale values within the target threshold interval corresponds to the position in the actual space where the height is close to the ground height of 0. Exemplarily, during threshold segmentation, the grayscale values of the pixel points in the second grayscale image whose grayscale values are within the target threshold interval can be thresholded to 255, and the grayscale values of the pixel points outside the target threshold interval can be thresholded to 0. At this time, the set composed of the pixel points with a grayscale value of 255 in the second grayscale image is the second pixel point set.

[0103] After determining the first pixel point set and the second pixel point set, take the intersection of the first pixel point set and the second pixel point set in the second grayscale image to obtain a third pixel point set. The image area determined according to the third pixel point set is the target ground image area. The target ground image area corresponds to the area in the actual ground where the height fluctuation is less than the preset fluctuation and the height is close to the ground height of 0.

[0104] In the embodiment of the present application, since the first pixel point set with relatively small height fluctuation can be determined through the threshold segmentation processing of the target gradient image, and the second pixel point set with a height close to the ground 0 can be determined through the threshold segmentation processing of the second grayscale image, it is possible to determine, based on the first pixel point set and the second pixel point set, the image area corresponding to the relatively flat ground area near the ground height of 0 as the target ground image area, thereby improving the accuracy of ground detection.

[0105] Optionally, the determining the target ground image area according to the first pixel point set and the second pixel point set includes:

[0106] Determine a target contour according to the first pixel point set and the second pixel point set;

[0107] If the area of the target contour is greater than a preset area threshold, determine the target ground image area according to the target contour.

[0108] In the embodiment of the present application, after obtaining the first pixel point set and the second pixel point set, the intersection of the first pixel point set and the second pixel point set can be obtained to get a third pixel point set. Then, erosion and dilation operations are performed on the image area formed by the third pixel point set to obtain a to-be-processed image. Contour detection is performed on the to-be-processed image to obtain a target contour.

[0109] After that, it is determined whether the area of the target contour is greater than a preset area threshold. If so, the target ground image area can be determined according to the target contour. The target ground image area can directly be the area where the target contour is located, or an image area including the target contour and a preset area around the target contour. If not, it means that the current relatively flat area close to the ground is too small and not suitable for the robot to pass through, and the area where the current target contour is located is determined as the image area corresponding to the non-ground area.

[0110] In the embodiments of the present application, since the target contour can be determined according to the first pixel point set and the second pixel point set, and when the area of the target contour is greater than the preset area threshold, the determination of the next target ground image area is performed, the image area corresponding to the passable ground area can be accurately screened out.

[0111] Optionally, the determining the target ground image area according to the target contour includes:

[0112] Determine the centroid of the target contour;

[0113] Determine the optimal seed point according to the centroid;

[0114] Process the second grayscale image by the flood filling algorithm according to the optimal seed point and a preset filling threshold to obtain the target ground image area.

[0115] In the embodiments of the present application, for a target contour with an area greater than the preset area threshold, the centroid of the target contour can be determined by a preset image centroid algorithm.

[0116] After that, according to the coordinates of the centroid, it is determined whether the centroid is located inside the target contour. If so, the centroid of the target contour is used as the optimal seed point. If not, according to the coordinates of the centroid, the pixel point that is in the same row or column as the centroid and is the closest to the centroid inside the target contour is determined as the optimal seed point inside the target contour from the second grayscale image.

[0117] After determining the optimal seed point, in the second grayscale image, using the optimal seed point as the position parameter of the flood filling algorithm and combining the preset filling threshold, the second grayscale image is processed by the flood filling algorithm to determine a larger image area near the optimal seed point, so that an image area that meets the conditions and is expanded based on the target contour can be obtained as the target ground image area.

[0118] In the embodiments of the present application, by accurately determining the optimal seed points and the flood filling algorithm, an expanded area can be determined based on the target contour as the target ground image area, so that the image area corresponding to the ground that was filtered out due to the strict threshold segmentation process before can be expanded into the target ground image area, enabling the subsequent determination of the corresponding passable ground area based on the target ground image area to be more accurate.

[0119] In some embodiments, after determining the target ground image area, the image area in the second grayscale image other than the target ground image area is determined as the non-ground image area, and this non-ground image area is the image corresponding to the non-ground area in the actual space. This non-ground area can be the area corresponding to obstacles, cliffs, steep slopes, steps, etc.

[0120] According to this non-ground image area and the second point cloud data, the point cloud data corresponding to the non-ground area in the actual space can be determined from the second point cloud data. Then, the robot makes corresponding motion decisions according to the point cloud data corresponding to the non-ground area. For example, when passing, it bypasses this non-ground area.

[0121] Through the ground detection method of the embodiments of the present application, it is possible to accurately identify the passable ground area and the non-ground area on the premise of controlling the hardware cost, thereby improving the navigation and path planning capabilities of the robot, enhancing the scene adaptation ability of the robot, and making the robot's movement more intelligent and safe.

[0122] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0123] Example 2:

[0124] Figure 2 The structural schematic diagram of a ground detection device provided by the embodiments of the present application is shown. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown:

[0125] The ground detection device includes: a first point cloud data acquisition unit 21, a first grayscale image determination unit 22, a second grayscale image determination unit 23, a second point cloud data determination unit 24, a target ground image area determination unit 25, and a positioning unit 26. Among them:

[0126] The first point cloud data acquisition unit 21 is used to acquire the first point cloud data corresponding to a preset detection area.

[0127] The first grayscale image determination unit 22 is used to normalize the height information of the first point cloud data into the image space to obtain the first grayscale image.

[0128] The second grayscale image determination unit 23 is configured to perform filtering processing on the first grayscale image to obtain a second grayscale image.

[0129] The second point cloud data determination unit 24 is configured to determine second point cloud data according to the second grayscale image.

[0130] The target ground image area determination unit 25 is configured to determine a target ground image area in the second grayscale image according to the second grayscale image and a preset threshold segmentation algorithm.

[0131] The positioning unit 26 is configured to locate a passable ground area according to the target ground image area and the second point cloud data.

[0132] Optionally, the first point cloud data acquisition unit 21 is specifically configured to acquire a depth image corresponding to the preset detection area; determine point cloud data in the camera coordinate system according to the depth image; and determine point cloud data in the robot coordinate system as the first point cloud data according to the point cloud data in the camera coordinate system and a preset coordinate transformation relationship.

[0133] Optionally, the first grayscale image determination unit 22 is specifically configured to convert the height information of the first point cloud data to a preset height interval to obtain target height data; and normalize the target height data to a preset grayscale interval to obtain a first grayscale image.

[0134] Optionally, the second grayscale image determination unit 23 is specifically configured to perform median filtering processing and dimensionality reduction and cropping processing on the first grayscale image to obtain the second grayscale image.

[0135] Optionally, the target ground image area determination unit 25 includes:

[0136] A target gradient image determination module, configured to perform gradient calculation on the second grayscale image to determine a target gradient image corresponding to the second grayscale image;

[0137] A first pixel point set determination module, configured to perform threshold segmentation processing on the target gradient image according to a first grayscale threshold to determine a first pixel point set in the target gradient image whose grayscale value is less than the first grayscale threshold;

[0138] A second pixel point set determination module, configured to perform threshold segmentation processing on the second grayscale image according to a target threshold interval to determine a second pixel point set in the second grayscale image whose grayscale value is within the target threshold interval;

[0139] A target ground image area determination module, configured to determine a target ground image area according to the first pixel point set and the second pixel point set.

[0140] Optionally, in the target ground image area determination module, specifically configured to determine a target contour according to the first pixel point set and the second pixel point set; if the area of the target contour is greater than a preset area threshold, then determine the target ground image area according to the target contour.

[0141] Optionally, in the target ground image area determination module, the determining the target ground image area according to the target contour includes: determining the centroid of the target contour; determining an optimal seed point according to the centroid; and performing flood filling algorithm on the second grayscale image according to the optimal seed point to obtain the target ground image area.

[0142] It should be noted that the information interaction, execution process, etc. between the above-mentioned device / units, due to being based on the same concept as the method embodiment of the present application, for the specific functions and the technical effects brought, please refer to the method embodiment part for details, and will not be elaborated here.

[0143] Example 3:

[0144] Figure 3 is a schematic diagram of an electronic device provided by an embodiment of the present application. As Figure 3 shown, the electronic device 3 in this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a ground detection program. When the processor 30 executes the computer program 32, the steps in the above-mentioned various ground detection method embodiments are implemented, such as Figure 1 the steps S101 to S106 shown. Alternatively, when the processor 30 executes the computer program 32, the functions of each module / unit in the above-mentioned device embodiments are implemented, such as Figure 2 the functions of the first point cloud data acquisition unit 21 to the positioning unit 26 shown.

[0145] Exemplarily, the computer program 32 can be divided into one or more modules / units, and the one or more modules / units are stored in the memory 31 and executed by the processor 30 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 32 in the electronic device 3.

[0146] The electronic device 3 can be a computing device such as a robot, a desktop computer, a notebook, a palm computer, and a cloud server. The electronic device may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art can understand that Figure 3 merely examples of the electronic device 3, which do not constitute a limitation on the electronic device 3, may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the electronic device may further include an input / output device, a network access device, a bus, etc.

[0147] The so-called processor 30 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0148] The memory 31 may be an internal storage unit of the electronic device 3, such as the hard disk or memory of the electronic device 3. The memory 31 may also be an external storage device of the electronic device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 3. Further, the memory 31 may also include both the internal storage unit and the external storage device of the electronic device 3. The memory 31 is used to store the computer program and other programs and data required by the electronic device. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0149] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0150] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0151] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in the form of hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0152] In the embodiments provided in this application, it should be understood that the disclosed device / electronic device and method can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0153] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0154] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0155] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, all or part of the processes in the above-described embodiment methods of the present application can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0156] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A ground detection method, characterized in that, Including: Obtain the first point cloud data corresponding to a preset detection area; Normalize the height information of the first point cloud data into the image space to obtain a first grayscale image; Perform filtering processing on the first grayscale image to obtain a second grayscale image; Determine the second point cloud data according to the second grayscale image; Determine the target ground image area in the second grayscale image according to the second grayscale image and a preset threshold segmentation algorithm, including: calculating the gradient of the second grayscale image to determine the target gradient image corresponding to the second grayscale image; performing threshold segmentation processing on the target gradient image according to a first grayscale threshold to determine a first pixel point set in the target gradient image with a grayscale value less than the first grayscale threshold; performing threshold segmentation processing on the second grayscale image according to a target threshold interval to determine a second pixel point set in the second grayscale image with a grayscale value within the target threshold interval; determining a target contour according to the first pixel point set and the second pixel point set; if the area of the target contour is greater than a preset area threshold, then determine the target ground image area according to the target contour; Locate the passable ground area according to the target ground image area and the second point cloud data.

2. The ground detection method according to claim 1, characterized in that, The obtaining the first point cloud data corresponding to a preset detection area includes: Obtain the depth image corresponding to the preset detection area; Determine the point cloud data in the camera coordinate system according to the depth image; Determine the point cloud data in the robot coordinate system as the first point cloud data according to the point cloud data in the camera coordinate system and a preset coordinate conversion relationship.

3. The ground detection method according to claim 1, characterized in that, The normalizing the height information of the first point cloud data into the image space to obtain a first grayscale image includes: Convert the height information of the first point cloud data to a preset height interval to obtain target height data; Normalize the target height data to a preset grayscale interval to obtain a first grayscale image.

4. The ground detection method according to claim 1, characterized in that, The performing filtering processing on the first grayscale image to obtain a second grayscale image includes: Perform median filtering processing and dimensionality reduction and cropping processing on the first grayscale image to obtain the second grayscale image.

5. The ground detection method according to claim 1, characterized in that, The determining the target ground image area according to the target contour includes: Determine the centroid of the target contour; Determine the optimal seed point according to the centroid; Process the second grayscale image through a flood filling algorithm according to the optimal seed point to obtain the target ground image area.

6. A ground detection device, characterized in that, Including: A first point cloud data acquisition unit for obtaining the first point cloud data corresponding to a preset detection area; A first grayscale image determination unit for normalizing the height information of the first point cloud data into the image space to obtain a first grayscale image; A second grayscale image determination unit for performing filtering processing on the first grayscale image to obtain a second grayscale image; A second point cloud data determination unit for determining the second point cloud data according to the second grayscale image; A target ground image area determination unit for determining the target ground image area in the second grayscale image according to the second grayscale image and a preset threshold segmentation algorithm; A positioning unit for positioning a passable ground area according to the target ground image area and the second point cloud data; The target ground image area determination unit includes: A target gradient image determination module for performing gradient calculation on the second grayscale image to determine a target gradient image corresponding to the second grayscale image; A first pixel point set determination module for performing threshold segmentation processing on the target gradient image according to a first grayscale threshold to determine a first pixel point set in the target gradient image whose grayscale value is less than the first grayscale threshold; A second pixel point set determination module for performing threshold segmentation processing on the second grayscale image according to a target threshold interval to determine a second pixel point set in the second grayscale image whose grayscale value is within the target threshold interval; A target ground image area determination module for determining a target ground image area according to the first pixel point set and the second pixel point set; The target ground image area determination module is specifically configured to determine a target contour according to the first pixel point set and the second pixel point set; if the area of the target contour is greater than a preset area threshold, then determine the target ground image area according to the target contour.

7. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the electronic device realizes the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the electronic device realizes the steps of the method according to any one of claims 1 to 5.

Images