Slope detection method, robot and storage medium
Detection of the slope area of the ground by a depth camera solves the problem of insufficient perception ability in the prior art, and achieves more efficient and accurate slope detection, reducing costs and improving the robot's environmental perception ability.
Patent Information
- Application Number
- CN202210600378.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-30
- Publication Date
- 2025-01-14
- Estimated Expiration
- 2042-05-30
AI Technical Summary
The prior art cannot effectively detect slope areas on the ground, resulting in insufficient perception of the robot's external environment.
The depth image collected by the depth camera is used to calculate the gradient value of the pixel point and the key pixel point, and determine the slope area of the ground, thereby improving the robot's perception of the external environment.
It realizes faster and more accurate detection of sloped areas on the ground, reducing costs, and improving robots' perception of the environment to ensure safety of movement.
Smart Images
Figure CN114882343B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of data processing technology, and in particular, relates to a slope detection method, a robot and a storage medium. Background Art
[0002] With the development of robots, robots have been able to move and navigate autonomously. The key to autonomous robot navigation is the detection of the ground. Through the detection of the ground, the infeasible areas on the ground, such as obstacles and cliffs, are identified, and the robot's movement route is planned according to the infeasible areas.
[0003] At present, ground detection mostly stops to determine whether there are obstacles or cliffs on the ground, and there is a lack of detection of slope areas on the ground, and the robot's perception ability of the external environment is insufficient. Summary of the invention
[0004] The embodiments of the present application provide a slope detection method, a robot, and a storage medium, which can solve the problem of the robot's insufficient perception of the external environment due to the inability to detect slope areas on the ground.
[0005] In a first aspect, an embodiment of the present application provides a slope detection method, comprising:
[0006] Acquire a first grayscale image of a depth image acquired by a depth camera, wherein the depth image includes a ground surface;
[0007] Calculating a gradient value of a first pixel point based on a grayscale value of the first pixel point in the first grayscale image, where the number of the first pixel points is multiple;
[0008] Determining a key pixel from the first pixel based on a gradient value of the first pixel;
[0009] Based on the grayscale value of the key pixel point and the grayscale value of the second pixel point, the slope area in the first grayscale image is determined, wherein the second pixel point is the first pixel point in the first grayscale image except the key pixel point, and the slope area in the first grayscale image is used to determine the slope area of the ground.
[0010] In a second aspect, an embodiment of the present application provides a slope detection device, comprising:
[0011] An image acquisition module, configured to acquire a first grayscale image of a depth image acquired by a depth camera, wherein the depth image includes a ground surface;
[0012] a gradient calculation module, configured to calculate a gradient value of a first pixel point based on a grayscale value of the first pixel point in the first grayscale image, wherein the number of the first pixel points is multiple;
[0013] A key point determination module, configured to determine a key pixel point from the first pixel points based on a gradient value of the first pixel point;
[0014] A slope detection module is used to determine the slope area in the first grayscale image based on the grayscale value of the key pixel point and the grayscale value of the second pixel point, wherein the second pixel point is the first pixel point in the first grayscale image other than the key pixel point, and the slope area in the first grayscale image is used to determine the slope area of the ground.
[0015] In a third aspect, an embodiment of the present application provides a terminal device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the slope detection method described in any one of the first aspects above when executing the computer program.
[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the slope detection method described in any one of the above-mentioned first aspects is implemented.
[0017] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the slope detection method described in any one of the above-mentioned first aspects.
[0018] The beneficial effect of the first aspect of the present application compared with the prior art is that the present application uses the grayscale value of the first pixel in the first grayscale image of the depth image captured by the depth camera to obtain the gradient value of the first pixel, and then uses the gradient value of the first pixel to obtain the key pixel, and uses the grayscale value of the key pixel and the grayscale value of other pixels to determine the slope area in the first grayscale image, and then determines the slope area of the ground. Compared with the prior art that cannot detect the slope area of the ground, the present application uses the depth image captured by the depth camera to determine the slope area. The area captured by the depth camera is large, and the resolution of the depth camera is high. The present application can determine the slope area of the ground more quickly and accurately, and improve the robot's perception of the external environment.
[0019] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0021] Figure 1 This is a schematic diagram of an application scenario of a slope detection method provided by an embodiment of the present application;
[0022] Figure 2 is a flowchart of a slope detection method provided by an embodiment of the present application;
[0023] Figure 3 is a flowchart of a method for determining a first grayscale image provided by an embodiment of the present application;
[0024] Figure 4 is a flow chart of a method for determining key pixels provided in an embodiment of the present application;
[0025] Figure 5 is a flow chart of a method for determining a slope angle of a pixel point provided in an embodiment of the present application;
[0026] Figure 6 is a structural schematic diagram of a slope detection device provided in an embodiment of the present application;
[0027] Figure 7 It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0028] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0029] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0030] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0031] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.
[0032] At present, when using laser radar or RGB camera for ground detection, the effect of ground detection can be achieved. The slope detection method provided by this application uses the depth image collected by the depth camera to determine the slope area of the ground. Since the depth image includes distance and spatial information, when using the depth camera to detect the slope of the ground, not only the slope area on the ground can be determined, but also the angle and type of slope can be determined. In addition, the depth camera is cheaper than the laser radar. Therefore, using the depth camera for road detection can save costs compared to using the laser radar for road detection.
[0033] Figure 1 Schematic diagram of application scenario of the slope detection method provided in the embodiment of the present application, the above slope detection method can be used to detect the slope area of the ground. Among them, the depth camera 10 is used to collect the depth image of the ground. The data processing device 20 is used to obtain the depth image collected by the depth camera 10, and output the slope area in the depth image according to the depth image.
[0034] Figure 2 A schematic flow chart of the slope detection method provided by the present application is shown, referring to Figure 2 , the method is described in detail as follows:
[0035] S101, obtaining a first grayscale image of a depth image captured by a depth camera, wherein the depth image includes a ground surface.
[0036] In this embodiment, the depth camera is a camera that can perform depth measurement. The depth camera can be a depth camera installed on the robot; the depth camera can also be a depth camera installed in a fixed area. The installation angle of the depth camera is based on the image collected on the ground. For example, the depth camera on the robot can collect information about the ground in front of the robot, and the front of the robot is the moving direction of the robot.
[0037] A depth image, also known as a distance image, refers to an image that uses the distance (depth) values of each point in the scene captured by a depth camera as pixel values. It directly reflects the geometric shape of the visible surface of the scene.
[0038] A grayscale image, also known as a grayscale image, divides white and black into several levels according to a logarithmic relationship, which is called grayscale.
[0039] In this embodiment, the grayscale value of the pixel in the first grayscale image is determined based on the pixel value (depth) of the pixel in the depth image.
[0040] S102: Calculate a gradient value of a first pixel in the first grayscale image based on a grayscale value of the first pixel, where the number of the first pixel is multiple.
[0041] In this embodiment, each pixel in the first grayscale image may correspond to a grayscale value. The first pixel may include each pixel in the first grayscale image, and the pixel in an area in the first grayscale image may be recorded as the first pixel, where the area includes the ground. For example, if the first grayscale image includes a chair, the pixel in the area other than the chair area may be recorded as the first pixel.
[0042] In this embodiment, the gradient value of the first pixel point may include a first-order gradient value and a second-order gradient value.
[0043] The first-order derivative of the first grayscale image is calculated to obtain the first-order gradient value of the first pixel. The derivative of the first-order gradient value is calculated to obtain the second-order gradient value of the first pixel.
[0044] Specifically, a coordinate system where the first grayscale image is located is established. For example, a coordinate system is established with the lower left corner of the first grayscale image as the origin and the two sides of the first grayscale image passing through the origin as the x-axis and the y-axis. The first-order gradient of the first pixel point in the x-axis direction and the first-order gradient on the y-axis are calculated to obtain the first-order gradient value of the first pixel point, and then the first-order gradient map of the first grayscale image is obtained. The gradient of the first-order gradient map is calculated to obtain the second-order gradient map of the first grayscale image.
[0045] Specifically, the values of the first pixels in the first-order gradient image that are greater than or equal to the first threshold are set to the first value (e.g., 255), and the values of other first pixels are set to the second value (e.g., 0), to obtain a first binary image. The first threshold can be set as needed.
[0046] The value of the first pixel point in the second-order gradient image that is less than or equal to the second threshold is set as the first value, and the values of other first pixel points are set as the second value to obtain a second binary image. The second threshold can be set as needed.
[0047] In the first grayscale image, the first pixel point that is the first value in the first binary image and also the first value in the second binary image is recorded as the third pixel point, the value of the third pixel point is set to the first value, and the values of other first pixel points are set to the second value, to obtain a third binary image.
[0048] S103: Determine key pixels from the first pixels based on the gradient value of the first pixels.
[0049] In this embodiment, the image marked with the gradient value of the first pixel is input into the key point extraction model to obtain the key pixel in the first pixel. The key point extraction model can be a convolutional neural network model, a fully connected neural network model, etc.
[0050] The number of key pixels is one or more.
[0051] S104, determining a slope area in the first grayscale image based on the grayscale value of the key pixel and the grayscale value of a second pixel, wherein the second pixel is a first pixel in the first grayscale image other than the key pixel.
[0052] In this embodiment, the slope area in the first grayscale image is used to determine the slope area of the ground.
[0053] In this embodiment, each key pixel point is a seed point, and the slope area corresponding to each key pixel point is calculated using a flood filling method.
[0054] In the embodiment of the present application, the gray value of the first pixel in the first gray image of the depth image collected by the depth camera is used to obtain the gradient value of the first pixel, and then the gradient value of the first pixel is used to obtain the key pixel, and the gray value of the key pixel and the gray value of other pixels are used to determine the slope area in the first gray image, and then the slope area of the ground is determined. The present application uses the depth image collected by the depth camera to determine the slope area. The area collected by the depth camera is large. For example, the depth camera can reach 480 lines, and the resolution of the depth camera is high. Compared with the prior art using laser radar (64 lines) to determine the slope area of the ground, the present application can determine the slope area of the ground more quickly and accurately. The present application uses a depth camera to detect the slope area of the ground, which reduces the use cost compared to using a laser radar to detect the slope area. In addition, the use of a depth camera to detect the ground does not require high hardware. For example, the central processing unit (CPU) and memory usage are not high, and low computing power hardware can be selected to reduce resource consumption. The depth camera has no strict restrictions on the installation angle, position and height, and is easy to install. Using a depth camera on the robot to determine the slope area of the ground can improve the robot's perception of the environment and ensure the robot's movement safety.
[0055] like Figure 3 As shown, in a possible implementation manner, the implementation process of step S101 may include:
[0056] S1011, converting the depth image into first point cloud data in a camera coordinate system, where the camera coordinate system is a coordinate system determined based on the posture of the depth camera.
[0057] In this embodiment, the data in the depth image is subjected to dimensionality reduction processing to obtain the first point cloud data in the camera coordinate system. The camera coordinate system includes an x-axis, a y-axis, and a z-axis, where the x-axis and the y-axis represent the coordinates of the pixel points, and the z-axis represents the pixel value (depth) of the pixel point. The depth image is converted into point cloud data in the camera coordinate system in order to prepare for converting the depth image into point cloud data in the standard coordinate system.
[0058] S1012: Convert the first point cloud data into second point cloud data in a standard coordinate system, where the standard coordinate system is a world coordinate system determined based on a horizontal plane.
[0059] In this embodiment, the first point cloud data is subjected to coordinate transformation to obtain the second point cloud data. The x-axis and y-axis of the standard coordinate system are parallel to the horizontal plane and perpendicular to each other; the z-axis in the standard coordinate system is perpendicular to the x-axis and the y-axis. The second point cloud data includes x-axis data, y-axis data, and z-axis data.
[0060] S1013, normalizing the height values in the second point cloud data to obtain normalized height values, where the height values are used to represent the degree to which the ground is higher than the origin of the standard coordinate system.
[0061] In this embodiment, the height value (z-axis data) in the second point cloud data is normalized, for example, normalized to the interval [-0.42m, 0.22m], to obtain the normalized height value, so as to make data processing simple and fast.
[0062] S1014: Obtain a first grayscale image of the depth image based on the normalized height values.
[0063] In this embodiment, the normalized height value is converted to grayscale image data (0-255) to obtain a second grayscale image. The second grayscale image is subjected to median filtering and dimensionality reduction to obtain a first grayscale image.
[0064] Optionally, the first grayscale image is subjected to inverse normalization processing to obtain filtered z-axis data, and the x-axis data, y-axis data and filtered z-axis data in the second point cloud data constitute the filtered point cloud data.
[0065] In the embodiment of the present application, the depth image is converted into a first grayscale image so as to determine the slope area according to the first grayscale image. The point cloud data is converted into an image, and the image is processed quickly, which can reach the millisecond level. Therefore, the slope area can be determined more easily and quickly using the first grayscale image.
[0066] like Figure 4 As shown, in a possible implementation manner, the implementation process of step S103 may include:
[0067] S1031, determining a third pixel point among the first pixel points that meets preset requirements, wherein the preset requirements include that the absolute value of the first-order gradient value is greater than or equal to the first threshold, and the absolute value of the second-order gradient value is less than or equal to the second threshold.
[0068] In this embodiment, the first pixel point with the first value in the third binary image is the third pixel point.
[0069] S1032: Determine a first candidate region in the first grayscale image based on a position of the third pixel in the first grayscale image.
[0070] In this embodiment, the first candidate region is determined in the third binary image, and the region in the first grayscale image corresponding to the third binary image is the first candidate region.
[0071] Specifically, the adjacent third pixel points are classified into one area, and the adjacent third pixel points are adjacent in position, so as to obtain a first candidate area. Adjacent may include adjacent in the x-axis direction, or adjacent in the y-axis direction. For example, the third pixel point A is the pixel point in the i-th row and j-th column in the first grayscale image, the third pixel point B is the pixel point in the i-th row and j+1-th column in the first grayscale image, the third pixel point C is the pixel point in the i+1-th row and j-th column in the first grayscale image, and the third pixel point D is the pixel point in the i+1-th row and j+1-th column in the first grayscale image. The third pixel point A, the third pixel point B, the third pixel point C, and the third pixel point D constitute a first candidate area.
[0072] S1033: Determine a second candidate region based on the area of the first candidate region, wherein the second candidate region is a first candidate region in the first candidate region whose area is greater than or equal to a preset area.
[0073] In this embodiment, the first candidate regions are eroded and expanded to determine the contour of each first candidate region, and then the area of each first candidate region is calculated according to the contour of each first candidate region.
[0074] S1034: Determine the centroid of the second candidate area.
[0075] In this embodiment, the centroid of the second candidate region may be within the second candidate region or outside the second region. Each second candidate region corresponds to a centroid.
[0076] S1035: Determine key pixel points from first pixel points included in the second candidate region based on the positional relationship between the second candidate region and the center of gravity of the second candidate region.
[0077] In this embodiment, different strategies are selected to determine key pixels according to whether the centroid is inside or outside the second candidate region. One centroid can determine one key pixel, so one second candidate region corresponds to one key pixel.
[0078] In an embodiment of the present application, a first candidate area is first determined based on the gradient value of a first pixel point, and then the first candidate area is screened based on its area to obtain a second candidate area, which is a slope area that is more accurate than the first candidate area. Finally, the key pixel points of the second candidate area are determined based on the centroid point of the second candidate area, so that when the slope area is subsequently determined based on the key pixel points, the slope area obtained is more accurate.
[0079] In a possible implementation, the implementation process of step S1035 may include:
[0080] S10351: At a first centroid point in the second candidate area, the key pixel point includes a first pixel point corresponding to the first centroid point.
[0081] In this embodiment, if the centroid of the second candidate region is within the second candidate region, the centroid is recorded as the first centroid. The first pixel corresponding to the first centroid is the key pixel of the second candidate region. The key pixel of the first grayscale image includes the first centroid.
[0082] S10352: Determine a fourth pixel point based on the position of the second centroid point in the first grayscale image and the second centroid point that is not in the second candidate area.
[0083] In this embodiment, the fourth pixel point is the first pixel point in the second candidate area that is in the same row as the second center of gravity point and corresponds to the second center of gravity point, and the first pixel point in the second candidate area that is in the same column as the second center of gravity point and corresponds to the second center of gravity point.
[0084] In this embodiment, if the centroid of the second candidate area is not within the second candidate area, the centroid is recorded as the second centroid. The first pixel point in the same row and column as the second centroid is searched in the first grayscale image, and the first pixel point in the second candidate area and in the same row and column as the second centroid is recorded as the fourth pixel point.
[0085] S10353, determining a fifth pixel point among the fourth pixel points which has the shortest distance from the second centroid point, wherein the key pixel points include the fifth pixel point.
[0086] In this embodiment, the distance between each fourth pixel and the second centroid is calculated to obtain each distance value. The fourth pixel corresponding to the minimum value among all distance values is recorded as the fifth pixel, and the key pixel of the second candidate area is the fifth pixel. The key pixel of the first grayscale image includes the fifth pixel.
[0087] In a possible implementation, the implementation process of step S104 may include:
[0088] A first difference between the grayscale value of the key pixel and the grayscale value of the second pixel is calculated.
[0089] If the first difference is within a preset interval, the slope area corresponding to the key pixel point is obtained based on the position of the second pixel point corresponding to the first difference within the preset interval in the first grayscale image, and the slope area in the first grayscale image includes the slope areas corresponding to all key pixel points.
[0090] In this embodiment, the slope area is determined by using a flood filling method according to key pixel points.
[0091] Specifically, each key pixel is taken as the center and gradually calculated and diffused outward. Specifically, with the key pixel as the center, the grayscale difference between the key pixel and the second pixel is calculated in sequence in the direction from near to far from the key pixel, and the grayscale difference is recorded as the first difference. When the first difference is within the preset interval, the diffusion continues, and the second pixel corresponding to the first difference within the preset interval and the corresponding key pixel form a slope area. When the first difference is not within the preset interval, the diffusion stops, and the final slope area is obtained.
[0092] The preset interval can be set as needed. A key pixel point corresponds to a slope area, and there may be intersections between different slope areas. The point cloud data of the first pixel point in the slope area is determined based on the second point cloud data.
[0093] If the first difference is not within the preset interval, the second pixel point corresponding to the first difference not within the preset interval and the corresponding key pixel point cannot form a slope area.
[0094] like Figure 5 As shown, in a possible implementation, after step S104, the method may further include:
[0095] S201, performing plane fitting processing on the slope region to obtain a fitting plane of the slope region.
[0096] In this embodiment, a plane fitting process is performed on each slope region to obtain a fitting plane of each slope region.
[0097] S202, calculating a slope angle between the fitting plane and a horizontal plane, where the slope angle between the fitting plane and the horizontal plane is a slope angle of the slope area.
[0098] S203, for a sixth pixel point, the sixth pixel point is the first pixel point in the first pixel points that exists in a slope area, and the slope angle of the slope area where the sixth pixel point is located is the slope angle of the sixth pixel point.
[0099] In this embodiment, if the first pixel point exists in a slope region, the first pixel point is recorded as the sixth pixel point. The slope angle of the sixth pixel point is the slope angle of the slope region where the sixth pixel point is located.
[0100] S204: For the seventh pixel point, the seventh pixel point is the first pixel point among the first pixel points that exists in at least two slope areas, and the maximum value of the slope angle in the slope area including the seventh pixel point is used as the slope angle of the seventh pixel point.
[0101] In this embodiment, if the first pixel point exists in at least two slope areas, the first pixel point is recorded as the seventh pixel point. The maximum value of the slope angles of the at least two slope areas is determined, and the maximum value of the slope angles of the at least two slope areas is used as the slope angle of the seventh pixel point.
[0102] For example, if the seventh pixel is in the slope area A and the slope area B, the slope angle of the slope area A is 20 degrees, the slope angle of the slope area B is 24 degrees, and the slope angle of the seventh pixel is 24 degrees.
[0103] In this embodiment, the slope angle of the slope area can be determined by a plane fitting method, and then the slope angle of each first pixel point in the slope area can be determined according to the slope angle, so as to output the slope angle of each first pixel point in the slope area and obtain richer slope data.
[0104] In a possible implementation, after step S103, the method may further include:
[0105] A second difference between the grayscale value of the first pixel point in the i-1th row and jth column and the grayscale value of the first pixel point in the i-th row and jth column in the first grayscale image is calculated, and the first pixel point in the i-th row and jth column is the first pixel point in the slope area.
[0106] If the second difference is greater than a preset value, it is determined that the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-1-th row and j-th column is downslope. The slope type of the first pixel point in the i-1-th row and j-th column relative to the first pixel point in the i-th row and j-th column is upslope. The preset value may be 0.
[0107] If the second difference is less than the preset value, it is determined that the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-1-th row and j-th column is upslope. The slope type of the first pixel point in the i-1-th row and j-th column relative to the first pixel point in the i-th row and j-th column is downslope.
[0108] If the second difference is equal to the preset value, it is determined that the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-1-th row and j-th column is parallel.
[0109] In this embodiment, a third difference value of the grayscale value of the first pixel point in the i-th row and j-th column minus the grayscale value of the first pixel point in the i-th row and j-1 column may also be calculated. If the third difference value is greater than a preset value, the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-th row and j-1 column is upslope. If the third difference value is less than a preset value, the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-th row and j-1 column is downslope. If the third difference value is equal to the preset value, the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-th row and j-1 column is parallel.
[0110] In this embodiment, if the depth camera is installed on the robot, the depth image is an image of the ground in front taken when the robot moves forward or needs to move forward. The first pixel point of the i-th row is closer to the depth camera than the first pixel point of the i-1th row. If the first pixel point of the i-th row and j-th column is a point in the slope area, the grayscale value of the first pixel point of the i-1th row and j-th column is calculated minus the grayscale value of the first pixel point of the i-th row and j-th column to obtain a fourth difference. If the fourth difference is greater than the preset value, the slope type of the first pixel point of the i-th row and j-th column is uphill, that is, when the robot moves to the first pixel point of the i-th row and j-th column, if it continues to move, it needs to go uphill. If the fourth difference is less than the preset value, the slope type of the first pixel point of the i-th row and j-th column is downhill, that is, when the robot moves to the first pixel point of the i-th row and j-th column, if it continues to move, it needs to go downhill.
[0111] In the embodiment of the present application, the slope type of each first pixel in the slope area can be determined according to the grayscale value of the pixel, and then the slope type of the first pixel in the slope area is output to obtain richer slope data.
[0112] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0113] Corresponding to the slope detection method described in the above embodiment, Figure 6 A structural block diagram of a slope detection device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.
[0114] Reference Figure 6 The device 300 may include: an image acquisition module 310, a gradient calculation module 320, a key point determination module 330 and a slope detection module 340.
[0115] The image acquisition module 310 is used to acquire a first grayscale image of a depth image acquired by a depth camera, wherein the depth image includes the ground;
[0116] A gradient calculation module 320, configured to calculate a gradient value of a first pixel point based on a grayscale value of the first pixel point in the first grayscale image, where the number of the first pixel points is multiple;
[0117] A key point determination module 330, configured to determine a key pixel point from the first pixel points based on a gradient value of the first pixel point;
[0118] The slope detection module 340 is used to determine the slope area in the first grayscale image based on the grayscale value of the key pixel point and the grayscale value of the second pixel point, wherein the second pixel point is the first pixel point in the first grayscale image other than the key pixel point, and the slope area in the first grayscale image is used to determine the slope area of the ground.
[0119] In a possible implementation, the gradient value of the first pixel point includes a first-order gradient value and a second-order gradient value;
[0120] The key point determination module 330 can be specifically used for:
[0121] Determine a third pixel point among the first pixel points that meets preset requirements, wherein the preset requirements include that an absolute value of a first-order gradient value is greater than or equal to a first threshold value, and an absolute value of a second-order gradient value is less than or equal to a second threshold value;
[0122] Determine a first candidate region in the first grayscale image based on a position of the third pixel in the first grayscale image;
[0123] Determine a second candidate region based on the area of the first candidate region, wherein the second candidate region is a first candidate region in the first candidate region with an area greater than or equal to a preset area;
[0124] Determine the centroid of the second candidate area;
[0125] Based on the positional relationship between the second candidate region and the centroid of the second candidate region, a key pixel point is determined from the first pixel points included in the second candidate region.
[0126] In a possible implementation, the key point determination module 330 may be specifically used to:
[0127] In a first centroid point of the second candidate area, the key pixel points include a first pixel point corresponding to the first centroid point;
[0128] The second centroid point is not within the second candidate area, and a fourth pixel point is determined based on the position of the second centroid point in the first grayscale image, wherein the fourth pixel point is the first pixel point in the second candidate area that is in the same row as the second centroid point and corresponds to the second centroid point, and the first pixel point in the second candidate area that is in the same column as the second centroid point and corresponds to the second centroid point;
[0129] A fifth pixel point among the fourth pixel points having the shortest distance from the second centroid point is determined, and the key pixel points include the fifth pixel point.
[0130] In a possible implementation, the slope detection module 340 may be specifically used to:
[0131] Calculating a first difference between the grayscale value of the key pixel and the grayscale value of the second pixel;
[0132] If the first difference is within a preset interval, the slope area corresponding to the key pixel point is obtained based on the position of the second pixel point corresponding to the first difference within the preset interval in the first grayscale image, and the slope area in the first grayscale image includes the slope areas corresponding to all key pixel points.
[0133] In a possible implementation, the image acquisition module 310 may be specifically used for:
[0134] Converting the depth image into first point cloud data in a camera coordinate system, where the camera coordinate system is a coordinate system determined based on the position and posture of the depth camera;
[0135] Converting the first point cloud data into second point cloud data in a standard coordinate system, where the standard coordinate system is a world coordinate system determined based on a horizontal plane;
[0136] Normalizing the height value in the second point cloud data to obtain a normalized height value, wherein the height value is used to represent the degree to which the ground is higher than the origin of the standard coordinate system;
[0137] Based on the normalized height values, a first grayscale image of the depth image is obtained.
[0138] In a possible implementation, the slope detection module 340 is also connected to:
[0139] A plane fitting module, used for performing plane fitting processing on the slope area to obtain a fitting plane of the slope area;
[0140] The angle calculation module is used to calculate the slope angle between the fitting plane and the horizontal plane, where the slope angle between the fitting plane and the horizontal plane is the slope angle of the slope area.
[0141] In a possible implementation, the angle calculation module is also connected to:
[0142] A first angle determination module is used for determining a sixth pixel point, the sixth pixel point being the first pixel point in a slope region among the first pixel points, and the slope angle of the slope region where the sixth pixel point is located is the slope angle of the sixth pixel point;
[0143] The second angle determination module is used to determine the slope angle of the seventh pixel point, which is the first pixel point among the first pixel points that exists in at least two slope areas, and to take the maximum value of the slope angle in the slope area including the seventh pixel point as the slope angle of the seventh pixel point.
[0144] In a possible implementation, the slope detection module 340 is also connected to:
[0145] a difference calculation module, used for calculating a first difference between a grayscale value of a first pixel point in the i-1th row and jth column and a grayscale value of a first pixel point in the i-th row and jth column in the first grayscale image, wherein the first pixel point in the i-th row and jth column is a pixel point in the slope area;
[0146] A first judgment module, configured to determine that the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-1-th row and j-th column is downslope if the first difference is greater than a preset value;
[0147] A second judgment module is used for determining that the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-1-th row and j-th column is uphill if the first difference is less than the preset value;
[0148] The third judgment module is used to determine that the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-1-th row and j-th column is parallel if the first difference is equal to the preset value.
[0149] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0150] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0151] The present application also provides a terminal device, see Figure 7 The terminal device 400 may include: at least one processor 410, a memory 420, and a computer program stored in the memory 420 and executable on the at least one processor 410. When the processor 410 executes the computer program, the steps in any of the above-mentioned method embodiments are implemented, for example Figure 2 Steps S101 to S104 in the illustrated embodiment. Alternatively, when the processor 410 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Figure 6 The functions of modules 310 to 340 are shown.
[0152] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 420, and executed by the processor 410 to complete the present application. The one or more modules / units may be a series of computer program segments capable of completing specific functions, and the program segments are used to describe the execution process of the computer program in the terminal device 400.
[0153] Those skilled in the art will understand that Figure 7 It is only an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as input and output devices, network access devices, buses, etc.
[0154] The processor 410 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0155] The memory 420 may be an internal storage unit of the terminal device, or an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. The memory 420 is used to store the computer program and other programs and data required by the terminal device. The memory 420 may also be used to temporarily store data that has been output or is to be output.
[0156] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.
[0157] The slope detection method provided in the embodiment of the present application can be applied to terminal devices such as computers, tablet computers, laptop computers, netbooks, personal digital assistants (PDAs), etc. The embodiment of the present application does not impose any restrictions on the specific type of terminal devices.
[0158] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0159] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0160] In the embodiments provided in the present application, it should be understood that the disclosed terminal equipment, devices and methods can be implemented in other ways. For example, the terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0161] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0162] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0163] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by one or more processors, the steps of each of the above-mentioned method embodiments can be implemented.
[0164] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by one or more processors, the steps of each of the above-mentioned method embodiments can be implemented.
[0165] Similarly, as a computer program product, when the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.
[0166] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.
[0167] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A slope detection method, characterized in that: include: Acquire a first grayscale image of a depth image acquired by a depth camera, wherein the depth image includes a ground surface; Calculating a gradient value of a first pixel point based on a grayscale value of the first pixel point in the first grayscale image, where the number of the first pixel points is multiple; Determining a key pixel from the first pixel based on a gradient value of the first pixel; Determine a slope area in the first grayscale image based on a grayscale value of the key pixel and a grayscale value of a second pixel, wherein the second pixel is a first pixel other than the key pixel in the first grayscale image, and the slope area in the first grayscale image is used to determine the slope area of the ground; The determining of the slope area in the first grayscale image based on the grayscale value of the key pixel and the grayscale value of the second pixel comprises: Calculating a first difference between the grayscale value of the key pixel and the grayscale value of the second pixel; If the first difference is within a preset interval, the slope area corresponding to the key pixel point is obtained based on the position of the second pixel point corresponding to the first difference within the preset interval in the first grayscale image, and the slope area in the first grayscale image includes the slope areas corresponding to all key pixel points.
2. The slope detection method according to claim 1, characterized in that: The gradient value of the first pixel point includes a first-order gradient value and a second-order gradient value; The determining a key pixel point from the first pixel point based on the gradient value of the first pixel point includes: Determine a third pixel point among the first pixel points that meets preset requirements, wherein the preset requirements include that an absolute value of a first-order gradient value is greater than or equal to a first threshold value, and an absolute value of a second-order gradient value is less than or equal to a second threshold value; Determine a first candidate region in the first grayscale image based on a position of the third pixel in the first grayscale image; Determine a second candidate region based on the area of the first candidate region, wherein the second candidate region is a first candidate region in the first candidate region with an area greater than or equal to a preset area; Determine the centroid of the second candidate area; Based on the positional relationship between the second candidate region and the centroid of the second candidate region, a key pixel point is determined from the first pixel points included in the second candidate region.
3. The slope detection method according to claim 2, characterized in that: The determining of the key pixel point from the first pixel points included in the second candidate area based on the positional relationship between the second candidate area and the center of gravity of the second candidate area includes: In a first centroid point of the second candidate area, the key pixel points include a first pixel point corresponding to the first centroid point; The second centroid point is not within the second candidate area, and a fourth pixel point is determined based on the position of the second centroid point in the first grayscale image, wherein the fourth pixel point is the first pixel point in the second candidate area that is in the same row as the second centroid point and corresponds to the second centroid point, and the first pixel point in the second candidate area that is in the same column as the second centroid point and corresponds to the second centroid point; A fifth pixel point among the fourth pixel points having the shortest distance from the second centroid point is determined, and the key pixel points include the fifth pixel point.
4. The slope detection method according to any one of claims 1 to 3, characterized in that: The step of acquiring a first grayscale image of a depth image acquired by a depth camera includes: Converting the depth image into first point cloud data in a camera coordinate system, where the camera coordinate system is a coordinate system determined based on the position and posture of the depth camera; Converting the first point cloud data into second point cloud data in a standard coordinate system, where the standard coordinate system is a world coordinate system determined based on a horizontal plane; Normalizing the height value in the second point cloud data to obtain a normalized height value, wherein the height value is used to represent the degree to which the ground is higher than the origin of the standard coordinate system; Based on the normalized height values, a first grayscale image of the depth image is obtained.
5. The slope detection method according to any one of claims 1 to 3, characterized in that: After determining the slope area in the first grayscale image based on the grayscale value of the key pixel and the grayscale value of the second pixel, the method further includes: Performing plane fitting processing on the slope area to obtain a fitting plane of the slope area; The slope angle between the fitting plane and the horizontal plane is calculated, and the slope angle between the fitting plane and the horizontal plane is the slope angle of the slope area.
6. The slope detection method according to claim 5, characterized in that: After calculating the slope angle between the fitting plane and the horizontal plane, the method further includes: For a sixth pixel point, the sixth pixel point is the first pixel point in a slope region among the first pixel points, and the slope angle of the slope region where the sixth pixel point is located is the slope angle of the sixth pixel point; For the seventh pixel point, the seventh pixel point is the first pixel point among the first pixel points that exists in at least two slope areas, and the maximum value of the slope angle in the slope area including the seventh pixel point is used as the slope angle of the seventh pixel point.
7. The slope detection method according to any one of claims 1 to 3, characterized in that: After determining the slope area in the first grayscale image based on the grayscale value of the key pixel and the grayscale value of the second pixel, the method further includes: Calculate a second difference between the grayscale value of the first pixel point in the i-1th row and jth column and the grayscale value of the first pixel point in the i-th row and jth column in the first grayscale image, wherein the first pixel point in the i-th row and jth column is the first pixel point in the slope area; If the second difference is greater than a preset value, determining that the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-1-th row and j-th column is downslope; If the second difference is less than the preset value, determining that the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-1-th row and j-th column is upslope; If the second difference is equal to the preset value, it is determined that the slope type of the first pixel point in the i-th row and j-th column relative to the first pixel point in the i-1-th row and j-th column is parallel.
8. A robot comprising a depth camera, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the slope detection method according to any one of claims 1 to 7 is implemented.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the slope detection method according to any one of claims 1 to 7 is implemented.