Road surface flatness detection method, calculation device, and computer readable medium

Abstract

The invention provides a road surface flatness detection method, an arithmetic device, and a computer readable medium. In the method, depth information is obtained. The depth information comprises depth values corresponding to the plurality of pixels. And converting the depth information into a height relation graph. The height values of those pixels in the height relationship map are converted from corresponding depth values, and the height value of each pixel is related to the height of the ground. And determining flatness categories corresponding to the pixels through a semantic segmentation model according to the height relation graph. This semantic segmentation model is trained for associations between one or more reference heights and corresponding flatness categories. Therefore, the road condition can be effectively detected.

Description

Technical Field

[0001] The present invention relates to a detection technology, and in particular to a road surface flatness detection method, a computing device and a computer-readable medium. Background Art

[0002] With the increase of the elderly population, how to provide the elderly with a safer auxiliary means of transportation is one of the important issues today. Therefore, the detection of road surface information becomes particularly important, so as to understand whether the road surface is flat and thus improve the safety of auxiliary means of transportation. Summary of the invention

[0003] The present invention provides a road surface flatness detection method, a computing device and a computer-readable medium, which perform road surface detection by integrating stereo vision technology and deep learning technology.

[0004] The method for detecting the flatness of a road surface according to an embodiment of the present invention includes (but is not limited to) the following steps: obtaining depth information. The depth information includes depth values ​​corresponding to a plurality of pixels. Converting the depth information into a height relationship graph. The height values ​​of the pixels in the height relationship graph are converted from the corresponding depth values, and the height value of each pixel is related to the height of the ground. Determining the flatness categories corresponding to the pixels through a semantic segmentation model based on the height relationship graph. The semantic segmentation model is trained to associate one or more reference heights with the corresponding flatness categories.

[0005] The computing device of the embodiment of the present invention includes a memory and a processor. The memory stores program code. The processor is coupled to the memory. The processor loads the program code and executes the following steps: obtain depth information, convert the depth information into a height relationship graph, and determine the flatness categories corresponding to those pixels through a semantic segmentation model based on the height relationship graph. The depth information includes depth values ​​corresponding to multiple pixels. The height values ​​of those pixels in the height relationship graph are converted from the corresponding depth values, and the height value of each pixel is related to the height of the ground. The semantic segmentation model is trained to associate one or more reference heights with the corresponding flatness categories.

[0006] The non-transitory computer-readable medium of the embodiment of the present invention is loaded with program code via a processor to execute the following steps: obtain depth information, convert the depth information into a height relationship graph, and determine the flatness categories corresponding to those pixels through a semantic segmentation model based on the height relationship graph. The depth information includes depth values ​​corresponding to a plurality of pixels. The height values ​​of those pixels in the height relationship graph are converted from the corresponding depth values, and the height value of each pixel is related to the height of the ground. The semantic segmentation model is trained to associate one or more reference heights with the corresponding flatness categories.

[0007] Based on the above, according to the road surface flatness detection method, computing device and computer readable medium of the embodiment of the present invention, the depth map is converted into a height relationship map, and the flatness type is identified through the speech segmentation model. In this way, the number of sensors can be greatly reduced, and more accurate road surface information (for example, road surface height data or distance information) can be obtained. In addition, it can distinguish whether it is a dangerous protrusion or its degree of depression, and determine whether it is a slope.

[0008] In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a block diagram of components of a computing device according to an embodiment of the present invention.

[0010] Figure 2 is a flow chart of a method for detecting the flatness of a road surface according to an embodiment of the present invention.

[0011] Figure 3 is a schematic diagram of depth values, height values, and projection angles according to an embodiment of the present invention.

[0012] Figure 4A and Figure 4B FIG. 4 is a schematic diagram illustrating flatness detection and verification according to an embodiment of the present invention.

[0013] Figure 4C and Figure 4D FIG. 4 is a schematic diagram illustrating flatness detection and verification according to another embodiment of the present invention.

[0014] Figure 5 is a flow chart of angle function decision making according to an embodiment of the present invention.

[0015] Fig. 6A FIG. 4 is a schematic diagram of a slope upward angle according to an embodiment of the present invention.

[0016] Figure 6B FIG. 4 is a schematic diagram of a downward slope angle according to an embodiment of the present invention.

[0017] The following are the descriptions of the reference numerals:

[0018] 10: Computing device

[0019] 11: Storage

[0020] 12: Processor

[0021] 50: Depth Camera

[0022] S210~S230, S510~S530: Steps

[0023] D, D1~D4: Depth value

[0024] CH: Sensing height

[0025] DIS: Distance

[0026] R1: Area to be tested

[0027] θ: Projection angle

[0028] 401: Slope Up

[0029] 402: Bump

[0030] 403: Depression

[0031] 404: Slope Down

[0032] α: Slope upward angle

[0033] HA1: First reference height

[0034] HA2: Second reference altitude

[0035] DIS1: First distance

[0036] DIS2: Second distance

[0037] β: Downward slope angle

[0038] HA3: Third reference height

[0039] HA4: Fourth reference height

[0040] DIS3: The third distance

[0041] DIS4: The fourth distance DETAILED DESCRIPTION

[0042] Figure 1 is a block diagram of components of a computing device 10 according to an embodiment of the present invention. Figure 1 The computing device 10 includes (but is not limited to) a memory 11 and a processor 12. The computing device 10 may be one or more desktop computers, notebook computers, smart phones, tablet computers, wearable devices, servers or other electronic devices.

[0043] The memory 11 may be any type of fixed or removable random access memory (RAM), read only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD) or similar element. In one embodiment, the memory 11 is used to store program code, software modules, configurations, data or files (e.g., depth values, height values, or calculation values), and will be described in detail in subsequent embodiments.

[0044] The processor 12 is coupled to the memory 11. The processor 12 may be a central processing unit (CPU), a graphics processing unit (GPU), or other programmable general-purpose or special-purpose microprocessor, a digital signal processor (DSP), a programmable controller, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a neural network accelerator, or other similar components or a combination of the above components. In one embodiment, the processor 12 is used to execute all or part of the operations of the computing device 10, and can load and execute various program codes, software modules, files, and data stored in the memory 11.

[0045] In one embodiment, the computing device 10 is connected to the depth camera 50 by wire or wirelessly. For example, through Wi-Fi, mobile network, Bluetooth or other wireless communication technology, or through Ethernet, fiber optic network, USB or other wired communication technology. In another embodiment, the computing device 10 has a built-in or external depth camera 50. The depth camera 50 can be a sensor based on time of flight (ToF), structured light or radar. In one embodiment, the depth camera 50 is used to obtain depth information. The depth information (or depth map) includes depth values ​​corresponding to multiple pixels. For example, based on the specifications of the sensing element of the depth camera 50, the depth information is a depth map of 1920×1080 pixels, and each pixel corresponds to a depth value. The depth value reflects the distance of the depth camera 50 relative to the object. It should be noted that the pixel quantity and size of the depth information can still be changed according to actual needs.

[0046] In some embodiments, the depth camera 50 is mounted on a mobile vehicle. The mobile vehicle can move autonomously or under control. For example, the mobile vehicle is an automated guided vehicle (AGV), an autonomous mobile robot (AMR), an electric vehicle, a car, or a bicycle.

[0047] Hereinafter, the method described in the embodiment of the present invention will be described with various devices, elements and modules in the computing device 10. Each process of the method can be adjusted according to the implementation situation, and is not limited thereto.

[0048] Figure 2 FIG. 1 is a flow chart of a method for detecting the flatness of a road surface according to an embodiment of the present invention. Figure 2 , the processor 12 obtains the depth information (step S210). As described above, the depth information includes depth values ​​corresponding to a plurality of pixels and can be referred to as a depth map. In one embodiment, the depth camera 50 can shoot a road surface. That is, the road surface is within the field of view (FOV) of the depth camera 50. In addition, in some embodiments, the depth camera 50 mounted on a mobile vehicle can shoot different areas of the road surface while moving.

[0049] The processor 12 converts the depth information into a height relationship map (step S220). Specifically, the height relationship map includes a plurality of pixels corresponding to the depth information. For example, if the depth information is a depth map of 1920×1080 pixels, the height relationship map includes height values ​​of 1920×1080 pixels.

[0050] The height values ​​of multiple pixels in the height relationship graph are converted from the corresponding depth values. The depth values ​​of one or more pixels can be converted into height values. The height value of each pixel in the height relationship graph is related to the height of the ground. The depression, protrusion and / or inclination of the ground are factors that affect the flatness of the ground. These factors will cause a height difference relative to the (flat) ground. Therefore, the height value can become an indicator for judging the flatness.

[0051] Figure 3 is a schematic diagram of depth value, height value and projection angle according to an embodiment of the present invention. Figure 3 , the sensing height CH is the height of the depth camera 50 for obtaining the depth information relative to the ground or the height relative to the ground when obtaining the depth information, the depth value D is the depth value of a certain pixel (hereinafter referred to as the first pixel) in the depth information corresponding to the area to be measured R1 (for example, a point or an area on the road surface, an obstacle, or any object), and the distance DIS is the distance between the standing point of the depth camera 50 on the road surface and the area to be measured R1.

[0052] In one embodiment, the processor 12 may determine the projection height of the depth value of the first pixel among the plurality of pixels under the projection angle. Figure 3 For example, the processor 12 may define the depth value D of the first pixel as the length of the first side of the triangle, define the projection height as the length of the second side of the triangle, and define the projection angle θ as the angle between the first side and the second side. In other words, the projection height is the length of the first side projected onto the second side under the condition that the first side and the second side have the projection angle θ.

[0053] The processor 12 can determine the height value of the first pixel in the height relationship diagram according to the difference between the sensed height CH and the projected height. In other words, the difference between the sensed height CH and the projected height is the height value. The mathematical expression of the height value H is:

[0054] H=CH-D*Cos(θ)…(1).

[0055] Since the sensing height CH is a known length, the difference between the sensing height CH and the projection height D*Cos(θ) (i.e., the difference between the sensing height CH and the projection height) can reflect the height of the test area R1 relative to the road surface where the depth camera 50 is located, and it can be known whether the test area R1 is lower than, higher than, or equal to higher than the road surface where the depth camera 50 is located. For example, if the projection height is equal to the sensing height, the test area R1 is equal to higher than the road surface where the depth camera 50 is located. If the projection height is less than the sensing height, the test area R1 is higher than the road surface where the depth camera 50 is located. If the projection height is greater than the sensing height, the test area R1 is lower than the road surface where the depth camera 50 is located.

[0056] In one embodiment, the processor 12 may determine the distance DIS of the detection area R1 corresponding to the first pixel according to the projection angle and the first side (the side corresponding to the depth value D). The mathematical expression of the distance DIS is:

[0057] DIS=D*Sin(θ)…(2).

[0058] If the to-be-detected region R1 is part of a suspended object or an obstacle, the above formula (2) can be used to determine the distance from the suspended object or the obstacle.

[0059] The projection angle is obtained based on the position of the first pixel among the plurality of pixels in the depth map. In one embodiment, the processor 12 may obtain reference depth information. The reference depth information includes depth values ​​of a plurality of pixels when the ground is flat. The reference depth information is depth information sensed by the depth camera 50 at a defined sensing height on a flat ground in advance. The depth map of the reference depth information reflects the depth value of the flat ground.

[0060] The processor 12 matches the reference depth information with multiple reference angles to obtain a matching result. Each pixel in the matching result matches a reference angle. The reference depth information is a depth map obtained by sensing a flat road surface. The processor 12 can define the depth value of any pixel in the reference depth information to correspond to the length of the hypotenuse of the reference triangle, and define the sensing height to correspond to the length of the adjacent side of the reference triangle. The angle between the hypotenuse and the adjacent side is the reference angle. The processor 12 can determine the corresponding reference angle based on the length of the hypotenuse and the length of the adjacent side. Since the reference depth information is sensed for a flat ground, the depth values ​​of all pixels on the same horizontal line in the depth map corresponding to the reference depth information are the same. In addition, those pixels on the same horizontal line also correspond to the same reference angle, so that the pixels are matched to the corresponding reference angles. By determining the reference angles corresponding to all horizontal lines in the depth map, all pixels in the reference depth information can be matched to the corresponding reference angles.

[0061] Next, the processor 12 may determine the projection angle corresponding to the first pixel based on the matching result. For example, the processor 12 may obtain the coordinates of the first pixel corresponding to the vertical axis in the two-dimensional coordinates, and find the reference angle corresponding to the same horizontal line from the matching result based on the horizontal line corresponding to this coordinate. The found reference angle may be used as the projection angle of the first pixel.

[0062] Please refer to Figure 2 , the processor 12 determines the flatness categories corresponding to those pixels through a semantic segmentation model based on the height relationship graph (step S230). Specifically, the semantic segmentation algorithm can be used to classify all pixels in the image. Compared with the object detection algorithm, the semantic segmentation algorithm is not limited to the bounding box and can determine the categories corresponding to all pixels separately. Since the road surface may have irregular shapes, the semantic segmentation algorithm is more suitable for road surface detection than the object detection algorithm and provides better computing efficiency. The semantic segmentation algorithm can be TriSDNet, DeconvNet, ReNet, pix2pix, HRNet or other algorithms.

[0063] The semantic segmentation model is trained to find the association between one or more reference heights and the corresponding flatness category. Based on the semantic segmentation algorithm, the relationship between the training sample and the corresponding mark or real result can be analyzed to obtain the law, so as to predict the unknown data through the law. For the preparation of the training sample, the road surface is actually sensed at a specific sensing height by the depth camera 50, and the depth information for training is obtained. The reference heights of all pixels in the reference height relationship diagram are converted from the corresponding depth values ​​in the depth information for training. For the conversion of depth value to reference height, reference can be made to the above-mentioned conversion of depth value to height value, which will not be repeated here. In addition, the corresponding real flatness category is marked for each pixel in the reference height relationship diagram. The semantic segmentation model is a machine learning model constructed after learning based on the above-mentioned training samples and the marked results / real results, and is used to infer the evaluation data (for example, the height relationship diagram obtained in step S220). In some application scenarios, the trained semantic segmentation model can reach the preset accuracy standard.

[0064] In one embodiment, the flatness categories predicted / output by the semantic segmentation model include convex, concave, slope up, and slope down. Concave means that there are more cases where the surface is higher than the flat road surface in a shorter distance. Concave means that there are more cases where the surface is lower than the flat road surface in a shorter distance. Slope up means that there is a gradual increase in height compared to the flat road surface in a longer distance. Slope down means that there is a gradual decrease in height compared to the flat road surface in a longer distance. In some embodiments, the flatness category further includes background. Background means non-road surface or objects beyond the senseable distance.

[0065] Figure 4A and Figure 4B is a schematic diagram illustrating flatness detection and verification according to an embodiment of the present invention. Figure 4A and Figure 4B , Figure 4A is the actual road surface corresponding to the depth image taken by the depth camera 50. Figure 4B For Figure 4A The upward slope 401 and the bump 402 detected from the actual road surface. The upward slope 401 may correspond to the upward slope of the actual road surface, and the bump 402 may correspond to the upward step of the actual road surface. In addition, the distance relative to the upward slope and the upward step may be determined based on formula (2).

[0066] Figure 4C and Figure 4D FIG. 1 is a schematic diagram illustrating flatness detection and verification according to another embodiment of the present invention. Figure 4C and Figure 4D , Figure 4C is the actual road surface corresponding to the depth image taken by the depth camera 50. Figure 4D For Figure 4CThe depression 403 and the downward slope 404 detected on the actual road surface. The depression 403 may correspond to the downward step of the actual road surface, and the downward slope 404 may correspond to the downward slope of the actual road surface. In addition, the distance relative to the downward slope and the downward step may be determined based on formula (2).

[0067] In one embodiment, the processor 12 may divide the multiple pixels in the height relationship diagram into multiple regions, and determine the flatness categories corresponding to the pixels in these regions respectively through the semantic segmentation model. Since multiple flatness categories may exist simultaneously within the sensing range of the depth camera 50, the sensing range may be divided into multiple sub-ranges. These sub-ranges correspond to the aforementioned multiple regions respectively. For example, the sensing range is divided into left, middle and right regions on average. For multiple regions, when a convexity or concave is detected in any region, it is helpful for the user to select the direction corresponding to other regions to continue moving, thereby avoiding collision with obstacles. It should be noted that the number and size of the regions can be changed according to actual needs.

[0068] Figure 5 is a flowchart of the angle function decision according to one embodiment of the present invention. Figure 5 , the processor 12 can determine the area to be tested (such as Figure 3 In response to determining that the flatness category of the area to be tested is slope upward, the processor 12 can determine the slope upward angle of the area to be tested by a first function corresponding to the slope upward (step S520). The slope upward angle is the upward inclination angle of the area to be tested relative to the flat ground. Fig. 6A is a schematic diagram of the upward slope angle α according to an embodiment of the present invention. Fig. 6A , the first function is:

[0069]

[0070] α is the upward slope angle, HA1 is the first reference height, HA2 is the second reference height, DIS1 is the first distance corresponding to the first reference height (that is, the projection length of the edge corresponding to the depth value D1 at the upward slope angle α), and DIS2 is the second distance corresponding to the second reference height (that is, the projection length of the edge corresponding to the depth value D2 at the upward slope angle α). The first reference height HA1 is a statistical value (for example, the mean, median, or mode) of the height values ​​of all pixels on a horizontal line on the vertical axis. The first reference height HA1 may be greater than 3 centimeters. For example, 3.5, 4, or 5 centimeters. The second reference height HA2 is a statistical value (for example, the mean, median, or mode) of the height values ​​of all pixels on another horizontal line on the vertical axis. The second reference height HA2 is greater than the first reference height HA1. The second reference height HA2 may be greater than 5 centimeters. For example, 5.5, 6, or 7 centimeters.

[0071] In response to determining that the flatness category of the area to be tested is slope downward, the processor 12 may determine the slope downward angle of the area to be tested by a second function corresponding to the slope downward (step S530). The slope downward angle is the downward inclination angle of the area to be tested relative to the flat ground. Figure 6B is a schematic diagram of a slope downward angle β according to an embodiment of the present invention. Figure 6B , the second function is:

[0072]

[0073] β is the downward slope angle, HA3 is the third reference height, HA4 is the fourth reference height, DIS3 is the third distance corresponding to the third reference height (that is, the projection length of the edge corresponding to the depth value D3 at the downward slope angle β), and DIS4 is the fourth distance corresponding to the fourth reference height (that is, the projection length of the edge corresponding to the depth value D4 at the downward slope angle β). The third reference height HA3 is a statistical value (for example, the mean, median or mode) of the height values ​​of all pixels on a horizontal line on the vertical axis. The third reference height HA3 may be less than -3 centimeters. For example, -3.5, -4 or -5 centimeters. The fourth reference height HA4 is a statistical value (for example, the mean, median or mode) of the height values ​​of all pixels on another horizontal line on the vertical axis. The fourth reference height HA4 is less than the third reference height HA3. The fourth reference height HA4 may be less than -5 centimeters. For example, -5.5, -6, or -7 centimeters. In this way, the inclination of the slope can be known.

[0074] In one embodiment, the processor 12 may determine the first reference height and the second reference height or determine the third reference height and the fourth reference height according to the error of the depth camera 50 that obtains the depth information. If the error is smaller, the spacing between the horizontal lines corresponding to the first reference height and the second reference height on the vertical axis is larger, or the spacing between the horizontal lines corresponding to the third reference height and the fourth reference height on the vertical axis is larger. If the error is larger, the spacing between the horizontal lines corresponding to the first reference height and the second reference height on the vertical axis is smaller, or the spacing between the horizontal lines corresponding to the third reference height and the fourth reference height on the vertical axis is smaller.

[0075] In one embodiment, the processor 12 may generate a control instruction for controlling a mobile vehicle on which the depth camera 50 is installed or other mobile vehicles according to the height relationship graph and its corresponding flatness category. The control instruction is related to the moving speed, rotation and / or stopping of the mobile vehicle. For example, the raised area is divided into 3 to 5 cm, 5 to 8 cm, and more than 8 cm higher than the flat road surface. Based on the height relationship diagram, the height and distance of the raised area in front can be obtained accordingly (for example, based on the above formulas (1) and (2)), and the following three situations can be assumed: (1) For a raised area higher than 3 to 5 centimeters, the control instruction can be to pass directly without decelerating; (2) For a raised area higher than 5 to 8 centimeters, the control instruction can be: decelerate by half when approaching to 1 meter away from the raised area, and stop when approaching to 0.5 meter away from the raised area; (3) For a raised area higher than 8 centimeters, the control instruction can be: decelerate by half when approaching to 1.5 meters away from the raised area, and stop when approaching to 1 meter away from the raised area.

[0076] For another example, the sunken areas are divided into 3 to 5 cm, 5 to 8 cm, and more than 8 cm lower than the flat road surface. Based on the height relationship diagram, the height and distance of the sunken area ahead can be known accordingly, and the following three situations can be assumed: (1) For sunken areas lower than 3 to 5 cm, the control instruction may be to pass directly without deceleration; (2) For sunken areas lower than 5 to 8 cm, the control instruction may be: slow down by half when approaching to 1 meter away from the sunken area, and stop when approaching to 0.5 meters away from the sunken area; (3) For sunken areas lower than 8 cm or more, the control instruction may be: slow down by half when approaching to 1.5 meters away from the sunken area, and stop when approaching to 1 meter away from the sunken area. The control instruction may also be to turn to avoid the raised or sunken area. However, the content of the control instruction can be adjusted according to actual needs.

[0077] Another embodiment of the present invention provides a computer readable medium, which records a computer program loaded into a processor to execute each step of the above-mentioned flatness detection method (such as Figure 2 to Figure 6B The computer program may include a plurality of program codes, and after the processor 12 loads and executes the program codes, the above-mentioned skeleton correction method and the functions of the computing device 10 can be completed.

[0078] In summary, in the road surface flatness detection method, computing device, and computer-readable medium of the embodiments of the present invention, the depth information is converted into a height relationship graph, and the flatness category corresponding to each pixel in the height relationship graph is identified through a semantic segmentation model. In this way, the road surface flatness detection can be performed through the depth information of a single depth camera, thereby reducing the number of sensors. The embodiments of the present invention can distinguish the types of ups and downs of the road surface, and can obtain the degree and spacing of the ups and downs, thereby assisting personnel or mobile vehicles to move, thereby avoiding accidents. In addition, compared with the object detection algorithm, the embodiments of the present invention are more suitable for application scenarios of road surface detection.

[0079] Although the present invention has been disclosed as above by embodiments, it is not intended to limit the present invention. Any ordinary technician in the field can make some modifications and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope defined by the attached claims.

Claims

1. A method for detecting the flatness of a road surface, comprising: Obtaining depth information, wherein the depth information includes depth values ​​corresponding to a plurality of pixels; Converting the depth information into a height relationship map, wherein the height values ​​of the plurality of pixels in the height relationship map are converted from the corresponding depth values, and the height value of each of the pixels is related to the height of the ground; as well as The flatness categories corresponding to the plurality of pixels are determined by a semantic segmentation model according to the height relationship graph, wherein the semantic segmentation model is trained to determine the relationship between at least one reference height and the corresponding flatness category.

2. The method for detecting the flatness of a road surface as claimed in claim 1, wherein the step of converting the depth information into the height relationship diagram comprises: determining a projection height of a depth value of a first pixel among the plurality of pixels at a projection angle, wherein the projection angle is derived based on a position of the first pixel among the plurality of pixels; and The height value of the first pixel in the height relationship map is determined according to a difference between a sensed height and the projected height, wherein the sensed height is a height relative to the ground when the depth information is obtained.

3. The method for detecting the flatness of a road surface as claimed in claim 2, further comprising: defining the depth value of the first pixel as the length of a first side of the triangle; defining the projected height as the length of the second side of the triangle; as well as The projection angle is defined as the included angle between the first side and the second side.

4. The method for detecting the flatness of a road surface as claimed in claim 3, further comprising: The distance of the to-be-detected area corresponding to the first pixel is determined according to the projection angle and the first side.

5. The method for detecting the flatness of a road surface as claimed in claim 2, further comprising: Acquire reference depth information, wherein the reference depth information includes depth values ​​of the plurality of pixels when the ground is flat; Matching the reference depth information with a plurality of reference angles to obtain a matching result, wherein each of the pixels in the matching result matches the reference angle; as well as The projection angle corresponding to the first pixel is determined according to the matching result.

6. The road surface flatness detection method as claimed in claim 1, wherein the flatness categories predicted by the semantic segmentation model include convex, concave, upward slope and downward slope.

7. The method for detecting the flatness of a road surface as claimed in claim 1, wherein the step of determining the flatness categories corresponding to the plurality of pixels by using the semantic segmentation model comprises: Dividing the plurality of pixels into a plurality of regions; as well as The smoothness categories corresponding to the pixels in the multiple regions are determined respectively by the semantic segmentation model.

8. The method for detecting the flatness of a road surface as claimed in claim 1, further comprising: Determine the flatness category of the area to be tested as slope upward; as well as The upward slope angle of the area to be tested is determined by a first function corresponding to the upward slope, wherein the first function is: α is the upward angle of the slope, HA1 is a first reference height, HA2 is a second reference height, DIS1 is a first distance corresponding to the first reference height, and DIS2 is a second distance corresponding to the second reference height.

9. The method for detecting the flatness of a road surface as claimed in claim 1, further comprising: Determine the flatness category of the area to be tested as slope downward; as well as The downward slope angle of the area to be tested is determined by a second function corresponding to the upward slope, wherein the second function is: β is the downward angle of the slope, HA3 is the third reference height, HA4 is the fourth reference height, DIS3 is the third distance corresponding to the third reference height, and DIS4 is the fourth distance corresponding to the fourth reference height.

10. The method for detecting the flatness of a road surface according to claim 1, further comprising: A control instruction for controlling a mobile vehicle is generated according to the height relationship graph and its corresponding flatness category, wherein the control instruction is related to at least one of a moving speed, a rotation and a stop of the mobile vehicle.

11. A computing device, comprising: Memory, storing program codes; as well as A processor, coupled to the memory, loads the program code and executes: Obtaining depth information, wherein the depth information includes depth values ​​corresponding to a plurality of pixels; Converting the depth information into a height relationship map, wherein the height values ​​of the plurality of pixels in the height relationship map are converted from the corresponding depth values, and the height value of each of the pixels is related to the height of the ground; as well as The flatness categories corresponding to the plurality of pixels are determined by a semantic segmentation model according to the height relationship graph, wherein the semantic segmentation model is trained to determine the relationship between at least one reference height and the corresponding flatness category.

12. The computing device of claim 11, wherein the processor further executes: determining a projection height of a depth value of a first pixel among the plurality of pixels at a projection angle, wherein the projection angle is derived based on a position of the first pixel among the plurality of pixels; and The height value of the first pixel in the height relationship map is determined according to a difference between a sensed height and the projected height, wherein the sensed height is a height of a depth camera for obtaining the depth information relative to the ground.

13. The computing device of claim 12, wherein the processor further executes: defining the depth value of the first pixel as the length of a first side of the triangle; defining the projected height as the length of the second side of the triangle; and The projection angle is defined as the included angle between the first side and the second side.

14. The computing device of claim 13, wherein the processor further executes: The distance of the to-be-detected area corresponding to the first pixel is determined according to the projection angle and the first side.

15. The computing device of claim 12, wherein the processor further executes: Acquire reference depth information, wherein the reference depth information includes depth values ​​of the plurality of pixels when the ground is flat; Matching the reference depth information with a plurality of reference angles to obtain a matching result, wherein each of the pixels in the matching result matches the reference angle; and The projection angle corresponding to the first pixel is determined according to the matching result. 16 . The computing device of claim 11 , wherein the flatness categories predicted by the semantic segmentation model include convex, concave, sloping up, and sloping down.

17. The computing device of claim 11, wherein the processor further executes: dividing the plurality of pixels into a plurality of regions; and The smoothness categories corresponding to the pixels in the multiple regions are determined respectively by the semantic segmentation model.

18. The computing device of claim 11, wherein the processor further executes: Determine that the flatness category of the area to be tested is slope up; and The upward slope angle of the area to be tested is determined by a first function corresponding to the upward slope, wherein the first function is: α is the upward angle of the slope, HA1 is a first reference height, HA2 is a second reference height, DIS1 is a first distance corresponding to the first reference height, and DIS2 is a second distance corresponding to the second reference height.

19. The computing device of claim 11, wherein the processor further executes: Determine that the flatness category of the area to be tested is slope downward; and The downward slope angle of the area to be tested is determined by a second function corresponding to the upward slope, wherein the second function is: β is the downward angle of the slope, HA3 is the third reference height, HA4 is the fourth reference height, DIS3 is the third distance corresponding to the third reference height, and DIS4 is the fourth distance corresponding to the fourth reference height.

20. A non-transitory computer readable medium, loaded with program code via a processor to perform the following steps: Obtaining depth information, wherein the depth information includes depth values ​​corresponding to a plurality of pixels; Converting the depth information into a height relationship map, wherein the height values ​​of the plurality of pixels in the height relationship map are converted from the corresponding depth values, and the height value of each of the pixels is related to the height of the ground; and The flatness categories corresponding to the plurality of pixels are determined by a semantic segmentation model according to the height relationship graph, wherein the semantic segmentation model is trained to determine the relationship between at least one reference height and the corresponding flatness category.

Images