Method and device for determining driving area of movable equipment, and electronic equipment
By collecting images and ranging data on autonomous driving equipment and identifying bumps, the problem of difficulty in identifying small obstacles in the prior art is solved, and more accurate driving area determination and higher driving comfort are achieved.
Patent Information
- Application Number
- CN201910257541.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-04-01
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2039-04-01
AI Technical Summary
Existing obstacle avoidance technology is difficult to identify small obstacles, such as lifted manhole covers and road bumps, and cannot effectively avoid these small obstacles, affecting the driving safety and comfort of autonomous driving equipment.
By collecting the current frame image on the movable device, determining the non-flat area, dividing the sampling point set and obtaining the height information set, calculating the bump degree to determine the driving area, and using visual detection and ranging sensor technology to identify and filter flat that does not affect the driving Area data reduces calculation noise and improves the accuracy of bump degree calculation.
It can better identify smaller obstacles, provide more reasonable driving routes and speed control, and improve the driving comfort and obstacle avoidance capabilities of autonomous driving equipment.
Smart Images

Figure CN111767761B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method for determining a driving area of a movable device, a determining device and an electronic device. Background Art
[0002] For mobile devices with driving functions, the importance of obstacle avoidance technology is self-evident. Especially in the field of autonomous driving, existing obstacle avoidance technology can only identify large obstacles such as pedestrians and vehicles, but cannot identify small obstacles such as opened manhole covers and potholes on the road. Summary of the invention
[0003] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a method for determining a driving area of a movable device, a determining device and an electronic device.
[0004] On the one hand, an embodiment of the present application provides a method for determining a driving area of a movable device, the method for determining a driving area of a movable device comprising: determining a first area based on a current frame image captured by an image acquisition device on the movable device; determining a first sampling point set based on the first area, and determining a first height information set corresponding to the first sampling point set, wherein the first height information set corresponds one-to-one to the first sampling point set; determining a second height information set corresponding to the first sampling point set, wherein the second height information set corresponds one-to-one to the first sampling point set; determining a bumpiness of the first area based on the first height information set and the second height information set; and determining the driving area of the movable device based on the bumpiness of the first area.
[0005] On the other hand, an embodiment of the present application provides a device for determining a driving area of a movable device, the device for determining a driving area of the movable device comprising: a first area determination module, used to determine a first area based on a current frame image acquired by an image acquisition device on the movable device; a first height information set determination module, used to determine a first sampling point set based on the first area, and determine a first height information set corresponding to the first sampling point set, wherein the first height information set corresponds one-to-one to the first sampling point set; a second height information set determination module, used to determine a second height information set corresponding to the first sampling point set, wherein the second height information set corresponds one-to-one to the first sampling point set; a bumpiness determination module, used to determine the bumpiness of the first area based on the first height information set and the second height information set; and a driving area determination module, used to determine the driving area of the movable device based on the bumpiness of the first area.
[0006] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program for executing the method for determining the driving area of a movable device mentioned in the above embodiment.
[0007] On the other hand, an embodiment of the present application provides an electronic device, which includes: a processor and a memory for storing instructions executable by the processor, wherein the processor is used to execute the method for determining the driving area of a movable device mentioned in the above embodiment.
[0008] The method for determining the driving area of a movable device provided in the embodiment of the present application achieves the purpose of determining the driving area of the movable device based on the current frame image collected, by determining the first area based on the current frame image collected, determining the first sampling point set based on the first area, then determining the first height information set and the second height information set corresponding to the first sampling point set, and determining the bumpiness of the first area based on the first height information set and the second height information set, and finally determining the driving area of the movable device based on the bumpiness of the first area. Since the embodiment of the present application uses the current frame image collected to determine the bumpiness of the first area, and determines the driving area of the movable device based on the bumpiness of the first area, compared with the existing method for determining the driving area of the movable device, the embodiment of the present application can better identify obstacles with relatively small volumes, thereby providing a more reasonable driving route for the movable device, and can more reasonably control the driving speed of the movable device, thereby improving the driving comfort of the movable device. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0010] Figure 1 It is the scene graph to which this application is applicable.
[0011] Figure 2 It is a flowchart of a method for determining a driving area of a movable device provided by an exemplary embodiment of the present application.
[0012] Figure 3 It is a schematic diagram of a flow chart of determining a first height information set corresponding to a first sampling point set provided by another exemplary embodiment of the present application.
[0013] Figure 4It is a schematic diagram of a flow chart of determining a second height information set corresponding to a first sampling point set provided by another exemplary embodiment of the present application.
[0014] Figure 5 It is a schematic diagram of a flow chart of determining a second world coordinate parameter set corresponding to a first sampling point set based on a second sampling point set provided by yet another exemplary embodiment of the present application.
[0015] Figure 6 It is a schematic diagram of a flow chart of determining the degree of bumpiness of a first area based on a first height information set and a second height information set provided by yet another exemplary embodiment of the present application.
[0016] Figure 7 It is a schematic diagram of a flow chart of determining a first world coordinate parameter set of a first sampling point set according to a ranging sensor provided by yet another exemplary embodiment of the present application.
[0017] Figure 8 It is a schematic diagram of a flow chart of determining a first world coordinate parameter set of a first sampling point set according to a ranging sensor provided by yet another exemplary embodiment of the present application.
[0018] Fig. 9 It is a schematic diagram of a flow chart of determining a second world coordinate parameter set corresponding to a first sampling point set based on a second sampling point set provided by yet another exemplary embodiment of the present application.
[0019] Fig.10 It is a schematic diagram of a flow chart of determining a second world coordinate parameter set corresponding to a first sampling point set based on a second sampling point set provided by yet another exemplary embodiment of the present application.
[0020] Fig.11 It is a flowchart of determining the driving area of a movable device based on the bumpiness of a first area provided by another exemplary embodiment of the present application.
[0021] Fig.12 It is a structural schematic diagram of a device for determining a driving area of a movable device provided by an exemplary embodiment of the present application.
[0022] Fig.13 It is a structural schematic diagram of a first height information set determination module of a device for determining a driving area of a movable equipment provided by another exemplary embodiment of the present application.
[0023] Fig.14 It is a structural schematic diagram of a second height information set determination module of a device for determining a driving area of a movable device provided by another exemplary embodiment of the present application.
[0024] Fig.15It is a structural schematic diagram of a second world coordinate parameter set determination unit of a device for determining a driving area of a movable device provided by yet another exemplary embodiment of the present application.
[0025] Fig.16 It is a structural schematic diagram of a bumpiness determination module of a device for determining a driving area of a movable equipment provided by yet another exemplary embodiment of the present application.
[0026] Fig.17 It is a structural schematic diagram of a first world coordinate parameter set determination unit of a device for determining a driving area of a movable device provided by yet another exemplary embodiment of the present application.
[0027] Fig.18 It is a structural schematic diagram of a first world coordinate parameter set determination unit of a device for determining a driving area of a movable device provided by yet another exemplary embodiment of the present application.
[0028] Fig.19 It is a structural schematic diagram of a second world coordinate parameter set determination unit of a device for determining a driving area of a movable device provided by yet another exemplary embodiment of the present application.
[0029] Fig. 20 It is a structural schematic diagram of a second world coordinate parameter set determination unit of a device for determining a driving area of a movable device provided by yet another exemplary embodiment of the present application.
[0030] Fig.21 It is a structural diagram of a driving area determination module of a device for determining a driving area of a movable device provided by yet another exemplary embodiment of the present application.
[0031] Fig. 22 It is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION
[0032] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.
[0033] Application Overview
[0034] As we all know, for movable devices with moving functions, such as cars with automatic driving functions and intelligent robots with automatic walking functions, obstacle avoidance technology is an important guarantee technology for realizing moving functions.
[0035] At present, obstacles in the driving area of mobile equipment are usually identified based on LiDAR technology. The technical principle of identifying obstacles based on LiDAR technology is: first, a high-precision map is established based on LiDAR technology, and then the distance, direction, height and even shape of the obstacle are obtained based on the point cloud data in the high-precision map. However, due to the low density of the point cloud data formed by LiDAR technology, LiDAR technology can only identify relatively large obstacles, such as pedestrians and vehicles, but cannot identify smaller obstacles such as opened manhole covers and potholes on the road.
[0036] In view of the above technical problems, the basic concept of the present application is to propose a method, a device and an electronic device for determining the driving area of a movable device. The method for determining the driving area of the movable device determines the first area based on the current frame image collected, determines the first sampling point set based on the first area, then determines the first height information set and the second height information set corresponding to the first sampling point set, determines the bumpiness of the first area based on the first height information set and the second height information set, and finally determines the driving area of the movable device based on the bumpiness of the first area, thereby achieving the purpose of determining the driving area of the movable device based on the current frame image collected. Since the embodiment of the present application determines the bumpiness of the first area using the current frame image collected, and determines the driving area of the movable device according to the bumpiness of the first area, compared with the existing method for determining the driving area of the movable device, the embodiment of the present application can better identify obstacles with relatively small volumes, thereby providing a more reasonable driving route for the movable device, and can more reasonably control the driving speed of the movable device, thereby improving the driving comfort of the movable device.
[0037] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.
[0038] Exemplary Systems
[0039] Figure 1 is the scene graph to which this application applies. Figure 1 As shown, the scenario to which the present application is applicable is a scenario for determining a driving area of a movable device, wherein the scenario for determining the driving area includes a server 1 and a movable device 2, and the server 1 obtains a current frame image captured by an image acquisition device on the movable device 2, and determines the driving area of the movable device based on the obtained current frame image.
[0040] Specifically, the mobile device 2 is used to collect the current frame image; the server 1 is used to determine the first area based on the current frame image collected by the image acquisition device on the mobile device 2, and then determine the first sampling point set based on the first area, and determine the first height information set corresponding to the first sampling point set, wherein the first height information set corresponds to the first sampling point set one-to-one, and then determine the second height information set corresponding to the first sampling point set, wherein the second height information set corresponds to the first sampling point set one-to-one, and finally determine the bumpiness of the first area based on the first height information set and the second height information set, and determine the driving area of the mobile device based on the bumpiness of the first area. Through this scenario, the amount of calculation of the mobile device 2 can be reduced.
[0041] It should be noted that the present application is also applicable to another scenario. Specifically, the scenario for determining the driving area includes a movable device 2. Specifically, the image acquisition device on the movable device 2 captures the current frame image, and determines the first area based on the captured current frame image, and then determines the first sampling point set based on the first area, and determines the first height information set corresponding to the first sampling point set, wherein the first height information set corresponds one-to-one to the first sampling point set, and then determines the second height information set corresponding to the first sampling point set, wherein the second height information set corresponds one-to-one to the first sampling point set, and finally determines the bumpiness of the first area based on the first height information set and the second height information set, and determines the driving area of the movable device based on the bumpiness of the first area. Through this scenario, the amount of calculation of the movable device 2 can be reduced. Through this scenario, it can be ensured that the movable device 2 can determine the driving area in real time.
[0042] Exemplary Methods
[0043] Figure 2 FIG. 1 is a flow chart of a method for determining a driving area of a movable device provided by an exemplary embodiment of the present application. Figure 2 As shown, the method for determining the driving area of a movable device provided in an embodiment of the present application includes the following steps.
[0044] Step 10: determine a first area based on a current frame image captured by an image acquisition device on the movable device.
[0045] The first area refers to a non-flat area. That is, in step 10, based on the collected current frame image, a visual detection method is used to determine the non-flat area in the current frame image.
[0046] The specific visual detection method used to determine the non-flat area can be set according to the actual situation of the current frame image, and the embodiment of the present invention does not impose a unified limitation on this. For example, the non-flat area of the current frame image is determined based on a preset neural network model.
[0047] Step 20: determine a first sampling point set based on the first area, and determine a first height information set corresponding to the first sampling point set, wherein the first height information set corresponds to the first sampling point set in a one-to-one manner.
[0048] In step 20, the first sampling point set may be determined by dividing the first area into n*m grids and using each grid point as the first sampling point, wherein n and m are both integers greater than or equal to 1, and the specific values of n and m may be determined according to the actual situation of the current frame image and the first area, and the embodiment of the present application does not impose a uniform limitation on this.
[0049] In addition, the first height information set corresponding to the first sampling point set mentioned in step 20 is determined based on the first area of the current frame image. That is, the first height information set is a height information set corresponding to the first sampling point set determined based on visual detection technology.
[0050] Step 30: determine a second height information set corresponding to the first sampling point set, where the second height information set corresponds to the first sampling point set in a one-to-one manner.
[0051] Step 40: Determine the degree of bumpiness of the first area based on the first height information set and the second height information set.
[0052] Step 50: determining a driving area of the movable device based on the bumpiness of the first area.
[0053] In actual application, the non-flat area (i.e., the first area) is first determined based on the current frame image captured by the image acquisition device on the movable device, and then a first sampling point set is determined based on the non-flat area, and a first height information set corresponding to the first sampling point set is determined, and then a second height information set corresponding to the first sampling point set is determined, and the bumpiness of the non-flat area is determined based on the first height information set and the second height information set, and finally the driving area of the movable device is determined based on the bumpiness of the non-flat area.
[0054] The method for determining the driving area of a movable device provided in the embodiment of the present application achieves the purpose of determining the driving area of the movable device based on the current frame image collected, by determining the first area based on the current frame image collected, determining the first sampling point set based on the first area, then determining the first height information set and the second height information set corresponding to the first sampling point set, and determining the bumpiness of the first area based on the first height information set and the second height information set, and finally determining the driving area of the movable device based on the bumpiness of the first area. Since the embodiment of the present application uses the current frame image collected to determine the bumpiness of the first area, and determines the driving area of the movable device based on the bumpiness of the first area, compared with the existing method for determining the driving area of the movable device, the embodiment of the present application can better identify obstacles with relatively small volumes, thereby providing a more reasonable driving route for the movable device, and can more reasonably control the driving speed of the movable device, thereby improving the driving comfort of the movable device.
[0055] In addition, since the embodiment of the present application first uses a visual detection method to determine the non-flat area in the current frame image, and then calculates the bumpiness of the driving area based on the determined non-flat area, therefore, compared with the method of directly calculating the bumpiness of the driving area based on the current frame image, the embodiment of the present application can fully filter the data of the flat area that does not affect driving, thereby fully reducing the amount of calculation and calculation noise, thereby fully improving the calculation accuracy of the bumpiness of the driving area.
[0056] Figure 3 FIG. 1 is a flow chart of determining a first height information set corresponding to a first sampling point set provided by another exemplary embodiment of the present application. Figure 2 The present application is extended based on the embodiment shown Figure 3 The embodiment shown is described below in detail. Figure 3 The embodiment shown and Figure 2 The differences and similarities of the illustrated embodiments are not described in detail.
[0057] like Figure 3 As shown, in the method for determining the driving area of a movable device provided in an embodiment of the present application, the step of determining a first height information set corresponding to a first sampling point set includes the following steps.
[0058] Step 21: determine a first world coordinate parameter set of a first sampling point set according to a ranging sensor.
[0059] In step 21, the ranging sensor can determine the first world coordinate parameter set corresponding to the first sampling point set based on binocular vision technology, laser radar technology, or RGBD camera technology, and the embodiment of the present application does not make a unified limitation on this.
[0060] Step 22: Determine a first height information set based on the first world coordinate parameter set.
[0061] It should be understood that each sampling point in the first sampling point set corresponds to a first world coordinate parameter in the first world coordinate parameter set, and the corresponding first world coordinate parameter includes corresponding first height information.
[0062] In actual application, the non-flat area (i.e., the first area) is first determined based on the current frame image captured by the image acquisition device on the movable device, and then a first sampling point set is determined based on the non-flat area, and a first world coordinate parameter set of the first sampling point set is determined according to the ranging sensor, and then a first height information set corresponding to the first sampling point set is determined based on the first world coordinate parameter set, and then a second height information set corresponding to the first sampling point set is determined, and the bumpiness of the non-flat area is determined based on the first height information set and the second height information set, and finally the driving area of the movable device is determined based on the bumpiness of the non-flat area.
[0063] The method for determining the driving area of a movable device provided in an embodiment of the present application achieves the purpose of determining the precise height information set corresponding to the first sampling point set by using a ranging sensor to determine the first world coordinate parameter set of the first sampling point set, and determining the first height information set based on the first world coordinate parameter set, thereby providing a prerequisite for improving the accuracy of the bumpiness of the first area determined based on the first height information set and the second height information set.
[0064] Figure 4 FIG. 1 is a flow chart of determining a second height information set corresponding to a first sampling point set provided by another exemplary embodiment of the present application. Figure 2 The present application is extended based on the embodiment shown Figure 4 The embodiment shown is described below in detail. Figure 4 The embodiment shown and Figure 2 The differences and similarities of the illustrated embodiments are not described in detail.
[0065] like Figure 4 As shown, in the method for determining the driving area of a movable device provided in an embodiment of the present application, the step of determining the second height information set corresponding to the first sampling point set (ie, step 30) includes the following steps.
[0066] Step 31: determine the boundary range of the first area.
[0067] Step 32: determine whether the boundary range of the first area meets the preset conditions.
[0068] The preset condition can be determined based on the accuracy of the two calculation methods (i.e., the calculation method based on the flat area and the calculation method based on the elevation angle data) mentioned in the following steps. For example, the preset condition is that the area of the first area does not exceed 60% of the total area. When the area of the first area (i.e., the non-flat area) is too large, it is difficult to use the calculation method based on the flat area to accurately determine the second world coordinate parameter set corresponding to the first sampling point set. In this case, the calculation method based on the elevation angle data can more accurately determine the second world coordinate parameter set corresponding to the first sampling point set.
[0069] In step 32, determine whether the boundary range of the first area meets the preset conditions. When it is determined that the boundary range of the first area meets the preset conditions, execute steps 33 to 36; when it is determined that the boundary range of the first area does not meet the preset conditions, execute steps 37 and 38.
[0070] Step 33: determine a second area based on the current frame image.
[0071] The second area refers to a flat area. That is, in step 33, based on the collected current frame image, a visual detection method is used to determine the flat area in the current frame image.
[0072] Similarly, the specific visual detection method used to determine the flat area can be set according to the actual situation of the current frame image, and the embodiment of the present invention does not impose a unified limitation on this. For example, the flat area of the current frame image is determined based on a preset neural network model.
[0073] Step 34: determine a second sampling point set based on the second area.
[0074] In step 34, the second sampling point set may be determined by dividing the second area into a plurality of grids and using each grid point as the second sampling point. The number of divided grids and the size of each grid may be determined according to the actual situation of the current frame image and the second area, and the embodiment of the present application does not impose a uniform limitation on this.
[0075] Step 35: Determine a second world coordinate parameter set corresponding to the first sampling point set based on the second sampling point set.
[0076] Since the second sampling point set is determined based on the second area (i.e., the flat area), and the first sampling point set is determined based on the first area (i.e., the non-flat area), the second world coordinate parameter set corresponding to the first sampling point set can be predicted based on the second sampling point set, that is, the world coordinate parameter set corresponding to the first sampling point set in the flat case is predicted.
[0077] Step 36: Determine a second height information set based on the second world coordinate parameter set.
[0078] Step 37, determining the elevation angle data of the image acquisition device on the mobile device when capturing the current frame image.
[0079] Step 38: Determine a second altitude information set based on the elevation angle data.
[0080] In the actual calculation process, since the elevation angle data is the angle data between the slope and the horizontal plane simplified in the current driving area, the second height information set corresponding to the first sampling point set can be predicted by using the elevation angle data.
[0081] In actual application, firstly, a non-flat area (i.e., a first area) is determined based on a current frame image captured by an image acquisition device on a movable device, then a first sampling point set is determined based on the non-flat area, and a first height information set corresponding to the first sampling point set is determined, then a boundary range of the non-flat area is determined and it is judged whether the boundary range of the non-flat area meets a preset condition, when it is judged that the boundary range of the non-flat area meets the preset condition, a flat area is determined based on the current frame image and a second sampling point set is determined based on the flat area, then a second world coordinate parameter set corresponding to the first sampling point set is determined based on the second sampling point set, and a second height information set is determined based on the second world coordinate parameter set, finally, a bumpiness of the non-flat area is determined based on the first height information set and the second height information set, and a driving area of the movable device is determined based on the bumpiness of the non-flat area.
[0082] In addition, when it is determined that the boundary range of the non-flat area does not meet the preset conditions, the elevation data of the image acquisition device on the movable device when shooting the current frame image is determined, and the second height information set is determined based on the elevation data, and finally the bumpiness of the non-flat area is determined based on the first height information set and the second height information set, and the driving area of the movable device is determined based on the bumpiness of the non-flat area.
[0083] The method for determining the driving area of a movable device provided in an embodiment of the present application first determines whether the boundary range of the first area meets the preset conditions. When the boundary range of the first area meets the preset conditions, the second height information set of the first sampling point set is determined based on the second area in the current frame image. When the boundary range of the first area does not meet the preset conditions, the second height information set of the first sampling point set is determined based on the elevation data of the image acquisition device on the movable device when shooting the current frame image. This achieves the purpose of selecting the calculation method of the second world coordinate parameter set corresponding to the first sampling point set according to the actual situation of the first area. Figure 2Compared with the illustrated embodiment, the embodiment of the present application can further improve the accuracy of the second world coordinate parameter set corresponding to the first sampling point set, thereby further improving the accuracy of the calculated bumpiness of the first area, and ultimately further improving the obstacle avoidance capability of the movable device, and further improving the driving comfort of the movable device.
[0084] It should be noted that in this application Figure 4 In the illustrated embodiment, step 37 and step 38 may not be included. That is, in another embodiment of the present application, in the step of determining the second height information set corresponding to the first sampling point set (i.e., step 30), steps 31 to 36 are included. That is, in the actual application process, the boundary range of the non-flat area (first area) is first determined, and when the boundary range of the first area meets the preset conditions, the flat area (i.e., the second area) is determined based on the current frame image, and the second height information set corresponding to the first sampling point set of the non-flat area is determined based on the flat area.
[0085] In the embodiment of the present application, since the second height information set corresponding to the first sampling point set in the non-flat area is determined based on the flat area, the second height information set can be determined based on the actual situation of the flat area, thereby making the determined second height information set have higher accuracy.
[0086] In addition, it should be noted that in this application Figure 4 In the illustrated embodiment, steps 33 to 36 may not be included. That is, in the step of determining the second height information set corresponding to the first sampling point set (i.e., step 30), steps 31, 32, 37, and 38 are included. That is, in the actual application process, the boundary range of the non-flat area (first area) is first determined, and when the boundary range of the first area does not meet the preset conditions, the elevation angle data when the image acquisition device on the mobile device captures the current frame image is determined, and the second height information set is determined based on the elevation angle data.
[0087] During the movement of the movable device, there may be a situation where the area of the non-flat area is too large, resulting in a smaller area of the flat area. In this case, it is difficult to implement the method mentioned in the above embodiment to determine the second height information set corresponding to the first sampling point set of the non-flat area based on the flat area. Therefore, the embodiment of the present application improves the wide application of the method for determining the second height information set mentioned in the embodiment of the present application by determining the second height information set corresponding to the first sampling point set based on the elevation data of the image acquisition device when shooting the current frame image.
[0088] Figure 5FIG. 1 is a flow chart of determining a second set of world coordinate parameters corresponding to a first set of sampling points based on a second set of sampling points provided by another exemplary embodiment of the present application. Figure 4 The present application is extended based on the embodiment shown Figure 5 The embodiment shown is described below in detail. Figure 5 The embodiment shown and Figure 4 The differences and similarities of the illustrated embodiments are not described in detail.
[0089] like Figure 5 As shown, in the method for determining the driving area of a movable device provided in an embodiment of the present application, the step of determining the second world coordinate parameter set corresponding to the first sampling point set based on the second sampling point set (ie, step 35) includes the following steps.
[0090] Step 351: Perform a surface fitting operation based on the second sampling points in the second sampling point set to generate a fitting surface.
[0091] Since the second sampling point set is determined based on the second area of the current frame image, and each second sampling point in the second sampling point set corresponds to a world coordinate parameter, then after inputting the world coordinate parameter data corresponding to the multiple second sampling points in the second sampling point set into the standard surface fitting equation, the surface fitting equation corresponding to the second area can be obtained, that is, the fitting surface corresponding to the second area is obtained.
[0092] Step 352: Determine a second world coordinate parameter set based on the fitted surface and the first sampling point set.
[0093] Then, after inputting the coordinate data of each first sampling point in the first sampling point set into the surface fitting equation generated in step 351, the second world coordinate parameter set corresponding to the first sampling point set can be calculated.
[0094] In the actual application process, firstly, a non-flat area (i.e., a first area) is determined based on a current frame image captured by an image acquisition device on the movable device, then a first sampling point set is determined based on the non-flat area, and a first height information set corresponding to the first sampling point set is determined, then a boundary range of the non-flat area is determined and it is judged whether the boundary range of the non-flat area meets a preset condition. When it is judged that the boundary range of the non-flat area meets the preset condition, a flat area is determined based on the current frame image and a second sampling point set is determined based on the flat area, then a surface fitting operation is performed based on a second sampling point in the second sampling point set to generate a fitting surface, and a second world coordinate parameter set is determined based on the fitting surface and the first sampling point set, and then a second height information set is determined based on the second world coordinate parameter set, and finally a bumpiness of the non-flat area is determined based on the first height information set and the second height information set, and a driving area of the movable device is determined based on the bumpiness of the non-flat area.
[0095] The method for determining the driving area of a movable device provided in the embodiment of the present application achieves the purpose of determining the second world coordinate parameter set corresponding to the first sampling point set based on the second sampling point set by performing a surface fitting operation on the second sampling point in the second sampling point set to generate a fitting surface, and then determining the second world coordinate parameter set corresponding to the first sampling point set based on the generated fitting surface. Since the fitting surface is generated based on the second sampling point in the second sampling point set, and the second sampling point set is generated based on the second area (i.e., the flat area) in the current frame image, compared with the second world coordinate parameter set determined based on the elevation angle data, the embodiment of the present application can fully guarantee the accuracy of the second world coordinate parameter set corresponding to the predicted first sampling point set, and can further improve the accuracy of the calculated bumpiness of the first area.
[0096] Figure 6 FIG. 1 is a flow chart of determining the degree of bumpiness of a first area based on a first height information set and a second height information set provided by another exemplary embodiment of the present application. Figure 2 The present application is extended based on the embodiment shown Figure 6 The embodiment shown is described below in detail. Figure 6 The embodiment shown and Figure 2 The differences and similarities of the illustrated embodiments are not described in detail.
[0097] like Figure 6 As shown, in the method for determining the driving area of a movable device provided in an embodiment of the present application, the step of determining the bumpiness of the first area based on the first height information set and the second height information set (ie, step 40) includes the following steps.
[0098] Step 41 : determining a plurality of differences between a first height information set and a corresponding second height information set.
[0099] Since the first height information set and the second height information set both correspond one-to-one to the first sampling point in the first sampling point set, there is also a one-to-one correspondence between the first height information set and the second height information set, and multiple differences between the first height information set and the corresponding second height information set can be determined.
[0100] Step 42: determining a variance parameter between the first height information set and the second height information set based on the plurality of differences, so as to determine the degree of bumpiness of the first area.
[0101] For example, if the first sampling point set includes n first sampling points, the coordinates of the n first sampling points are (x1, y1), (x2, y2)…(x n ,y n ), the first height information set determined based on the first sampling point set is (z1, z2…z n ), the second height information set determined based on the first sampling point set is (s1, s2…s n ). Then, the calculation formula of the variance parameter determined based on the first height information set and the second height information set is:
[0102] P=(z1-s1) 2 +(z2-s2) 2 +…(z n -s n ) 2 (1)
[0103] Formula (1) can calculate the difference between the first height information and the second height information corresponding to each first sampling point, and summarize the differences corresponding to all first sampling points to obtain a variance parameter P for characterizing the degree of difference between the first height information set and the second height information set.
[0104] Since the variance parameter P can measure the difference between the expected second height information set and the actual first height information set, the bumpiness of the first area can be determined based on the variance parameter P between the first height information set and the second height information set. It should be understood that the larger the value of the variance parameter P, the higher the bumpiness of the corresponding first area, and vice versa, the lower the bumpiness of the corresponding first area.
[0105] It should be noted that the calculation method of the variance parameter P is not limited to the above formula (1), as long as it can characterize the degree of difference between the first height information set and the corresponding second height information set. For example, the calculation formula of the variance parameter determined based on the first height information set and the second height information set can also be the following formula (2).
[0106] P=|z1-s1|+|z2-s2|+…|z n -s n | (2)
[0107] In actual application, a non-flat area (i.e., a first area) is first determined based on a current frame image captured by an image acquisition device on the movable device, and then a first sampling point set is determined based on the non-flat area, and a first height information set corresponding to the first sampling point set is determined, and then a second height information set corresponding to the first sampling point set is determined, and then multiple differences between the first height information set and the corresponding second height information set are determined, and a variance parameter between the first height information set and the second height information set is determined based on the multiple differences to determine the bumpiness of the first area, and finally the driving area of the movable device is determined based on the bumpiness of the non-flat area.
[0108] The method for determining the driving area of a movable device provided in the embodiment of the present application achieves the purpose of determining the bumpiness of the first area based on the first height information set and the second height information set by determining multiple differences between the first height information set and the corresponding second height information set, and determining the variance parameter between the first height information set and the second height information set based on the multiple differences. Since the variance parameter can accurately characterize the degree of difference between the expected second height information set and the actual first height information set, the bumpiness of the first area can be accurately determined by using the embodiment of the present application, thereby providing a more reasonable driving route for the movable device.
[0109] Figure 7 FIG. 1 is a flow chart of determining a first world coordinate parameter set of a first sampling point set according to a distance measuring sensor provided by another exemplary embodiment of the present application. Figure 3 The present application is extended based on the embodiment shown Figure 7 The embodiment shown is described below in detail. Figure 7 The embodiment shown and Figure 3 The differences and similarities of the illustrated embodiments are not described in detail.
[0110] like Figure 7 As shown, in the method for determining the driving area of a movable device provided in an embodiment of the present application, the step of determining a first world coordinate parameter set of a first sampling point set according to a ranging sensor (ie, step 21) includes the following steps.
[0111] Step 211: Determine N sets of third world coordinate parameters corresponding to the first sampling point set at N moments according to the ranging sensor, where N is an integer greater than or equal to 1.
[0112] Step 212: Determine a first world coordinate parameter set corresponding to the first sampling point set according to the N third world coordinate parameter sets.
[0113] In actual application, the non-flat area (i.e., the first area) is first determined based on the current frame image captured by the image acquisition device on the movable device, and then a first sampling point set is determined based on the non-flat area, and N third world coordinate parameter sets corresponding to the first sampling point set at N moments are determined according to the ranging sensor, and a first world coordinate parameter set corresponding to the first sampling point set is determined according to the N third world coordinate parameter sets, and then a first height information set corresponding to the first sampling point set is determined based on the first world coordinate parameter set, and then a second height information set corresponding to the first sampling point set is determined, and the bumpiness of the non-flat area is determined based on the first height information set and the second height information set, and finally the driving area of the movable device is determined based on the bumpiness of the non-flat area.
[0114] Since the first sampling point set is determined based on the current frame image captured by the image acquisition device on the mobile device, when the mobile device is in a driving state and the ranging sensor is located on the mobile device, there may be a situation where the measured first world coordinate parameter set is inaccurate. Based on this, the embodiment of the present application uses the ranging sensor to measure multiple third world coordinate parameter sets corresponding to the first sampling point set at multiple different times, and determines the first world coordinate parameter set corresponding to the first sampling point set based on the multiple third world coordinate parameter sets, which effectively avoids the situation where the first world coordinate parameter set measured by the ranging sensor has a large measurement error, thereby further improving the accuracy of the first world coordinate parameter set corresponding to the first sampling point set, and thus providing a prerequisite for finally and accurately determining the driving area of the mobile device.
[0115] Figure 8 FIG. 1 is a flow chart of determining a first world coordinate parameter set of a first sampling point set according to a distance measuring sensor provided by another exemplary embodiment of the present application. Figure 7 The present application is extended based on the embodiment shown Figure 8 The embodiment shown is described below in detail. Figure 8 The embodiment shown and Figure 7 The differences and similarities of the illustrated embodiments are not described in detail.
[0116] like Figure 8As shown, in the method for determining the driving area of a movable device provided in an embodiment of the present application, the step of determining the first world coordinate parameter set corresponding to the first sampling point set (i.e., step 212) according to N third world coordinate parameter sets includes the following steps.
[0117] Step 2121, weighted calculation is performed on the N third world coordinate parameter sets to obtain the first world coordinate parameter set, wherein the closer the movable device is to the first sampling point set among the N moments, the greater the weight of the third world coordinate parameter set corresponding to the moment.
[0118] In actual application, firstly, a non-flat area (i.e., a first area) is determined based on a current frame image captured by an image acquisition device on a movable device, then a first sampling point set is determined based on the non-flat area, N third world coordinate parameter sets corresponding to the first sampling point set at N moments are determined according to a ranging sensor, and weighted calculation is performed on the N third world coordinate parameter sets to obtain a first world coordinate parameter set, then a first height information set corresponding to the first sampling point set is determined based on the first world coordinate parameter set, and then a second height information set corresponding to the first sampling point set is determined, and a bumpiness of the non-flat area is determined based on the first height information set and the second height information set, and finally a driving area of the movable device is determined based on the bumpiness of the non-flat area.
[0119] The method for determining the driving area of a movable device provided in the embodiment of the present application fully considers the influence of the distance factor between the movable device and the first sampling point set on the accuracy of the measured third world coordinate parameter set by defining that the closer the movable device is to the first sampling point set among the N moments, the greater the weight of the third world coordinate parameter set corresponding to the moment. Figure 7 Compared with the illustrated embodiment, the embodiment of the present application can further improve the accuracy of the first world coordinate parameter set determined based on N third world coordinate parameter sets, thereby providing a prerequisite for more accurately determining the driving area of the movable device.
[0120] Fig. 9 FIG. 1 is a flow chart of determining a second set of world coordinate parameters corresponding to a first set of sampling points based on a second set of sampling points provided by another exemplary embodiment of the present application. Figure 4 The present application is extended based on the embodiment shown Fig. 9 The embodiment shown is described below in detail. Fig. 9 The embodiment shown and Figure 4 The differences and similarities of the illustrated embodiments are not described in detail.
[0121] like Fig. 9As shown, in the method for determining the driving area of a movable device provided in an embodiment of the present application, the step of determining the second world coordinate parameter set corresponding to the first sampling point set based on the second sampling point set (ie, step 35) includes the following steps.
[0122] Step 353: Determine M fourth world coordinate parameter sets corresponding to the first sampling point set at M moments based on the second sampling point set, where M is an integer greater than or equal to 1.
[0123] Step 354: Determine a second world coordinate parameter set corresponding to the first sampling point set according to the M fourth world coordinate parameter sets.
[0124] In the actual application process, firstly, a non-flat area (i.e., a first area) is determined based on a current frame image captured by an image acquisition device on the movable device, then a first sampling point set is determined based on the non-flat area, and a first height information set corresponding to the first sampling point set is determined, then a boundary range of the non-flat area is determined and it is judged whether the boundary range of the non-flat area meets a preset condition. When it is judged that the boundary range of the non-flat area meets the preset condition, a flat area is determined based on the current frame image and a second sampling point set is determined based on the flat area, then M fourth world coordinate parameter sets corresponding to the first sampling point set at M moments are determined based on the second sampling point set, and a second world coordinate parameter set corresponding to the first sampling point set is determined according to the M fourth world coordinate parameter sets, then a second height information set is determined based on the second world coordinate parameter set, and finally a bumpiness of the non-flat area is determined based on the first height information set and the second height information set, and a driving area of the movable device is determined based on the bumpiness of the non-flat area.
[0125] The method for determining the driving area of a movable device provided in the embodiment of the present application achieves the purpose of determining the second world coordinate parameter set corresponding to the first sampling point set based on the second sampling point set by determining the M fourth world coordinate parameter sets corresponding to the first sampling point set at M moments based on the second sampling point set, and determining the second world coordinate parameter set corresponding to the first sampling point set based on the M fourth world coordinate parameter sets. Figure 4 Compared with the illustrated embodiment, the embodiment of the present application can fully guarantee the accuracy of the determined second world coordinate parameter set, further avoid the situation where the measured second world coordinate parameter set is inaccurate due to measurement calculation errors, and thus provide a prerequisite for ultimately accurately determining the driving area of the movable device.
[0126] Fig.10 FIG. 1 is a flow chart of determining a second set of world coordinate parameters corresponding to a first set of sampling points based on a second set of sampling points provided by another exemplary embodiment of the present application. Fig. 9The present application is extended based on the embodiment shown Fig.10 The embodiment shown is described below in detail. Fig.10 The embodiment shown and Fig. 9 The differences and similarities of the illustrated embodiments are not described in detail.
[0127] like Fig.10 As shown, in the method for determining the driving area of a movable device provided in an embodiment of the present application, the step of determining the second world coordinate parameter set corresponding to the first sampling point set (i.e., step 354) according to M fourth world coordinate parameter sets includes the following steps.
[0128] Step 3541, weighted calculation is performed on the M fourth world coordinate parameter sets to obtain a second world coordinate parameter set, wherein the moment when the movable device is closer to the first sampling point set among the M moments has a greater weight of the fourth world coordinate parameter set.
[0129] In the actual application process, firstly, a non-flat area (i.e., a first area) is determined based on a current frame image captured by an image acquisition device on the movable device, then a first sampling point set is determined based on the non-flat area, and a first height information set corresponding to the first sampling point set is determined, then a boundary range of the non-flat area is determined and it is judged whether the boundary range of the non-flat area meets a preset condition. When it is judged that the boundary range of the non-flat area meets the preset condition, a flat area is determined based on the current frame image and a second sampling point set is determined based on the flat area, then M fourth world coordinate parameter sets corresponding to the first sampling point set at M moments are determined based on the second sampling point set, and a weighted calculation is performed on the M fourth world coordinate parameter sets to obtain a second world coordinate parameter set, then a second height information set is determined based on the second world coordinate parameter set, and finally a bumpiness of the non-flat area is determined based on the first height information set and the second height information set, and a driving area of the movable device is determined based on the bumpiness of the non-flat area.
[0130] The method for determining the driving area of a movable device provided in the embodiment of the present application fully considers the influence of the distance factor between the movable device and the first sampling point set on the accuracy of the measured fourth world coordinate parameter set by defining that the closer the movable device is to the first sampling point set among the M moments, the greater the weight of the fourth world coordinate parameter set corresponding to the moment. Fig. 9 Compared with the illustrated embodiment, the embodiment of the present application can further improve the accuracy of the second world coordinate parameter set determined based on the M fourth world coordinate parameter sets, thereby providing a prerequisite for more accurately determining the driving area of the movable device.
[0131] Fig.11FIG. 1 is a flow chart of determining the driving area of a movable device based on the bumpiness of a first area provided by another exemplary embodiment of the present application. Figure 2 The present application is extended based on the embodiment shown Fig.11 The embodiment shown is described below in detail. Fig.11 The embodiment shown and Figure 2 The differences and similarities of the illustrated embodiments are not described in detail.
[0132] like Fig.11 As shown, in the method for determining the driving area of a movable device provided in an embodiment of the present application, the step of determining the driving area of the movable device based on the bumpiness of the first area (ie, step 50) includes the following steps.
[0133] Step 51, determining whether the bumpiness of the first area is greater than a first bumpiness threshold.
[0134] In step 51 , when it is determined that the degree of jolting in the first area is greater than the first jolting threshold, step 53 is executed; when it is determined that the degree of jolting in the first area is less than or equal to the first jolting threshold, step 52 is executed.
[0135] Step 52: determine that the first area is a normal driving area of the movable device.
[0136] Step 53: Determine whether the bumpiness of the first area is greater than a second bumpiness threshold.
[0137] In step 53 , when it is determined that the degree of jolting in the first area is greater than the second jolting threshold, step 55 is executed; when it is determined that the degree of jolting in the first area is less than or equal to the second jolting threshold, step 54 is executed.
[0138] Step 54: determine the first area as a deceleration driving area of the movable device.
[0139] Step 55: determine the first area as a prohibited driving area for movable equipment.
[0140] In the embodiment of the present application, the second bump threshold should be greater than the first bump threshold. In addition, the first bump threshold and the second bump threshold can be set according to the actual situation of the movable device and the driving area, and the embodiment of the present application does not make a unified limitation on this.
[0141] It should be noted that, in the deceleration driving area, multiple bump thresholds can also be preset, and the deceleration driving area can be divided into multi-level deceleration areas of different speeds based on the preset multiple bump thresholds, so as to further ensure the driving stability of the movable device and further improve the driving comfort of the movable device.
[0142] In the actual application process, firstly, a non-flat area (i.e., a first area) is determined based on the current frame image captured by the image acquisition device on the movable device, then a first sampling point set is determined based on the non-flat area, and a first height information set corresponding to the first sampling point set is determined, and then a second height information set corresponding to the first sampling point set is determined, and the bumpiness of the non-flat area is determined based on the first height information set and the second height information set, and finally, it is determined whether the bumpiness of the non-flat area is greater than a first bumpiness threshold. When it is determined that the bumpiness of the non-flat area is less than or equal to the first bumpiness threshold, the non-flat area is determined to be a normal driving area of the movable device. When it is determined that the bumpiness of the non-flat area is greater than the first bumpiness threshold, it is determined whether the bumpiness of the non-flat area is greater than the second bumpiness threshold. When it is determined that the bumpiness of the non-flat area is less than or equal to the second bumpiness threshold, the non-flat area is determined to be a deceleration driving area of the movable device. When it is determined that the bumpiness of the non-flat area is greater than the second bumpiness threshold, the non-flat area is determined to be a prohibited driving area of the movable device.
[0143] The method for determining the driving area of a movable device provided in an embodiment of the present application realizes a multi-level division of the driving area by comparing the determined bumpiness of the first area with the preset first and second bumpiness thresholds, and determining the driving area of the movable device according to the comparison result, thereby further improving the driving stability of the movable device and providing a more reasonable driving route for the movable device.
[0144] Exemplary Devices
[0145] Fig.12 FIG. 1 is a schematic diagram of a structure of a device for determining a driving area of a movable device provided by an exemplary embodiment of the present application. Fig.12 As shown, the device for determining the driving area of a movable device provided in an embodiment of the present application includes:
[0146] A first region determining module 100, configured to determine a first region based on a current frame image acquired by an image acquisition device on a movable device;
[0147] A first height information set determining module 200, configured to determine a first sampling point set based on the first area, and determine a first height information set corresponding to the first sampling point set, wherein the first height information set corresponds to the first sampling point set in a one-to-one manner;
[0148] A second height information set determining module 300, configured to determine a second height information set corresponding to the first sampling point set, wherein the second height information set corresponds to the first sampling point set in a one-to-one manner;
[0149] A bumpiness determination module 400, configured to determine a bumpiness of a first area based on a first height information set and a second height information set;
[0150] The driving area determination module 500 is used to determine the driving area of the movable device based on the bumpiness of the first area.
[0151] Fig.13 1 is a schematic diagram of the structure of a first height information set determination module of a device for determining a driving area of a movable device provided by another exemplary embodiment of the present application. Fig.12 The present application is extended based on the embodiment shown Fig.13 The embodiment shown is described below in detail. Fig.13 The embodiment shown and Fig.12 The differences and similarities of the illustrated embodiments are not described in detail.
[0152] like Fig.13 As shown, in the device for determining the driving area of a movable device provided in an embodiment of the present application, the first height information set determining module 200 includes:
[0153] A first world coordinate parameter set determining unit 210, configured to determine a first world coordinate parameter set of a first sampling point set according to a ranging sensor;
[0154] The first height information set determining unit 220 is configured to determine the first height information set based on the first world coordinate parameter set.
[0155] Fig.14 1 is a schematic diagram of the structure of a second height information set determination module of a device for determining a driving area of a movable device provided by another exemplary embodiment of the present application. Fig.12 The present application is extended based on the embodiment shown Fig.14 The embodiment shown is described below in detail. Fig.14 The embodiment shown and Fig.12 The differences and similarities of the illustrated embodiments are not described in detail.
[0156] like Fig.14 As shown, in the device for determining the driving area of a movable device provided in an embodiment of the present application, the second height information set determining module 300 includes:
[0157] A boundary range determining unit 310, configured to determine a boundary range of a first area;
[0158] The boundary range determination unit 320 is used to determine whether the boundary range of the first area meets a preset condition;
[0159] A second region determining unit 330, configured to determine a second region based on the current frame image;
[0160] A second sampling point set determining unit 340, configured to determine a second sampling point set based on the second area;
[0161] A second world coordinate parameter set determining unit 350, configured to determine a second world coordinate parameter set corresponding to the first sampling point set based on the second sampling point set;
[0162] A second height information set determining unit 360, configured to determine a second height information set based on a second world coordinate parameter set;
[0163] The elevation angle data determining unit 370 is used to determine the elevation angle data of the image acquisition device on the movable device when capturing the current frame image;
[0164] The altitude information set determining unit 380 is configured to determine a second altitude information set based on the elevation angle data.
[0165] Fig.15 1 is a schematic diagram of the structure of a second world coordinate parameter set determination unit of a device for determining a driving area of a movable device provided by another exemplary embodiment of the present application. Fig.14 The present application is extended based on the embodiment shown Fig.15 The embodiment shown is described below in detail. Fig.15 The embodiment shown and Fig.14 The differences and similarities of the illustrated embodiments are not described in detail.
[0166] like Fig.15 As shown, in the device for determining the driving area of a movable device provided in an embodiment of the present application, the second world coordinate parameter set determining unit 350 includes:
[0167] The fitting surface generating subunit 3510 is used to perform a surface fitting operation based on the second sampling points in the second sampling point set to generate a fitting surface;
[0168] The second world coordinate parameter set determining subunit 3520 is used to determine the second world coordinate parameter set based on the fitting surface and the first sampling point set.
[0169] Fig.16 1 is a schematic diagram of the structure of a bumpiness determination module of a device for determining a driving area of a movable device provided by another exemplary embodiment of the present application. Fig.12 The present application is extended based on the embodiment shown Fig.16 The embodiment shown is described below in detail. Fig.16 The embodiment shown and Fig.12 The differences and similarities of the illustrated embodiments are not described in detail.
[0170] like Fig.16 As shown, in the device for determining the driving area of a movable device provided in an embodiment of the present application, the bumpiness determination module 400 includes:
[0171] A difference determination unit 410, configured to determine a plurality of differences between a first height information set and a corresponding second height information set;
[0172] The bumpiness determination unit 420 is configured to determine a variance parameter between the first height information set and the second height information set based on a plurality of difference values, so as to determine the bumpiness of the first area.
[0173] Fig.17 1 is a schematic diagram of the structure of a first world coordinate parameter set determination unit of a device for determining a driving area of a movable device provided by another exemplary embodiment of the present application. Fig.13 The present application is extended based on the embodiment shown Fig.17 The embodiment shown is described below in detail. Fig.17 The embodiment shown and Fig.13 The differences and similarities of the illustrated embodiments are not described in detail.
[0174] like Fig.17 As shown, in the device for determining the driving area of a movable device provided in an embodiment of the present application, the first world coordinate parameter set determining unit 210 includes:
[0175] A third world coordinate parameter set determination subunit 2110 is used to determine N third world coordinate parameter sets corresponding to the first sampling point set at N moments according to the ranging sensor;
[0176] The first world coordinate parameter set determining subunit 2120 is configured to determine the first world coordinate parameter set corresponding to the first sampling point set according to the N third world coordinate parameter sets.
[0177] Fig.18 1 is a schematic diagram of the structure of a first world coordinate parameter set determination unit of a device for determining a driving area of a movable device provided by another exemplary embodiment of the present application. Fig.17 The present application is extended based on the embodiment shown Fig.18 The embodiment shown is described below in detail. Fig.18 The embodiment shown and Fig.17 The differences and similarities of the illustrated embodiments are not described in detail.
[0178] like Fig.18 As shown, in the device for determining the driving area of a movable device provided in an embodiment of the present application, the first world coordinate parameter set determining subunit 2120 includes:
[0179] The calculation subunit 21210 is used to perform weighted calculation on the N third world coordinate parameter sets to obtain the first world coordinate parameter set, wherein the closer the movable device is to the first sampling point set among the N moments, the greater the weight of the third world coordinate parameter set corresponding to the moment.
[0180] Fig.19 1 is a schematic diagram of the structure of a second world coordinate parameter set determination unit of a device for determining a driving area of a movable device provided by another exemplary embodiment of the present application. Fig.14 The present application is extended based on the embodiment shown Fig.19 The embodiment shown is described below in detail. Fig.19 The embodiment shown and Fig.14 The differences and similarities of the illustrated embodiments are not described in detail.
[0181] like Fig.19 As shown, in the device for determining the driving area of a movable device provided in an embodiment of the present application, the second world coordinate parameter set determining unit 350 includes:
[0182] The fourth world coordinate parameter set determining subunit 3530 is configured to determine, based on the second sampling point set, M fourth world coordinate parameter sets corresponding to the first sampling point set at M moments;
[0183] The second world coordinate parameter set determining subunit 3540 is configured to determine the second world coordinate parameter set corresponding to the first sampling point set according to the M fourth world coordinate parameter sets.
[0184] Fig. 20 1 is a schematic diagram of the structure of a second world coordinate parameter set determination unit of a device for determining a driving area of a movable device provided by another exemplary embodiment of the present application. Fig.19 The present application is extended based on the embodiment shown Fig. 20 The embodiment shown is described below in detail. Fig. 20 The embodiment shown and Fig.19 The differences and similarities of the illustrated embodiments are not described in detail.
[0185] like Fig. 20 As shown, in the device for determining the driving area of a movable device provided in an embodiment of the present application, the second world coordinate parameter set determining subunit 3540 includes:
[0186] The weighted calculation subunit 35410 is used to perform weighted calculation on the M fourth world coordinate parameter sets to obtain the second world coordinate parameter set, wherein the moment when the movable device is closer to the first sampling point set among the M moments has a greater weight of the fourth world coordinate parameter set.
[0187] Fig.21 1 is a schematic diagram of the structure of a driving area determination module of a device for determining a driving area of a movable device provided by another exemplary embodiment of the present application. Fig.12 The present application is extended based on the embodiment shown Fig.21 The embodiment shown is described below in detail. Fig.21 The embodiment shown and Fig.12 The differences and similarities of the illustrated embodiments are not described in detail.
[0188] like Fig.21 As shown, in the device for determining the driving area of a movable device provided in an embodiment of the present application, the driving area determining module 500 includes:
[0189] A first judging unit 510 is used to determine whether the bumpiness of the first area is greater than a first bumpiness threshold;
[0190] A first determining unit 520, configured to determine that the first area is a normal driving area of the movable device;
[0191] A second judging unit 530 is used to determine whether the bumpiness of the first area is greater than a second bumpiness threshold;
[0192] A second determining unit 540 is used to determine that the first area is a deceleration driving area of the movable device;
[0193] The third determining unit 550 is configured to determine that the first area is a prohibited driving area for movable equipment.
[0194] It should be understood that Figures 8 to 13The present invention provides a first area determination module 100, a first height information set determination module 200, a second height information set determination module 300, a bumpiness determination module 400 and a driving area determination module 500 in the driving area determination device of the provided movable device, and a first world coordinate parameter set determination unit 210 and a first height information set determination unit 220 included in the first height information set determination module 200, and a third world coordinate parameter set determination subunit 2110 and a first world coordinate parameter set determination subunit 2120 included in the first world coordinate parameter set determination unit 210, and a calculation subunit 21210 included in the first world coordinate parameter set determination subunit 2120, and a boundary range determination unit 310, a boundary range judgment unit 320, a second area determination unit 330, and a second sampling point set determination unit 340 included in the second height information set determination module 300. 0, the second world coordinate parameter set determination unit 350, the second altitude information set determination unit 360, the elevation data determination unit 370 and the altitude information set determination unit 380, as well as the fitting surface generation subunit 3510, the second world coordinate parameter set determination subunit 3520, the fourth world coordinate parameter set determination subunit 3530 and the second world coordinate parameter set determination subunit 3540 included in the second world coordinate parameter set determination subunit 350, and the weighted calculation subunit 35410 included in the second world coordinate parameter set determination subunit 3540, as well as the difference determination unit 410 and the bumpiness determination unit 420 included in the bumpiness determination module 400, and the first judgment unit 510, the first determination unit 520, the second judgment unit 530, the second determination unit 540 and the third determination unit 550 included in the driving area determination module 500. The operations and functions can refer to the above Figures 2 to 11 The method for determining the driving area of the movable device provided is not described here in detail to avoid repetition.
[0195] Exemplary Electronic Devices
[0196] Below, reference Fig. 22 To describe an electronic device according to an embodiment of the present application. Fig. 22 A block diagram of an electronic device according to an embodiment of the present application is illustrated.
[0197] like Fig. 22 As shown, the electronic device 60 includes one or more processors 601 and a memory 602 .
[0198] The processor 601 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 60 to perform desired functions.
[0199] The memory 602 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 601 may run the program instructions to implement the method for determining the driving area of the movable device of each embodiment of the present application described above and / or other desired functions. Various contents such as the current frame image may also be stored in the computer-readable storage medium.
[0200] In one example, the electronic device 60 may further include: an input device 603 and an output device 604, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0201] The input device 603 may include, for example, a keyboard, a mouse, etc.
[0202] The output device 604 can output various information to the outside, including the determined driving area information, etc. The output device 604 can include, for example, a display, a communication network and a remote output device connected thereto, etc.
[0203] Of course, to simplify, Fig. 22 Only some of the components related to the present application in the electronic device 60 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 60 may also include any other appropriate components.
[0204] Exemplary computer program products and computer-readable storage media
[0205] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method for determining the driving area of a movable device according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.
[0206] The computer program product may be written in any combination of one or more programming languages to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0207] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method for determining the driving area of a movable device according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.
[0208] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0209] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.
[0210] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.
[0211] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.
[0212] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
[0213] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.
Claims
1. A method for determining a driving area of a movable device, include: Determine a first area based on a current frame image acquired by an image acquisition device on the movable device, wherein the first area is a non-flat area in the current frame image determined by a visual detection method; Determine a first sampling point set based on the first area, and determine a first height information set corresponding to the first sampling point set, wherein the first height information set corresponds to the first sampling point set in a one-to-one manner; Determine a second height information set corresponding to the first sampling point set, where the second height information set corresponds one-to-one to the first sampling point set; determining a degree of bumpiness of the first area based on the first altitude information set and the second altitude information set; A driving area of the movable device is determined based on the bumpiness of the first area, and the driving area includes one of a normal driving area, a decelerated driving area, and a prohibited driving area.
2. The method according to claim 1, in, The determining a first height information set corresponding to the first sampling point set includes: Determine a first world coordinate parameter set of the first sampling point set according to a ranging sensor; The first height information set is determined based on the first world coordinate parameter set.
3. The method according to claim 1, in, The determining a second height information set corresponding to the first sampling point set includes: Determining the boundary range of the first area; When the boundary range of the first area meets the preset condition, determining the second area based on the current frame image; determining a second set of sampling points based on the second area; Determine a second world coordinate parameter set corresponding to the first sampling point set based on the second sampling point set; The second height information set is determined based on the second world coordinate parameter set.
4. The method according to claim 3, in, The determining, based on the second sampling point set, a second world coordinate parameter set corresponding to the first sampling point set includes: Performing a surface fitting operation based on the second sampling points in the second sampling point set to generate a fitting surface; The second set of world coordinate parameters is determined based on the fitted surface and the first set of sampling points.
5. The method according to claim 3, further comprising: include: When the boundary range of the first area does not meet the preset condition, determining the elevation angle data of the image acquisition device on the movable device when shooting the current frame image; The second set of altitude information is determined based on the elevation angle data.
6. The method according to claim 1, in, The determining the degree of bumpiness of the first area based on the first altitude information set and the second altitude information set includes: Determining a plurality of differences between the first altitude information set and a corresponding second altitude information set; A variance parameter between the first height information set and the second height information set is determined based on the plurality of differences to determine a degree of bumpiness of the first area.
7. The method according to claim 2, in, The determining of the first world coordinate parameter set of the first sampling point set according to the ranging sensor includes: Determine, according to the ranging sensor, N sets of third world coordinate parameters corresponding to the first sampling point set at N moments; A first world coordinate parameter set corresponding to the first sampling point set is determined according to the N third world coordinate parameter sets.
8. The method according to claim 7, in, The determining, according to the N third world coordinate parameter sets, a first world coordinate parameter set corresponding to the first sampling point set comprises: A weighted calculation is performed on the N third world coordinate parameter sets to obtain the first world coordinate parameter set, wherein the weight of the third world coordinate parameter set corresponding to a moment in the N moments when the movable device is closer to the first sampling point set is greater.
9. The method according to claim 3, in, The determining, based on the second sampling point set, a second world coordinate set corresponding to the first sampling point set includes: Determine, based on the second sampling point set, M fourth world coordinate parameter sets corresponding to the first sampling point set at M moments; A second world coordinate parameter set corresponding to the first sampling point set is determined according to the M fourth world coordinate parameter sets.
10. The method according to claim 9, in, The determining, according to the M fourth world coordinate parameter sets, a second world coordinate parameter set corresponding to the first sampling point set comprises: A weighted calculation is performed on the M fourth world coordinate parameter sets to obtain the second world coordinate parameter set, wherein the weight of the fourth world coordinate parameter set corresponding to a moment in the M moments when the movable device is closer to the first sampling point set is greater.
11. The method according to claim 1, in, The determining of the driving area of the movable device based on the bumpiness of the first area includes: When the bumpiness of the first area is greater than a first bumpiness threshold and less than a second bumpiness threshold, the first area is determined to be a deceleration driving area of the movable equipment.
12. The method according to claim 1, in, The determining of the driving area of the movable device based on the bumpiness of the first area includes: When the bumpiness of the first area is greater than a second bumpiness threshold, the first area is determined to be a prohibited driving area for the movable device.
13. A device for determining a driving area of a movable device, include: A first region determination module, configured to determine a first region based on a current frame image acquired by an image acquisition device on the movable device, wherein the first region is a non-flat region in the current frame image determined by a visual detection method; a first height information set determining module, configured to determine a first sampling point set based on the first area, and determine a first height information set corresponding to the first sampling point set, wherein the first height information set corresponds to the first sampling point set in a one-to-one manner; A second height information set determining module, configured to determine a second height information set corresponding to the first sampling point set, wherein the second height information set corresponds to the first sampling point set in a one-to-one manner; a bumpiness determination module, configured to determine the bumpiness of the first area based on the first height information set and the second height information set; The driving area determination module is used to determine the driving area of the movable device based on the bumpiness of the first area, and the driving area includes one of a normal driving area, a deceleration driving area, and a prohibited driving area.
14. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the method for determining a driving area of a movable device according to any one of claims 1 to 12.
15. An electronic device, the electronic device include: processor; a memory for storing instructions executable by the processor; The processor is used to execute the method for determining the driving area of the movable device as described in any one of claims 1 to 12.
