An efficient driving assistance method, device, equipment and vehicle

Abstract

The present invention discloses an efficient driving assistance method, device, equipment and vehicle, which relates to the field of autonomous driving technology. The driving assistance method includes: a sensor data acquisition step, which acquires three-dimensional point data of the sensor detection area; a three-dimensional point sampling perception area calculation step, which acquires the current vehicle motion information, acquires the passable area of ​​the sensor detection area in the historical information, and samples the three-dimensional point data; a passable area calculation step, which fits the elevation data of the sensor detection area according to the sampled three-dimensional point data, and calculates the passable area of ​​the sensor detection area; a driving assistance step, which repeats the above steps and continuously calculates the parameters of the assisted driving according to the passable area data. This technical solution samples the sensor data based on the vehicle's motion information, reduces the amount of perception calculation, and improves the accuracy of road perception and driving safety by estimating the vehicle's passable and impassable areas.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to driving assistance methods, devices, equipment, storage media, program products and autonomous driving vehicles. Background Art

[0002] Autonomous driving technology is a technology that uses computers to achieve assisted driving or unmanned driving. It relies on sensor systems such as visible light cameras, millimeter wave radars, lidars, inertial navigation systems, and global positioning systems, so that computers can partially or completely replace human drivers to automatically and safely operate vehicles.

[0003] In existing technologies, autonomous driving is primarily applied to standard road scenarios. Standard roads, for example, are roads with specific road markings and signs, as specified by road traffic laws and regulations. In standard road scenarios, autonomous driving technology can obtain standardized environmental perception information, such as road markings and signs. It can utilize both standard and high-precision maps provided by standardized mapping surveys and surveys, and can also exchange standardized information with the transportation infrastructure within intelligent transportation systems. Relying on this prior information, existing technologies in standard road scenarios can filter out information irrelevant to road traffic, thereby reducing computational effort and improving perception efficiency.

[0004] However, autonomous driving technology can be applied not only on standard roads but also on non-standard roads. These include natural environments such as the wild, agricultural environments such as rural roads, roads within industrial parks, and specialized operational scenarios such as mines.

[0005] In non-standard road scenarios, existing technologies cannot utilize prior information such as road markings and maps to save computational effort and improve perception efficiency. Therefore, it is necessary to study the technical issues of how to save computational effort and improve perception efficiency in non-standard road scenarios. Summary of the Invention

[0006] The invention provides an efficient driving assistance method, device, equipment, storage medium, and vehicle to solve the problems existing in the above-mentioned prior art.

[0007] According to a first aspect of the present invention, there is provided a driving assistance method, comprising:

[0008] A sensor data acquisition step, acquiring three-dimensional point data of a sensor detection area;

[0009] The 3D point sampling perception area calculation step obtains the current vehicle motion information, obtains the passable area of ​​the sensor detection area in the historical information, and samples the 3D point data;

[0010] The passable area calculation step is to fit the elevation data of the sensor detection area based on the sampled three-dimensional point data to calculate the passable area of ​​the sensor detection area;

[0011] Driving assistance steps, repeat the above steps and continuously calculate the parameters of assisted driving based on the passable area data.

[0012] According to a second aspect of the present invention, there is provided a driving assistance device comprising:

[0013] A sensor data acquisition module acquires three-dimensional point data of the sensor detection area;

[0014] The 3D point sampling perception area calculation module obtains the current vehicle motion information, obtains the passable area of ​​the sensor detection area in the historical information, and samples the 3D point data;

[0015] The traversable area calculation module fits the elevation data of the sensor detection area based on the sampled three-dimensional point data and calculates the traversable area of ​​the sensor detection area;

[0016] The driving assistance module, based on the above modules, continuously calculates the parameters of assisted driving based on the passable area data.

[0017] According to a third aspect of the present invention, there is provided an electronic device, comprising:

[0018] at least one processor, memory, and an interface for communicating with other electronic devices;

[0019] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to perform the driving assistance method according to the first aspect.

[0020] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the driving assistance method according to the first aspect.

[0021] According to a fifth aspect of the present invention, there is provided a computer program product, comprising a computer program, which implements the driving assistance method according to the first aspect when executed by a processor.

[0022] According to a sixth aspect of the present invention, there is provided an autonomous driving vehicle comprising the electronic device according to the third aspect.

[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] By establishing a sampling perception area for sensor data, information about traversable areas that are more useful for the vehicle's next move is obtained. Sensor data is sampled accordingly, reducing the amount of perception computation required. Furthermore, the establishment of the sampling perception area comprehensively considers the vehicle's motion information and, based on the assumption that the vehicle will continue inertial motion for a period of time, improves the pertinence and practicality of the perception model. By distinguishing between traversable and impassable areas, the vehicle can effectively plan its route, avoiding impassable areas during driving, improving vehicle efficiency, road perception accuracy, and driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are provided for a better understanding of the present invention and do not constitute a limitation of the present invention.

[0027] Figure 1 A schematic diagram of a non-standard road scene according to an embodiment of the present invention is shown;

[0028] Figure 2 A schematic diagram of a driving assistance method according to an embodiment of the present invention is shown;

[0029] Figure 3 A schematic diagram showing sensor data provided according to an embodiment of the present invention is shown;

[0030] Figure 4 A schematic diagram showing a traversable area in an initial state provided by one embodiment of the present invention;

[0031] Figure 5 A schematic diagram showing a sensor detection area and an inertial motion area provided according to an embodiment of the present invention is shown;

[0032] Figure 6 A schematic diagram of a three-dimensional point sampling perception area provided according to an embodiment of the present invention is shown;

[0033] Figure 7 A schematic diagram of an impassable area provided according to an embodiment of the present invention is shown;

[0034] Figure 8 A schematic diagram of a traversable area according to an embodiment of the present invention is shown;

[0035] Figure 9 A schematic diagram of a driving assistance device provided according to an embodiment of the present invention is shown;

[0036] Figure 10A schematic diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0037] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0038] The technical terms involved include:

[0039] Standard road scenarios: Standard roads are legal roads that comply with road traffic laws and regulations and have specific road markings and signs. In standard road scenarios, autonomous driving technology can obtain accurate road information through standardized road markings and signs, and can also exchange standardized information with transportation infrastructure to obtain the necessary environmental information for autonomous driving.

[0040] Non-standard road scenarios: Non-standard road scenarios refer to roads that lack the road markings and other information required by road traffic laws and regulations. Non-standard road scenarios include natural scenes such as the wild, agricultural environments such as rural dirt roads, internal scenes such as roads within industrial parks, and specific operational scenarios such as mines.

[0041] Sensor data: Sensor data refers to data acquired by sensors. In embodiments of the present invention, depending on the circumstances, it may also specifically refer to data acquired by sensors related to autonomous driving. Common sensors include visible light cameras, infrared cameras, depth cameras, millimeter-wave radars, and lidars. Sensor data can be raw sensor data directly generated by the sensor, or sensor data that has undergone preprocessing, registration, conversion, fusion, feature extraction, and other processing.

[0042] Point cloud: A point cloud is a collection of point data. Point clouds can be obtained using photogrammetry or laser measurement. Laser measurement-derived point clouds include 3D coordinates (X, Y, Z) and laser echo intensity, while photogrammetry-derived point clouds include 3D coordinates (X, Y, Z) and color information (RGB).

[0043] 3D point: A 3D point is a point with 3D coordinate attributes. For example, a point in a point cloud is a 3D point.

[0044] Coordinate system: When sensors collect data, the coordinate system of the acquired 3D points is generally the sensor coordinate system. Depending on the needs of data processing, the coordinate system of the 3D points may need to be converted to another coordinate system, such as the ground coordinate system.

[0045] Elevation data: Elevation data refers to data that describes the spatial distribution of topography; data collection and measurement are performed through contour lines or other three-dimensional models, and then the data is interpolated; elevation data collection and measurement can be based on regular grids or irregular grids.

[0046] Sensor detection area: The sensor detection area is the area that the vehicle's sensors can detect while driving. The terrain in this area requires measurement to identify the portion of the area where the vehicle can normally travel. The sensor detection area is essentially a three-dimensional surface. However, in practical applications, it is often assumed that this three-dimensional surface can be projected onto a horizontal plane without overlap. In other words, the sensor detection area does not include areas such as roofs, cave ceilings, and ceilings where projections could overlap. The sensor detection area is often described using elevation data.

[0047] Inertial Movement Area: The inertial movement area is the area where the vehicle is most likely to move, calculated based on current vehicle motion information. To avoid duplicate data storage, the inertial movement area does not directly store the sensor detection area data itself. Instead, it uses a real number ranging from 0 to 1 to express the probability of the vehicle moving into that area. When the value approaches 0, the vehicle is completely unlikely to move into that area. When the value approaches 1, the vehicle is very likely to move into that area.

[0048] Passable Area: The passable area describes the traversable area within the sensor detection area. This passable area is essentially a subset of the sensor detection area. To avoid duplicate data storage, the passable area does not directly store the sensor detection area data itself. Instead, it uses a real number ranging from 0 to 1 to represent the degree of passability of the corresponding area. When the value approaches 0, the corresponding area is close to being completely impassable, and when the value approaches 1, the corresponding area is close to being completely passable.

[0049] Example 1

[0050] Figure 1 It is a schematic diagram of a non-standard road scene.

[0051] In current existing technologies, autonomous driving technology is mainly applied to standard road scenarios. In standard road scenarios, accurate road information can be obtained through standardized road markings, signs, and other information.

[0052] There is also strong demand for autonomous driving in non-standard road scenarios. These include natural environments like the wild, agricultural environments like rural dirt roads, internal environments like roads within industrial parks, and specialized operational scenarios like mines. The road surfaces in non-standard road scenarios are often uneven and prone to change. For example, in agricultural environments, the road surface may fluctuate due to vehicle traffic and rain erosion; in mining scenarios, the road surface may change due to the accumulation of minerals.

[0053] In summary, driving assistance in non-standard road scenarios presents at least the following difficulties: In non-standard road scenarios, the road markings, signs, maps, and other information found in standard road scenarios cannot be utilized. This means that prior knowledge cannot be used to focus perception on the road area, nor can computational overhead be reduced by excluding perception of non-road areas. Consequently, when the sensor detects an area, it must perceive all areas, resulting in a high level of perception computation.

[0054] This embodiment estimates vehicle passable and inaccessible areas more accurately based on vehicle motion information and historical traffic information, thereby constructing a more reasonable perception model.

[0055] Figure 2 A schematic diagram of a driving assistance method is shown.

[0056] The driving assistance method provided by an embodiment of the present invention includes the following steps:

[0057] S110 sensor data acquisition step, acquiring three-dimensional point data of the sensor detection area;

[0058] S120 is a three-dimensional point sampling perception area calculation step, which obtains current vehicle motion information, obtains a passable area of ​​the sensor detection area in historical information, and samples the three-dimensional point data;

[0059] S130: a traversable area calculation step, fitting the elevation data of the sensor detection area based on the sampled three-dimensional point data to calculate the traversable area of ​​the sensor detection area;

[0060] S140 driving assistance step, repeating the above steps, and continuously calculating the parameters of the assisted driving based on the passable area data.

[0061] In this embodiment, the sensor data acquisition step S110 includes:

[0062] Acquiring raw sensor data, wherein the raw sensor data includes three-dimensional point data;

[0063] Based on whether the scene is outdoors, perform ground 3D point filtering to obtain filtered 3D point data;

[0064] Convert the coordinates of 3D point data to the ground coordinate system;

[0065] Output the 3D point data after coordinate transformation.

[0066] The beneficial effects of step S110 include: the three-dimensional point data acquired by the sensor provides accurate data for sensing the detection area of ​​the sensor.

[0067] In this embodiment, the three-dimensional point sampling perception area calculation step S120 includes:

[0068] Obtain current vehicle motion information and calculate inertial motion area;

[0069] Obtain the passable area of ​​the sensor detection area in the historical information; initially, the passable area is the entire sensor detection area.

[0070] Establishing a three-dimensional point sampling perception area based on the inertial motion area and the passable area; the three-dimensional point sampling perception area is a model representing the density of three-dimensional point sampling in each sub-area of ​​the sensor detection area;

[0071] The 3D point data is sampled according to the 3D point sampling perception area; the area with relatively high probability of passage is sampled relatively densely; the area with relatively low probability of passage is sampled relatively sparsely;

[0072] Output the sampled 3D point data.

[0073] The beneficial effects of step S120 include: by establishing a 3D point sampling perception area, information about the traversable area that is more useful for the vehicle's next move is obtained, and 3D point data is sampled accordingly, reducing the amount of perception computation. Furthermore, establishing the 3D point sampling perception area comprehensively considers the vehicle's motion information and, based on the assumption that the vehicle will continue inertial motion for a period of time, improves the effectiveness of the perception model.

[0074] In this embodiment, the traversable area calculation step S130 includes:

[0075] Fit the elevation data of the sensor detection area based on the sampled 3D point data;

[0076] Calculate the impassable area based on at least one of the following information from the elevation data: terrain slope, terrain span, vehicle off-road performance, and vehicle loading requirements;

[0077] The impassable area is removed from the sensor detection area, and the traversable area of ​​the sensor detection area is calculated; and the traversable area is merged into the traversable area of ​​the sensor detection area in the historical information.

[0078] The beneficial effects of step S130 include: based on the sampled three-dimensional points, the amount of calculation during perception will be effectively reduced, thereby improving the speed and real-time performance of perception.

[0079] In addition, when calculating impassable and passable areas, terrain slope information can be used to exclude terrain that exceeds the vehicle's climbing ability; the terrain span and tire size in the vehicle's off-road performance can be used to ignore small-span potholes; and the full load level and spillage standards in the vehicle loading requirements can be used to limit the vehicle's passability on terrain with larger potholes.

[0080] In this embodiment, in the driving assistance step S140, the parameters of the assisted driving include at least one of the following information:

[0081] Vehicle speed, vehicle acceleration, vehicle angular velocity, vehicle start signal, vehicle brake signal, and vehicle driving route.

[0082] The beneficial effects of step S140 include: the vehicle can perform effective route planning through the passable area, thereby avoiding the impassable area while driving, thereby improving the efficiency of vehicle driving.

[0083] This embodiment does not limit the application scenario and specific implementation, which can be determined according to actual conditions and will not be described in detail here.

[0084] This embodiment can be implemented alone or together with other embodiments.

[0085] Example 2

[0086] This embodiment provides a detailed description of the sensor data related content of the driving assistance method, and the rest is the same as that of the first embodiment.

[0087] Figure 3 A schematic diagram of sensor data is shown.

[0088] The raw sensor data consists of three-dimensional point data.

[0089] One way is to directly obtain three-dimensional point data through lidar.

[0090] Another way is to use a visible light camera to obtain a depth image of the sensor detection area and convert the depth image into three-dimensional point data.

[0091] For three-dimensional point data, filtering is required to obtain information about the ground environment related to vehicle passage.

[0092] If the scene is outdoors, you don't have to worry about the collected 3D points being relative to the ceiling.

[0093] If the scene is not outdoors, such as in a mine or inside a gymnasium, the three-dimensional points on the cave top or ceiling need to be filtered out.

[0094] To filter out the top or ceiling of a cave, we can consider the number of times a 3D point appears in the vertical direction. For example, in the simplest environment without any obstructions, a 3D point will appear twice in the vertical direction: once on the ground and once on the top of the cave.

[0095] In addition, when unreasonable isolated points appear in the 3D point data, they can also be removed by filtering.

[0096] The beneficial effects of the above-mentioned sensor data filtering include: being able to obtain sensor detection area information related to vehicle trafficability, being able to adapt to both outdoor and non-outdoor scenes, and obtaining appropriate 3D point data in various scenes.

[0097] This embodiment can be implemented alone or together with other embodiments.

[0098] Example 3

[0099] This embodiment provides a detailed description of the coordinate system and traversable area of ​​the driving assistance method, and the rest is the same as that of the first embodiment.

[0100] Figure 4 A schematic diagram showing the traversable area in the initial state.

[0101] Regarding the coordinate system, the sensor detection area adopts a three-dimensional coordinate system, where the X axis is vertically upward, and the Y and Z axes are on the horizontal plane. Figure 1 、 Figure 3 、 Figure 4 The diagrams schematically show the longitudinal sections of the X and Z axes, while points A and B schematically indicate the full range of the sensor detection area on the Z axis.

[0102] The passable area does not directly store the sensor detection area data itself, but uses a real number ranging from 0 to 1 to express the degree of passability of the corresponding area. When the value approaches 0, the corresponding area is close to being completely impassable, and when the value approaches 1, the corresponding area is close to being completely passable.

[0103] The execution of step S110 to step S140 is a continuous iterative process.

[0104] When entering the loop for the first time, the default passable area is all sub-areas of the sensor detection area. Figure 4 As shown, the initial traversable area .in, Index of the sub-area of ​​the sensor detection area. y represents the Y-axis coordinate, and z represents the Z-axis coordinate.

[0105] When the loop is entered for the i-th time, step S130 calculates a new passable area. The value of will change, but will still be in the range of real numbers from 0 to 1.

[0106] Passable area It can be a function, such as a piecewise function or a continuous function. It can also be a discrete value, such as dividing the sensor detection area into several sub-areas using a grid. Indicates the grid label.

[0107] The beneficial effects of the above-mentioned passable area include: the value range of the passable area The value range is the same as that of the sensor detection area. The passable area describes each sub-area with a value ranging from 0 to 1, which improves the efficiency of data representation.

[0108] This embodiment can be implemented alone or together with other embodiments.

[0109] Example 4

[0110] This embodiment provides a detailed description of the inertial motion area related content of the driving assistance method, and the rest is the same as that of the first embodiment.

[0111] Figure 5 A schematic diagram showing the sensor detection area and the inertial motion area.

[0112] like Figure 5 Points C to D in FIG. 1 schematically illustrate the value range of the sensor detection area on the Z axis.

[0113] When the detection capability of the sensor remains unchanged, the sensor detection area will continue to move forward as the vehicle moves forward.

[0114] Since it's impossible to effectively estimate whether a vehicle can reach areas that cannot be detected, the inertial motion area can generally only be estimated within the sensor detection area. In other words, the inertial motion area is generally a subset of the sensor detection area.

[0115] Obtain current vehicle motion information and calculate an inertial motion area. The inertial motion area represents the area to which the vehicle is most likely to go, calculated based on the current vehicle motion information. The vehicle motion information includes at least one of the following:

[0116] Vehicle speed, vehicle acceleration, vehicle angular velocity, vehicle start signal, vehicle brake signal, and vehicle driving route.

[0117] Figure 5 The inertial motion area in the figure is composed of two triangles. The left triangle represents the reverse inertial motion area formed by the vehicle's potential reverse movement at the current position; the right triangle represents the forward inertial motion area formed by the vehicle's potential forward movement at the current position. Points E and F schematically indicate the range of the inertial motion area on the Z axis.

[0118] Figure 5 , the triangle on the left is smaller than the triangle on the right, indicating that the vehicle's reversing speed is less than the vehicle's forward speed, so the reversing inertial motion area is smaller than the forward inertial motion area.

[0119] Furthermore, the left and right triangles indicate that the farther away from the vehicle (the leftmost and rightmost corners), the less likely it is to be in the inertial motion region. This is because for the vehicle to reach a location far away at the next moment, it would need to be at a very high speed, which is limited by the vehicle's own performance, making this possibility relatively small.

[0120] Figure 5 Only one functional form of the inertial motion region is shown in FIG. , and the inertial motion region can also be enclosed by any curve, such as a straight line, a hyperbola, a parabola, etc. The specific form depends on the characteristics of the vehicle's inertial motion and is not limited here.

[0121] Furthermore, when the vehicle's current speed is high, the inertial motion area in the forward direction will be significantly larger than the inertial motion area in the reverse direction. When the vehicle has a high angular velocity (such as when turning), the inertial motion area along the angular velocity in the forward direction will be significantly larger than the inertial motion area along the reverse direction. The reverse is also true, which will not be elaborated here.

[0122] Figure 5 In the figure, the inertial motion area forms a large triangle structure. The X-axis coordinate corresponding to the top vertex of the large triangle is 1. This means that the top vertex of the large triangle is the area where the vehicle is most likely to be, that is, the vehicle's current location.

[0123] The benefits of the inertial motion region include: By calculating the inertial motion region, it lays the foundation for establishing a 3D point sampling perception region. The inertial motion region provides information about the traversable area that is more useful for the vehicle's next move, which is then used to sample 3D point data, reducing the amount of perception calculations.

[0124] This embodiment can be implemented alone or together with other embodiments.

[0125] Example 5

[0126] This embodiment provides a detailed description of the three-dimensional point sampling perception area of ​​the driving assistance method, and the rest is the same as that of the first embodiment.

[0127] Figure 6 A schematic diagram of the three-dimensional point sampling perception area is shown.

[0128] Get the passable area of ​​the sensor detection area in the historical information; the initial passable area is the entire sensor detection area, such as Figure 4 The traversable area As shown;

[0129] Get the inertial motion area, such as Figure 5 in As shown;

[0130] According to the inertial motion area and the passable area, a three-dimensional point sampling perception area is established, such as Figure 6 in shown.

[0131] The 3D point sampling perception area is a model that represents the 3D point sampling density of each sub-area of ​​the sensor detection area. The 3D point sampling perception area is represented by a real number ranging from 0 to 1. When the value approaches 0, it means that the probability of vehicle passage is relatively low, and the corresponding area is close to no sampling or the sampling is the sparsest. When the value approaches 1, it means that the probability of vehicle passage is relatively high, and the corresponding area is close to retaining the original data sampling density or interpolating to achieve sufficient sampling density.

[0132] like Figure 6 As shown, the three-dimensional point sampling perception area A more complex formula is used:

[0133]

[0134] in, means both must be greater than zero;

[0135] Indicates the maximum sampling density of the two;

[0136] The above formula avoids overly conservative estimation of the inertial motion region and adopts the maximum possible sampling density for all traversable areas within the inertial motion region. Points E and F schematically indicate the range of the 3D point sampling perception area on the Z axis.

[0137] In addition, the 3D point sampling perception area Other forms are also possible, such as:

[0138]

[0139] The above formula directly uses the product of the inertial motion area and the historically traversable area. This is a relatively intuitive sampling density strategy that takes both factors into account. However, the sampling density under this strategy is relatively low.

[0140] It can also be:

[0141]

[0142] The above formula directly uses the maximum value of the inertial motion area and the historically traversable area. If computing power is sufficient, a sampling strategy with the maximum possible density can be used.

[0143]

[0144] The above formula directly uses the minimum value of the inertial motion area and the historical traversable area. When computing power is insufficient, a lower density sampling strategy can be used.

[0145] After the 3D point sampling perception area is calculated, it is used to sample the 3D points. Figure 6 As shown, in Within the scope, according to The sampling density is determined by the specific value of .

[0146] Assume that the three-dimensional points obtained by the lidar are The original density at is , then the determined sampling density is .

[0147] The beneficial effects of the 3D point sampling perception area include: by establishing the 3D point sampling perception area, information about the traversable area that is more useful for the vehicle's next move is obtained, and 3D point data is sampled based on this information, reducing the amount of perception computation. Furthermore, the establishment of the 3D point sampling perception area comprehensively considers the vehicle's motion information and, based on the assumption that the vehicle will continue inertial motion for a period of time, improves the effectiveness of the perception model.

[0148] This embodiment can be implemented alone or together with other embodiments.

[0149] Example 6

[0150] This embodiment provides a detailed description of the impassable area and the passable area of ​​the driving assistance method, and the rest is the same as that of the first embodiment.

[0151] Figure 7 A schematic diagram showing an impassable area; Figure 8 A schematic diagram of the traversable area is shown.

[0152] According to the sampled 3D point data, the elevation data of the sensor detection area is fitted; Figure 1 、 Figure 3 As shown, the elevation data is fitted based on the three-dimensional points of the sensor detection area obtained by the sensor. Calculating elevation data using point cloud points belongs to the existing technology and will not be described in detail here.

[0153] The impassable area is calculated based on at least one of the following information of the elevation data: terrain slope, terrain span, vehicle off-road performance, and vehicle loading requirements. Figure 1 、 Figure 3 Two types of candidate impassable areas are schematically shown, namely slightly concave terrain and sharply raised terrain. Assuming that according to the off-road performance and loading requirements of the vehicle, Figure 1 、 Figure 3 The slightly concave terrain in the middle is measured for its slope and span, and is considered a vehicle-accessible area; Figure 1 、 Figure 3 The terrain slope and span of the medium-sized sharp protrusions are measured and it is found that the area is not passable by vehicles.

[0154] Therefore, Figure 7 As shown, point G and point H schematically indicate the value range of the impassable area on the Z axis. Figure 7 Impassable areas in The value range from point E to point F is all 1, which is a special case. Assume Figure 1 、 Figure 3 All sharp raised terrain is impassable. Generally speaking, the edges of raised or sunken terrain are passable, while the area near the center may be impassable ( Figure 7 not shown).

[0155] Remove the impassable area from the sensor detection area, calculate the passable area of ​​the sensor detection area, and merge it into the passable area of ​​the sensor detection area in the historical information. Figure 8 As shown, when entering the current cycle of steps S110 to S140, the passable area in the historical information is shown as the range from point A to point B on the Z axis. The sensor detection area is Figure 4 The range of the passable area detected in this cycle is shown as the range from point C to point D on the Z axis. Therefore, the range of the passable area detected in this cycle is shown as the range from point C to point G and from point H to point D on the Z axis.

[0156] The beneficial effects of this embodiment include: based on the sampled three-dimensional points, the amount of calculation during perception will be effectively reduced, thereby improving the speed and real-time performance of perception.

[0157] In addition, when calculating impassable and passable areas, terrain slope information can be used to exclude terrain that exceeds the vehicle's climbing ability; the terrain span and tire size in the vehicle's off-road performance can be used to ignore small-span potholes; and the full load level and spillage standards in the vehicle loading requirements can be used to limit the vehicle's passability on terrain with larger potholes.

[0158] This embodiment can be implemented alone or together with other embodiments.

[0159] Example 7

[0160] In order to solve the above-mentioned problem of non-standard road scenes, an embodiment of the present invention provides a driving assistance device. Figure 9 Shown, including:

[0161] The sensor data acquisition module 110 acquires three-dimensional point data of the sensor detection area;

[0162] The 3D point sampling perception area calculation module 120 obtains the current vehicle motion information, obtains the passable area of ​​the sensor detection area in the historical information, and samples the 3D point data;

[0163] The traversable area calculation module 130 is configured to fit the elevation data of the sensor detection area based on the sampled three-dimensional point data and calculate the traversable area of ​​the sensor detection area;

[0164] The driving assistance module 140 , based on the above modules, continuously calculates the parameters of the assisted driving according to the passable area data.

[0165] The beneficial effects of each module of the above-mentioned model training device and driving assistance device can be found in the above-mentioned embodiments and will not be repeated here.

[0166] It is worth noting that the embodiments of the present invention do not limit the specific implementation of the application scenario of the driving assistance device, which can be determined according to actual conditions and will not be elaborated here.

[0167] It should be noted that it should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called by a processing element; they can also all be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the processing module can be a separately established processing element, or it can be integrated into a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called by a processing element of the above device to perform the functions of the above processing module. The implementation of other modules is similar. In addition, these modules can all or partly be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0168] This embodiment can be implemented alone or together with other embodiments.

[0169] Example 8

[0170] like Figure 10 As shown, in this embodiment, an electronic device 600 includes:

[0171] At least one processor 601, a memory 608, and an interface 609 for communicating with other electronic devices; the memory 608 stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to execute the driving assistance method in the aforementioned embodiment.

[0172] The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are intended to be examples only and are not intended to limit the implementation of the present invention described and / or claimed herein. The electronic device may be the first device described above, or it may be a vehicle control device, or a control center on a vehicle, which is not limited by this solution.

[0173] like Figure 10As shown, the electronic device also includes: one or more ROMs 602, RAMs 603, buses, I / O interfaces, input units 606, output units 607, etc., as well as interfaces for connecting various components, including high-speed interfaces and low-speed interfaces, and interfaces for communicating with other electronic devices. The various components are connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, with each device providing some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). In this embodiment, a processor 601 is used as an example.

[0174] Memory 608 is the non-transient computer-readable storage medium provided by the present invention. The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform the method provided by the present invention. The non-transient computer-readable storage medium of the present invention stores computer instructions, which are used to enable a computer to perform the method provided by the present invention. Memory 608, as a non-transient computer-readable storage medium, can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiment of the present invention. The processor 601 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in memory 608, that is, implements the method in the above method embodiment.

[0175] The memory 608 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of electronic devices controlled by the autonomous vehicle, etc. In addition, the memory 608 may include a high-speed random access memory and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 608 may optionally include a memory remotely located relative to the processor 601, and these remote memories may be connected to the electronic devices for data processing via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0176] The various components of the electronic device may be connected via a bus or other means. In this embodiment, connection via a bus is taken as an example.

[0177] The input unit 606 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device for data processing, such as input devices such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, a pointer, one or more mouse buttons, a trackball, a joystick, etc. The output unit 607 can include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The display device can include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device can be a touch screen.

[0178] This embodiment can be implemented alone or together with other embodiments.

[0179] Embodiment 9

[0180] According to this embodiment, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the driving assistance method according to the aforementioned embodiment.

[0181] This embodiment can be implemented alone or together with other embodiments.

[0182] Example 10

[0183] According to this embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the driving assistance method according to the above embodiment is implemented.

[0184] The computer-readable storage medium and computer program product storing the computer program described in the above embodiments, these computing programs (also referred to as programs, software, software applications, or codes) include machine instructions for programmable processors, and these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or device (e.g., a disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor. This embodiment does not specifically limit it.

[0185] This embodiment can be implemented alone or together with other embodiments.

[0186] Example 11

[0187] According to this embodiment, an autonomous driving vehicle is provided, comprising the model training device according to the above embodiment or the driving assistance device according to the above embodiment.

[0188] It is understood that this embodiment is also applicable to manned vehicles, which can assist in controlling the vehicle's operation by providing driver prompts or automatic control based on acquired road information. Some vehicles are equipped with a trip computer or on-board unit (OBU), while some vehicles are equipped with user terminals such as mobile phones and users holding user terminals. The mobile phone, trip computer, or OBU in the vehicle can serve as electronic equipment for implementing model training or driving assistance.

[0189] It can be understood that this embodiment is also applicable to an intelligent transportation network, which may include multiple vehicles that can communicate wirelessly, traffic control equipment that communicate wirelessly with each vehicle, remote servers, roadside equipment, and base stations, wherein the remote server or traffic control equipment can also control traffic facilities, etc.

[0190] This embodiment does not limit the type, quantity, or application scenario of vehicles.

[0191] This embodiment can be implemented alone or together with other embodiments.

[0192] It should be understood that various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implementation in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device. The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication (e.g., a communication network) in any form or medium. Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet. A computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0193] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present invention can be achieved. The present invention is not limited here.

[0194] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A driving assistance method, comprising: A sensor data acquisition step, acquiring three-dimensional point data of a sensor detection area; The 3D point sampling perception area calculation step obtains the current vehicle motion information, obtains the passable area of ​​the sensor detection area in the historical information, and samples the 3D point data; The passable area calculation step is to fit the elevation data of the sensor detection area based on the sampled three-dimensional point data to calculate the passable area of ​​the sensor detection area; Driving assistance step, repeating the above steps to continuously calculate the parameters of assisted driving based on the passable area data; The three-dimensional point sampling perception area calculation step includes: Obtain current vehicle motion information and calculate inertial motion area; Obtain the passable area of ​​the sensor detection area in the historical information; initially, the passable area is the entire sensor detection area. Establishing a three-dimensional point sampling perception area based on the inertial motion area and the passable area; the three-dimensional point sampling perception area is a model representing the density of three-dimensional point sampling in each sub-area of ​​the sensor detection area; The 3D point data is sampled according to the 3D point sampling perception area; the area with relatively high probability of passage is sampled relatively densely; the area with relatively low probability of passage is sampled relatively sparsely; Output the sampled 3D point data.

2. The method according to claim 1, wherein the sensor data acquisition step comprises: Acquiring raw sensor data, wherein the raw sensor data includes three-dimensional point data; Based on whether the scene is outdoors, perform ground 3D point filtering to obtain filtered 3D point data; Convert the coordinates of 3D point data to the ground coordinate system; Output the 3D point data after coordinate transformation.

3. The method according to claim 1, wherein the traversable area calculation step comprises: Fit the elevation data of the sensor detection area based on the sampled 3D point data; Calculate the impassable area based on at least one of the following information from the elevation data: terrain slope, terrain span, vehicle off-road performance, and vehicle loading requirements; Remove the impassable area from the sensor detection area and calculate the passable area of ​​the sensor detection area; It is incorporated into the traversable area of ​​the sensor detection area in the historical information.

4. The method according to claim 1, wherein the parameters of the assisted driving include at least one of the following information: Vehicle speed, vehicle acceleration, vehicle angular velocity, vehicle start signal, vehicle brake signal, and vehicle driving route.

5. A driving assistance device comprising: A sensor data acquisition module acquires three-dimensional point data of the sensor detection area; The 3D point sampling perception area calculation module obtains the current vehicle motion information, obtains the passable area of ​​the sensor detection area in the historical information, and samples the 3D point data; The traversable area calculation module fits the elevation data of the sensor detection area based on the sampled three-dimensional point data and calculates the traversable area of ​​the sensor detection area; The driving assistance module, based on the above modules, continuously calculates the parameters of assisted driving based on the passable area data; The three-dimensional point sampling perception area calculation module, Obtain current vehicle motion information and calculate inertial motion area; Obtain the passable area of ​​the sensor detection area in the historical information; initially, the passable area is the entire sensor detection area. Establishing a three-dimensional point sampling perception area based on the inertial motion area and the passable area; the three-dimensional point sampling perception area is a model representing the density of three-dimensional point sampling in each sub-area of ​​the sensor detection area; Sampling 3D point data according to the 3D point sampling perception area; In the areas with relatively high traffic probability, relatively dense sampling is performed; in the areas with relatively low traffic probability, relatively sparse sampling is performed; Output the sampled 3D point data.

6. An electronic device, characterized in that: include: at least one processor, memory, and an interface for communicating with other electronic devices; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to perform the method according to any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 4.

9. An autonomous driving vehicle comprising the electronic device as claimed in claim 6.

Images