Travelable area detection method, device, apparatus, and storage medium

Abstract

Embodiments of the present application provide a drivable area detection method, device, equipment and storage medium, the method comprises: when it is detected that the driving scene of the vehicle meets the target ROI parameter switching condition, adjusting the current ROI parameter of the vehicle to the target ROI parameter corresponding to the target ROI parameter switching condition, and then performing drivable area detection according to the target ROI parameter. It can be seen that in the embodiments of the present application, different ROI parameters can be used for different road driving scenes, which is beneficial to reduce the calculation energy consumption, so as to not only save the calculation resources, but also improve the local detection accuracy.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicles, and more particularly to a method, apparatus, device, and storage medium for detecting drivable areas. Background Technology

[0002] In Advanced Driver Assistance Systems (ADAS), drivable area detection plays a crucial role. It can reduce the false detection rate of target objects and can also be used for assisted distance measurement. Especially when there are irregularly shaped obstacles in the road scene, drivable area detection can effectively detect these objects, thereby improving the perception capabilities of the autonomous driving system.

[0003] Currently, the main methods for drivable area detection include monocular vision-based methods and binocular vision-based methods. Compared to monocular vision-based methods, binocular vision-based methods are applicable to more complex driving scenarios; however, their effective detection distance is shorter than that of monocular vision-based methods. Therefore, binocular vision-based drivable area detection methods typically employ higher resolution to achieve long-range, large-area detection.

[0004] However, the Region of Interest (ROI) parameter in the binocular vision-based drivable area detection method in related technologies is fixed. This means that the above-mentioned binocular vision-based drivable area detection method requires a large amount of computing resources regardless of the road driving scenario. Therefore, the computing power consumption of the above-mentioned binocular vision-based drivable area detection method is very large in any road driving scenario. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for detecting drivable areas, which can not only save computing resources but also improve local detection accuracy.

[0006] In a first aspect, embodiments of this application provide a method for detecting drivable areas, including:

[0007] Obtain the vehicle's driving status data and drivable area data at the current vehicle time;

[0008] Based on the above driving status data and drivable area data, determine whether the driving scenario of the above vehicles meets the switching conditions of the Region of Interest (ROI) parameters.

[0009] If the vehicle's driving scenario meets the ROI parameter switching conditions, then the vehicle's current ROI parameters will be adjusted to the target ROI parameters.

[0010] Based on the target ROI parameters, obtain the drivable area data of the vehicle at the next vehicle time.

[0011] In this embodiment, when the vehicle's driving scenario meets the target ROI parameter switching condition, the vehicle's current ROI parameters are adjusted to the target ROI parameters corresponding to the target ROI parameter switching condition, and then drivable area detection is performed based on the target ROI parameters. Therefore, in this embodiment, different ROI parameters can be used for different road driving scenarios, which helps reduce computational energy consumption, thereby saving computational resources and improving local detection accuracy.

[0012] In one possible implementation, the target ROI parameters include at least one of the following: image preprocessing parameters, point cloud generation parameters, or drivable area generation parameters;

[0013] The image preprocessing parameters include at least one of the following: the size parameter of the image ROI, the position parameter of the image ROI, or the number of image scaling layers;

[0014] Point cloud generation parameters include at least one of the following: support point grid step size parameter, support point sparsity parameter, or support point distribution method parameter;

[0015] The parameters for generating the drivable area include: the placeholder grid resolution parameter.

[0016] In one possible implementation, the ROI parameter switching condition includes any of the following:

[0017] Preset ROI parameter switching conditions for highway driving scenarios, preset ROI parameter switching conditions for congested road driving scenarios, or preset ROI parameter switching conditions for narrow space uphill and downhill road driving scenarios.

[0018] Among them, the preset ROI parameter switching conditions for the congested road driving scenario include: the vehicle is traveling in the current lane at a speed less than the first preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the current lane at a first preset distance from the vehicle.

[0019] The preset ROI parameter switching conditions for driving scenarios on narrow, uphill and downhill roads include: the vehicle is traveling in the current lane at a speed less than the first preset speed along the same driving direction as the previous vehicle at that moment; the accelerator pedal state and brake pedal state of the vehicle change intermittently; and the vehicle has a movement distance greater than the second preset distance in the direction perpendicular to the horizontal ground.

[0020] In one possible implementation, the preset ROI parameter switching conditions for the highway driving scenario include any one of the following: the preset ROI parameter switching conditions for the first sub-scenario, the preset ROI parameter switching conditions for the second sub-scenario, the preset ROI parameter switching conditions for the third sub-scenario, or the preset ROI parameter switching conditions for the fourth sub-scenario.

[0021] The preset ROI parameter switching conditions for the first sub-scene include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there are no obstacles within the third preset distance of the vehicle in the adjacent lane of the current lane.

[0022] The preset ROI parameter switching conditions for the second sub-scene include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the adjacent lane of the current lane at a third preset distance from the vehicle.

[0023] The preset ROI parameter switching conditions for the third sub-scene include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the current lane at a third preset distance from the vehicle.

[0024] The preset ROI parameter switching conditions for the fourth sub-scenario include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along a direction different from the previous vehicle's travel direction.

[0025] In one possible implementation, based on the target ROI parameters, the drivable area data of the vehicle at the next vehicle time step is obtained, including:

[0026] Based on the target ROI parameters, the estimated driving state data of the vehicle at the current vehicle time, the drivable area data, and the binocular image data of the vehicle at the next vehicle time, the estimated driving state data and point cloud data of the vehicle at the next vehicle time are obtained.

[0027] Based on the estimated driving status data and point cloud data of the vehicle at the next vehicle time, obtain the drivable area data of the vehicle at the next vehicle time.

[0028] In one possible implementation, if the target ROI parameters include image preprocessing parameters and point cloud generation parameters, based on the target ROI parameters, the estimated driving state data of the vehicle at the current vehicle time, the drivable area data, and the binocular image data of the vehicle at the next vehicle time, the estimated driving state data and point cloud data of the vehicle at the next vehicle time are obtained, including:

[0029] Based on the image preprocessing parameters, the estimated driving state data of the vehicle at the current vehicle time, and the drivable area data, the binocular image data of the vehicle at the next vehicle time is preprocessed to obtain the image processing data of the vehicle at the next vehicle time.

[0030] The driving state estimation data of the vehicle at the current vehicle time and the binocular image data of the vehicle at the next vehicle time are used to perform state estimation processing to obtain the driving state estimation data of the vehicle at the next vehicle time.

[0031] Based on the point cloud generation parameters, point cloud generation processing is performed on the image processing data and driving state estimation data of the vehicle at the next vehicle time step to obtain the point cloud data of the vehicle at the next vehicle time step.

[0032] In one possible implementation, if the target ROI parameters also include: drivable area generation parameters, based on the vehicle's estimated driving state data and point cloud data at the next vehicle time step, the drivable area data for the next vehicle time step is obtained, including:

[0033] Based on the drivable area generation parameters, the estimated driving state data and point cloud data of the vehicle at the next vehicle time are processed to generate the drivable area data of the vehicle at the next vehicle time.

[0034] In one possible implementation, before obtaining the drivable area data of the vehicle at the next vehicle time based on the target ROI parameter, the method further includes: projecting the drivable area data of the vehicle at the current vehicle time onto the ROI corresponding to the target ROI parameter, so that at the next vehicle time, not only can computational energy consumption be reduced, but also drivable area detection can be performed accurately.

[0035] In one possible implementation, the driving status data includes at least one of the following: driving speed, driving direction, accelerator pedal status, and brake pedal status.

[0036] Secondly, embodiments of this application provide a drivable area detection device, comprising:

[0037] The first acquisition module is used to acquire the vehicle's driving status data and drivable area data at the current vehicle time.

[0038] The judgment module is used to determine whether the driving scenario of the vehicle meets the switching conditions of the region of interest (ROI) parameters based on the above driving status data and drivable area data.

[0039] The adjustment module is used to adjust the vehicle's current ROI parameters to the target ROI parameters if the vehicle's driving scenario meets the ROI parameter switching conditions.

[0040] The second acquisition module is used to acquire the drivable area data of the vehicle at the next vehicle time based on the target ROI parameters.

[0041] In one possible implementation, the target ROI parameters include at least one of the following: image preprocessing parameters, point cloud generation parameters, or drivable area generation parameters;

[0042] The image preprocessing parameters include at least one of the following: the size parameter of the image ROI, the position parameter of the image ROI, or the number of image scaling layers;

[0043] Point cloud generation parameters include at least one of the following: support point grid step size parameter, support point sparsity parameter, or support point distribution method parameter;

[0044] The parameters for generating the drivable area include: the placeholder grid resolution parameter.

[0045] In one possible implementation, the ROI parameter switching condition includes any of the following:

[0046] Preset ROI parameter switching conditions for highway driving scenarios, preset ROI parameter switching conditions for congested road driving scenarios, or preset ROI parameter switching conditions for narrow space uphill and downhill road driving scenarios.

[0047] Among them, the preset ROI parameter switching conditions for the congested road driving scenario include: the vehicle is traveling in the current lane at a speed less than the first preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the current lane at a first preset distance from the vehicle.

[0048] The preset ROI parameter switching conditions for driving scenarios on narrow, uphill and downhill roads include: the vehicle is traveling in the current lane at a speed less than the first preset speed along the same driving direction as the previous vehicle at that moment; the accelerator pedal state and brake pedal state of the vehicle change intermittently; and the vehicle has a movement distance greater than the second preset distance in the direction perpendicular to the horizontal ground.

[0049] In one possible implementation, the preset ROI parameter switching conditions for the highway driving scenario include any one of the following: the preset ROI parameter switching conditions for the first sub-scenario, the preset ROI parameter switching conditions for the second sub-scenario, the preset ROI parameter switching conditions for the third sub-scenario, or the preset ROI parameter switching conditions for the fourth sub-scenario.

[0050] The preset ROI parameter switching conditions for the first sub-scene include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there are no obstacles within the third preset distance of the vehicle in the adjacent lane of the current lane.

[0051] The preset ROI parameter switching conditions for the second sub-scene include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the adjacent lane of the current lane at a third preset distance from the vehicle.

[0052] The preset ROI parameter switching conditions for the third sub-scene include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the current lane at a third preset distance from the vehicle.

[0053] The preset ROI parameter switching conditions for the fourth sub-scenario include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along a direction different from the previous vehicle's travel direction.

[0054] In one possible implementation, the second acquisition module includes:

[0055] The first acquisition unit is used to acquire the estimated driving state data and point cloud data of the vehicle at the next vehicle time based on the target ROI parameters, the estimated driving state data of the vehicle at the current vehicle time, the drivable area data, and the binocular image data of the vehicle at the next vehicle time.

[0056] The second acquisition unit is used to acquire the drivable area data of the vehicle at the next vehicle time based on the estimated driving state data and point cloud data of the vehicle at the next vehicle time.

[0057] In one possible implementation, if the target ROI parameters include image preprocessing parameters and point cloud generation parameters, the first acquisition unit is specifically used for:

[0058] Based on the image preprocessing parameters, the estimated driving state data of the vehicle at the current vehicle time, and the drivable area data, the binocular image data of the vehicle at the next vehicle time is preprocessed to obtain the image processing data of the vehicle at the next vehicle time.

[0059] The driving state estimation data of the vehicle at the current vehicle time and the binocular image data of the vehicle at the next vehicle time are used to perform state estimation processing to obtain the driving state estimation data of the vehicle at the next vehicle time.

[0060] Based on the point cloud generation parameters, point cloud generation processing is performed on the image processing data and driving state estimation data of the vehicle at the next vehicle time step to obtain the point cloud data of the vehicle at the next vehicle time step.

[0061] In one possible implementation, if the target ROI parameters also include: drivable area generation parameters, the second acquisition unit is specifically used for:

[0062] Based on the drivable area generation parameters, the estimated driving state data and point cloud data of the vehicle at the next vehicle time are processed to generate the drivable area data of the vehicle at the next vehicle time.

[0063] In one possible implementation, the device further includes:

[0064] The projection module is used to project the drivable area data of the vehicle at the current vehicle time onto the ROI corresponding to the target ROI parameter.

[0065] In one possible implementation, the driving status data includes at least one of the following: driving speed, driving direction, accelerator pedal status, and brake pedal status.

[0066] Thirdly, embodiments of this application provide a drivable area detection device, including: a processor, a memory, and a communication interface;

[0067] The communication interface is used to acquire data to be processed.

[0068] The memory is used to store program instructions;

[0069] The processor is used to call and execute program instructions stored in the memory. When the processor executes the program instructions stored in the memory, the drivable area detection device is used to execute the method described in any implementation of the first aspect on the data to be processed to obtain the processed data.

[0070] The communication interface is also used to output processed data.

[0071] Fourthly, embodiments of this application provide a chip that includes the drivable area detection device described in any implementation of the third aspect above.

[0072] Fifthly, embodiments of this application provide an in-vehicle device, including the drivable area detection device described in any implementation of the third aspect above.

[0073] Sixthly, embodiments of this application provide a computer-readable storage medium for storing a computer program that implements the method described in any implementation of the first aspect above.

[0074] In a seventh aspect, embodiments of this application provide a chip system including a processor, and may further include a memory and a communication interface for implementing the method described in any implementation of the first aspect. Exemplarily, the chip system may be composed of chips, or may include chips and other discrete devices.

[0075] Eighthly, embodiments of this application provide a program that, when executed by a processor, is used to perform the method described in any implementation of the first aspect described above.

[0076] Ninthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the method described in any implementation of the first aspect described above. Attached Figure Description

[0077] Figure 1 This is a schematic diagram of the hardware system architecture in a vehicle provided in an embodiment of this application;

[0078] Figure 2 This is a schematic diagram of the architecture of the software system in the vehicle provided in the embodiments of this application;

[0079] Figure 3 This is a schematic diagram of the execution timing of the software system in the vehicle provided in an embodiment of this application;

[0080] Figure 4 A schematic diagram of the main components of the drivable area detection method provided in the embodiments of this application. Figure 1 ;

[0081] Figure 5 A schematic diagram of the main components of the drivable area detection method provided in the embodiments of this application. Figure 2 ;

[0082] Figure 6 This illustration shows the impact of adjusting point cloud generation parameters on system performance in the embodiments of this application. Figure 1 ;

[0083] Figure 7 This illustration shows the impact of adjusting point cloud generation parameters on system performance in the embodiments of this application. Figure 2 ;

[0084] Figure 8 A schematic flowchart of a drivable area detection method provided in an embodiment of this application;

[0085] Figure 9 Schematic diagram of an image containing a region of interest (ROI) provided for embodiments of this application. Figure 1 ;

[0086] Figure 10 Schematic diagram of an image containing a region of interest (ROI) provided for embodiments of this application. Figure 2 ;

[0087] Figure 11 Schematic diagram of an image containing a region of interest (ROI) provided for embodiments of this application. Figure 3 ;

[0088] Figure 12 Schematic diagram of an image containing a region of interest (ROI) provided for embodiments of this application. Figure 4 ;

[0089] Figure 13 Schematic diagram of an image containing a region of interest (ROI) provided for embodiments of this application. Figure 5 ;

[0090] Figure 14 This is a schematic diagram illustrating the execution timing of the drivable area detection method provided in the embodiments of this application;

[0091] Figure 15 This is a schematic diagram of the structure of a drivable area detection device provided in an embodiment of this application;

[0092] Figure 16 This is a schematic diagram of the drivable area detection device provided in another embodiment of this application. Detailed Implementation

[0093] First, the application scenarios and some terms involved in the embodiments of this application will be explained.

[0094] The drivable area detection method, apparatus, device, and storage medium provided in this application embodiment can be applied to drivable area detection scenarios based on binocular vision in different road driving scenarios.

[0095] For example, the drivable area detection method, apparatus, device, and storage medium provided in this application can be applied to drivable area detection scenarios in highway driving scenarios, congested road driving scenarios, and narrow space uphill and downhill road driving scenarios (such as multi-level parking garage scenarios). The above-mentioned highway driving scenarios, congested road driving scenarios, and narrow space uphill and downhill road driving scenarios are described below.

[0096] 1) Highway driving scenario:

[0097] For example, the vehicle's speed is greater than a second preset speed (e.g., 40 km / h).

[0098] 2) Driving in congested traffic:

[0099] For example, the vehicle's speed is less than a first preset speed (e.g., 20 km / h), and there is an obstacle within a first preset distance (e.g., 15 m) of the vehicle in the current lane.

[0100] 3) Driving scenarios on narrow spaces and uphill / downhill roads:

[0101] For example, the vehicle's speed is less than a first preset speed (e.g., 20 km / h), the accelerator pedal and brake pedal states of the vehicle change intermittently (e.g., the driver intermittently presses the accelerator and brake pedals), and the vehicle moves a distance greater than a second preset distance (e.g., 0.5 m) in a direction perpendicular to the horizontal ground. For example, the vehicle moves 6 m in the Z-axis (or the axis perpendicular to the ground) within 1 second.

[0102] Of course, the drivable area detection method, apparatus, device and storage medium provided in the embodiments of this application can also be applied to other scenarios, and the embodiments of this application do not limit this.

[0103] In this embodiment of the application, the entity executing the drivable area detection method provided in this embodiment can be a drivable area detection device. Exemplarily, the drivable area detection device can be a chip, chip system, circuit, or module, etc., and this application does not impose any limitations.

[0104] For example, the drivable area detection device involved in the embodiments of this application can be a chip system; of course, it can also be other computing devices with data and / or image processing functions.

[0105] Figure 1 This is a schematic diagram of the hardware system architecture in a vehicle provided in an embodiment of this application. Figure 1 As shown in the schematic diagram, the hardware system architecture in the vehicle may include, but is not limited to: a binocular camera 10, a chip system 11, an electronic control unit (ECU) 12, a controller 13, and a controller area network (CAN) bus 14. The binocular camera 10 is used to acquire image data; the CAN bus 14 is used to provide vehicle driving status data; the chip system 11 is used to detect drivable areas based on the image data acquired by the binocular camera 10 and the data provided by the CAN bus 14; the ECU 12 is used to determine control decisions based on the detection results of the chip system 11 and the data provided by the CAN bus 14; and the controller 13 is used to control the movement of the vehicle based on the control decisions of the ECU 12. It should be understood that the chip system 11 can employ the drivable area detection method provided in the embodiments of this application.

[0106] Figure 2 This is a schematic diagram of the architecture of a software system in a vehicle provided in an embodiment of this application. Figure 2As shown in the diagram, the architecture of the software system in the vehicle may include, but is not limited to, the following layers: a drive layer 20, a business software layer 21, a planning and control layer 22, and an execution layer 23. The drive layer 20 is used to read data from all onboard sensors within the vehicle, including, but not limited to, image data from a binocular camera and / or data provided by the CAN bus. The business software layer 21 is used to perform tasks such as vehicle detection, pedestrian detection, and / or drivable area detection. The planning and control layer 22 is used to perform path planning based on all detection results from the business software layer 21 (e.g., drivable area detection results) and generate control commands that are transmitted to the execution layer 23. The execution layer 23 is used to invoke onboard devices within the vehicle according to the control commands generated by the planning and control layer 22, thereby controlling the vehicle's movement.

[0107] Figure 3 This is a schematic diagram illustrating the execution timing of a software system in a vehicle, as provided in an embodiment of this application. Figure 3 As shown, 1) the driving layer acquires data from all onboard sensors in the vehicle, such as image data from the binocular camera; 2) the business software layer acquires the image data from the binocular camera from the driving layer, then performs image calibration processing on the image data from the binocular camera, then performs drivable area detection, and finally outputs the detection results to the planning and control layer; 3) the planning and control layer performs path planning and generates control commands based on the detection results, and conveys the control commands to the execution layer; 4) the execution layer calls the onboard equipment in the vehicle according to the control commands to control the movement of the vehicle.

[0108] To address the issue that the ROI parameters in existing binocular vision-based drivable area detection methods are fixed, meaning that these methods require significant computational resources regardless of the road driving scenario, resulting in high energy consumption, this application addresses this problem. In this embodiment, when a vehicle's driving scenario meets the ROI parameter switching conditions, the vehicle's current ROI parameters are adjusted to the target ROI parameters corresponding to those conditions. Drivable area detection is then performed based on the target ROI parameters. Therefore, this application embodiment allows for the use of different ROI parameters for different road driving scenarios, which helps reduce computational energy consumption, thereby saving computational resources and improving local detection accuracy.

[0109] Figure 4 A schematic diagram of the main components of the drivable area detection method provided in the embodiments of this application. Figure 1 .like Figure 4As shown, the main components of the drivable area detection method provided in this application embodiment may include, but are not limited to: an image preprocessing part, an ego motion estimation part, a point cloud generation part, a drivable area generation part, and a scene adaptive ROI decision part.

[0110] The scene-adaptive ROI decision-making component automatically detects the vehicle's current driving scene. When the vehicle's driving scene meets the ROI parameter switching conditions, it adjusts the vehicle's current ROI parameters to the target ROI parameters corresponding to the currently met ROI parameter switching conditions. This achieves the goal of reducing computational energy consumption and improving local detection accuracy. It should be noted that the aforementioned target ROI parameters may include, but are not limited to, at least one of the following: image preprocessing parameters related to ROI adjustment corresponding to the image preprocessing component, point cloud generation parameters related to ROI adjustment corresponding to the point cloud generation component, and / or drivable region generation parameters related to ROI adjustment corresponding to the drivable region generation component.

[0111] Figure 5 A schematic diagram of the main components of the drivable area detection method provided in the embodiments of this application. Figure 2 In the above Figure 4 Based on the illustrated embodiment, Figure 5 The illustrated embodiment provides a detailed description of each major component. For example... Figure 5 As shown, the main components of the drivable area detection method provided in this application embodiment may include, but are not limited to: an image preprocessing part, a self-motion estimation part, a point cloud generation part, a drivable area generation part, and a scene adaptive ROI decision part.

[0112] The image preprocessing section is used to preprocess the image data from the stereo camera, enabling preprocessing functions such as image ROI setting, image scaling, image color conversion (e.g., color to grayscale), and image enhancement.

[0113] The self-motion estimation part is used to realize the vehicle positioning function.

[0114] The point cloud generation section is used to implement functions such as feature descriptor generation, support point generation, support point triangulation, disparity map generation, and point cloud generation for stereo camera image data, as well as depth map estimation and 3D point cloud generation.

[0115] The drivable area generation section is used to implement functions such as grid setting, updating the elevation map based on the estimation results of the self-motion estimation section, adding newly generated point clouds to the elevation map, and updating the drivable area to generate Stixels (i.e. obstacle areas represented by vertical stripes).

[0116] The scene-adaptive ROI decision-making part is used to automatically detect the current driving scene of the vehicle based on the longitudinal data (e.g., accelerator pedal status, brake pedal status, driving speed, and / or driving direction) and drivable area data provided by the CAN bus at the current vehicle time. When the vehicle's driving scene meets the ROI parameter switching conditions, the part adjusts the vehicle's current ROI parameters to the target ROI parameters corresponding to the currently met ROI parameter switching conditions, thereby achieving the goal of reducing computational energy consumption and improving local detection accuracy.

[0117] The following embodiments of this application will describe the image preprocessing parameters related to ROI adjustment in the image preprocessing part, the point cloud generation parameters related to ROI adjustment in the point cloud generation part, and the drivable area generation parameters related to ROI adjustment in the drivable area generation part.

[0118] 1) Image preprocessing parameters related to ROI adjustment in the image preprocessing section

[0119] For example, the image preprocessing parameters involved in the embodiments of this application may include, but are not limited to: image ROI setting parameters, and / or, image scaling layers; wherein, the image ROI setting parameters may include, but are not limited to: image ROI size parameters, and / or, image ROI position parameters.

[0120] Table 1 illustrates the impact of adjusting image preprocessing parameters on system performance.

[0121]

[0122] 2) Point cloud generation parameters related to ROI adjustment in the point cloud generation section

[0123] For example, the point cloud generation parameters involved in the embodiments of this application may include, but are not limited to, at least one of the following: support point grid step size parameter, support point sparsity parameter, or support point distribution method parameter.

[0124] Table 2 illustrates the impact of adjusting point cloud generation parameters on system performance.

[0125]

[0126] Figure 6 This illustration shows the impact of adjusting point cloud generation parameters on system performance in the embodiments of this application. Figure 1 1) For example Figure 6 As shown in the arrow direction, the farther the ROI distance, the smaller the support point grid step size parameter. For example, the width and height step sizes change from 8*8 to 6*6 and then 4*4 to improve detection accuracy and reduce the probability of missed detections. 2) As Figure 6 As shown, the sparser the support point sparsity parameter, the lower the computational energy consumption, but the corresponding detection accuracy also decreases. Therefore, a sparser distribution can be used for close-range ROIs, and a denser distribution can be used for distant ROIs. 3) The distribution formula indicated by the support point distribution method parameter can include, but is not limited to: uniform distribution or concentrated distribution on the detected known objects. For close-range ROIs, a uniform distribution is usually used to reduce the false negative rate. For distant ROIs, the support points are usually distributed on known objects to improve detection accuracy and reduce the false negative rate.

[0127] Figure 7 This illustration shows the impact of adjusting point cloud generation parameters on system performance in the embodiments of this application. Figure 2 ,like Figure 7 As shown, this illustrates the impact of the sparse distribution of support points on the detection accuracy of drivable areas. To improve the computational efficiency of pixel depth estimation, sparsely distributed support points are first used, and then the depth of these support points is calculated. For other pixels, the depth is approximated by triangulating the support point region. (See figure.) Figure 7 As shown in (a), the denser the support points, the more accurate the depth estimation of other pixels will be. Figure 7 As shown in (b), the sparser the support points, the coarser the depth estimation for other pixels will be.

[0128] 3) Driving area generation parameters related to ROI adjustment for the driving area generation section.

[0129] For example, the drivable area generation parameters involved in the embodiments of this application may include, but are not limited to, the placeholder grid resolution parameters.

[0130] Table 3 illustrates the impact of adjusting the drivable area generation parameters on system performance.

[0131]

[0132] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0133] The technical solutions of this application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0134] Figure 8 This is a schematic flowchart illustrating a drivable area detection method provided in an embodiment of this application. Figure 8 As shown, the method in this application embodiment may include:

[0135] Step S801: Obtain the vehicle's driving status data and drivable area data at the current vehicle time.

[0136] In this step, the drivable area detection device can acquire the vehicle's driving status data at the current moment via the CAN bus. For example, the driving status data may include, but is not limited to, at least one of the following: driving speed, driving direction, accelerator pedal status, and brake pedal status. It should be understood that driving speed can correspond to the wheel speed odometer inside the vehicle, driving direction can correspond to the steering wheel angle inside the vehicle, and driving direction can also correspond to the vehicle's turn signal status (for example, if the turn signal is on, it can be determined that the vehicle's driving direction has changed; if the turn signal is off, it can be determined that the vehicle's driving direction has not changed).

[0137] It should be understood that the drivable area data of the vehicle at the current vehicle time can be detected by the drivable area generation part of the drivable area detection device. For example, the drivable area detection device can perform drivable area generation processing based on the estimated driving state data of the vehicle at the current vehicle time and the point cloud data of the vehicle at the current vehicle time, thereby obtaining the drivable area data of the vehicle at the current vehicle time. The specific method for obtaining the drivable area data of the vehicle at the current vehicle time will be discussed in the subsequent parts of this application embodiment (see...). Figure 14 The illustrated embodiment will be described.

[0138] For example, the aforementioned drivable area data may include, but is not limited to, at least one of the following: whether there are obstacles in the ROI, the location information of the obstacles, and the size information of the obstacles.

[0139] Step S802: Based on driving status data and drivable area data, determine whether the vehicle's driving scenario meets the switching conditions for the Region of Interest (ROI) parameters.

[0140] In this embodiment of the application, the drivable area detection device may be preset with at least one region of interest (ROI) parameter switching condition, so that based on the vehicle's driving status data and drivable area data at the current vehicle time, it can be determined in real time whether the vehicle's driving scenario meets a certain ROI parameter switching condition.

[0141] Optionally, the preset ROI parameter switching conditions in the drivable area detection device may include, but are not limited to, at least one of the following: preset ROI parameter switching conditions for highway driving scenarios, preset ROI parameter switching conditions for congested road driving scenarios, or preset ROI parameter switching conditions for narrow space uphill and downhill road driving scenarios.

[0142] For example, the preset ROI parameter switching conditions for a congested road driving scenario may include, but are not limited to: the vehicle is traveling in the current lane at a speed less than a first preset speed (e.g., 20 km / h) along the direction of travel of the previous vehicle (e.g., the turn signal is off), and there is an obstacle in the current lane within a first preset distance (e.g., 15 m) from the vehicle.

[0143] For example, the preset ROI parameter switching conditions for driving scenarios on narrow, uphill or downhill roads may include, but are not limited to: the vehicle is traveling in the current lane at a speed less than a first preset speed (e.g., 20 km / h) along the same driving direction as the previous vehicle at that moment (e.g., the turn signal is off), the accelerator pedal and brake pedal states of the vehicle change intermittently (e.g., the driver intermittently presses the accelerator pedal and brake pedal), and the vehicle has a movement distance greater than a second preset distance (e.g., 0.5 m) in the direction perpendicular to the horizontal ground. For example, the vehicle's movement direction moves 6 m in the Z-axis (or the axis perpendicular to the ground) within 1 second.

[0144] For example, the preset ROI parameter switching conditions for the highway driving scenario may include, but are not limited to, at least one of the following: preset ROI parameter switching conditions for the first sub-scenario, preset ROI parameter switching conditions for the second sub-scenario, preset ROI parameter switching conditions for the third sub-scenario, or preset ROI parameter switching conditions for the fourth sub-scenario.

[0145] The preset ROI parameter switching conditions for the first sub-scene may include, but are not limited to: the vehicle is traveling in the current lane at a speed greater than the second preset speed (e.g., 40 km / h) along the direction of travel of the previous vehicle (e.g., the turn signal is off), and there are no obstacles within the third preset distance (e.g., 50 m) of the vehicle in the adjacent lane of the current lane.

[0146] The preset ROI parameter switching conditions for the second sub-scene may include, but are not limited to: the vehicle is traveling in the current lane at a speed greater than the second preset speed (e.g., 40 km / h) along the direction of travel of the previous vehicle (e.g., the turn signal is off), and there is an obstacle in the adjacent lane of the current lane at a third preset distance from the vehicle (e.g., an obstacle is detected in the adjacent lane of the current lane in the ROI, and as the vehicle moves forward, the obstacle gets closer and closer and has reached the boundary of the ROI).

[0147] The preset ROI parameter switching conditions for the third sub-scene may include, but are not limited to: the vehicle is traveling in the current lane at a speed greater than the second preset speed (e.g., 40 km / h) along the direction of travel of the previous vehicle (e.g., the turn signal is off), and there is an obstacle in the current lane at a third preset distance from the vehicle (e.g., an obstacle is detected in the current lane of the ROI, and as the vehicle moves forward, the obstacle gets closer and closer and has reached the boundary of the ROI).

[0148] The preset ROI parameter switching conditions for the fourth sub-scenario may include, but are not limited to: the vehicle is traveling in the current lane at a speed greater than the second preset speed (e.g., 40 km / h) along a direction different from the previous vehicle's direction of travel (e.g., the turn signal is on).

[0149] In this step, the drivable area detection device can determine whether the current driving scenario of the vehicle meets a certain ROI parameter switching condition (for ease of description, it can be called the target ROI parameter switching condition) based on the vehicle's driving status data and drivable area data obtained in step S801 at the current vehicle time.

[0150] It should be understood that the target ROI parameter switching conditions may include, but are not limited to, any of the following: preset ROI parameter switching conditions for highway driving scenarios, preset ROI parameter switching conditions for congested road driving scenarios, or preset ROI parameter switching conditions for narrow space uphill and downhill road driving scenarios.

[0151] Step S803: If the vehicle's driving scenario meets the ROI parameter switching conditions, then adjust the vehicle's current ROI parameters to the target ROI parameters.

[0152] In this embodiment of the application, the drivable area detection device may be pre-set with ROI parameters corresponding to each of the above-mentioned at least one Region of Interest (ROI) parameter switching conditions, so that when the current driving scenario of the vehicle is detected to meet a certain ROI parameter switching condition, the current ROI parameter of the vehicle can be adjusted to the ROI parameter corresponding to the currently met ROI parameter switching condition (for ease of description, it can be referred to as the target ROI parameter).

[0153] In this step, if the vehicle's current driving scenario meets at least one of the above-mentioned Region of Interest (ROI) parameter switching conditions (which can be referred to as the target ROI parameter switching condition for ease of description), the drivable area detection device can adjust the vehicle's current ROI parameters to the target ROI parameters corresponding to the above-mentioned target ROI parameter switching condition.

[0154] For example, if the drivable area detection device can be pre-set with ROI parameter 1 corresponding to ROI parameter switching condition 1, ROI parameter 2 corresponding to ROI parameter switching condition 2, and ROI parameter 3 corresponding to ROI parameter switching condition 3, and the current driving scenario of the vehicle is detected to meet the above-mentioned ROI parameter switching condition 2, then the drivable area detection device can adjust the current ROI parameter of the vehicle to the above-mentioned ROI parameter 2.

[0155] For example, the target ROI parameters mentioned above may include, but are not limited to, at least one of the following: image preprocessing parameters, point cloud generation parameters, or drivable area generation parameters.

[0156] The image preprocessing parameters may include, but are not limited to, at least one of the following: the size parameter of the image ROI, the position parameter of the image ROI, or the number of image scaling layers.

[0157] Point cloud generation parameters may include, but are not limited to, at least one of the following: support point grid step size parameter, support point sparsity parameter, or support point distribution method parameter.

[0158] The parameters for generating the drivable area may include, but are not limited to, the placeholder grid resolution parameter.

[0159] It should be understood that in the process of adjusting the current ROI parameters of the vehicle to the target ROI parameters corresponding to the above-mentioned target ROI parameter switching conditions in the embodiments of this application, the corresponding parameters in the current ROI parameters of the vehicle that are different from those in the target ROI parameters will be modified to be the same as those in the target ROI parameters, but the corresponding parameters in the current ROI parameters of the vehicle that are the same as those in the target ROI parameters will be retained. In addition, other parameters in the current ROI parameters of the vehicle that are not included in the target ROI parameters may also be retained.

[0160] The following embodiments of this application describe the ROI parameter adjustment methods when the current driving scenario of the vehicle meets different ROI parameter switching conditions.

[0161] 1) The vehicle's driving scenario meets the preset ROI parameter switching conditions for the above-mentioned highway driving scenario:

[0162] In one possible implementation, if the current driving scenario of the vehicle is detected to meet the preset ROI parameter switching conditions of the first sub-scenario, that is, the target ROI parameter switching conditions are the preset ROI parameter switching conditions of the first sub-scenario, the drivable area detection device can adjust the current ROI parameters of the vehicle to the target ROI parameters corresponding to the preset ROI parameter switching conditions of the first sub-scenario.

[0163] For ease of understanding, the following embodiments of this application provide a schematic table of preset ROI parameter switching conditions for the first sub-scene, and the implementation method is introduced in conjunction with the schematic table.

[0164] Table 4 is a schematic diagram of the preset ROI parameter switching conditions for the first sub-scene.

[0165]

[0166]

[0167] It should be noted that the values ​​of each parameter in Table 4 are only illustrative, and the specific values ​​can be determined according to the actual situation. This application does not limit this.

[0168] For example, as shown in Table 4, the target ROI parameters corresponding to the preset ROI parameter switching conditions of the first sub-scene mentioned above may include at least one of the following main adjustment parameters: the size parameter of the image ROI (e.g., the ROI can cover a range of 50m-80m), the location parameter of the image ROI (e.g., the ROI can be located at a position more than 50m away from the vehicle), the support point grid step size parameter (e.g., the grid step size parameter is 4*4), and the placeholder grid resolution parameter (e.g., the placeholder grid resolution parameter is that the grid size is set to 0.8m).

[0169] As another example, as shown in Table 4, the target ROI parameter corresponding to the preset ROI parameter switching condition of the first sub-scene may also include at least one of the following auxiliary adjustment parameters: number of image scaling layers (e.g., the number of image scaling layers is 2 layers), support point sparsity parameter (e.g., the support point density indicated by the support point sparsity parameter is greater than the preset density), and support point distribution mode parameter (e.g., the support point distribution mode parameter is used to indicate the distribution mode mainly based on obstacle distribution).

[0170] Of course, the target ROI parameters corresponding to the preset ROI parameter switching conditions of the first sub-scenario mentioned above may also include other parameters, which are not limited in this embodiment.

[0171] It should be understood that if the target ROI parameters include any image preprocessing parameters corresponding to the image preprocessing part, and these image preprocessing parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the image preprocessing part; if the target ROI parameters include any point cloud generation parameters corresponding to the point cloud generation part, and these point cloud generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the point cloud generation part; if the target ROI parameters include any drivable area generation parameters corresponding to the drivable area generation part, and these drivable area generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the drivable area generation part.

[0172] In another possible implementation, if the vehicle's driving scenario is detected to meet the preset ROI parameter switching conditions of the second sub-scenario, that is, the target ROI parameter switching conditions are the preset ROI parameter switching conditions of the second sub-scenario, then the drivable area detection device can adjust the vehicle's current ROI parameters to the target ROI parameters corresponding to the preset ROI parameter switching conditions of the second sub-scenario.

[0173] For ease of understanding, the following embodiments of this application provide a schematic table of preset ROI parameter switching conditions for the second sub-scene, and the implementation method is introduced in conjunction with the schematic table.

[0174] Table 5 is a schematic diagram of the preset ROI parameter switching conditions for the second sub-scene.

[0175]

[0176] It should be noted that the values ​​of each parameter in Table 5 are only illustrative, and the specific values ​​can be determined according to the actual situation. This application does not limit this.

[0177] For example, the relevant content of the target ROI parameters corresponding to the preset ROI parameter switching conditions of the second sub-scene can be referred to the relevant content of the target ROI parameters corresponding to the preset ROI parameter switching conditions of the first sub-scene, and will not be repeated here.

[0178] It should be understood that if the target ROI parameters include any image preprocessing parameters corresponding to the image preprocessing part, and these image preprocessing parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the image preprocessing part; if the target ROI parameters include any point cloud generation parameters corresponding to the point cloud generation part, and these point cloud generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the point cloud generation part; if the target ROI parameters include any drivable area generation parameters corresponding to the drivable area generation part, and these drivable area generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the drivable area generation part.

[0179] It should be understood that if the vehicle's current ROI parameters include the target ROI parameters, or if the parameters in the vehicle's current ROI parameters that correspond to the target ROI parameters are all the same as the target ROI parameters, i.e., the parameters have not been adjusted, then the drivable area detection device does not need to be reinitialized.

[0180] Figure 9 Schematic diagram of an image containing a region of interest (ROI) provided for embodiments of this application. Figure 1 ,like Figure 9 As shown, an obstacle is detected in the adjacent lane of the current driving lane in the ROI. As the vehicle moves forward, the obstacle gets closer and closer and reaches the boundary of the ROI. When the obstacle continues to move relative to the vehicle and goes beyond the boundary of the ROI, since the obstacle is no longer in the current driving lane, it will not affect the safe driving of the vehicle. The drivable area detection device can continue to track the obstacle by using the motion trajectory prediction method based on the vehicle's driving state estimation data, until the obstacle disappears from the image data collected by the vehicle's binocular camera.

[0181] In another possible implementation, if the vehicle's driving scenario is detected to meet the preset ROI parameter switching conditions of the aforementioned third sub-scenario, that is, the target ROI parameter switching conditions are the preset ROI parameter switching conditions of the third sub-scenario, then the drivable area detection device can adjust the vehicle's current ROI parameters to the target ROI parameters corresponding to the preset ROI parameter switching conditions of the aforementioned third sub-scenario.

[0182] For ease of understanding, the following embodiments of this application provide a schematic table of preset ROI parameter switching conditions for the third sub-scene, and the implementation method is introduced in conjunction with the schematic table.

[0183] Table 6 is a schematic diagram of the preset ROI parameter switching conditions for the third sub-scene.

[0184]

[0185] It should be noted that the values ​​of each parameter in Table 6 are only illustrative, and the specific values ​​can be determined according to the actual situation. This application does not limit this.

[0186] For example, as shown in Table 6, the target ROI parameters corresponding to the preset ROI parameter switching conditions of the third sub-scene mentioned above may include at least one of the following main adjustment parameters: the size parameter of the image ROI (e.g., the ROI can cover the range of 0m-50m), the position parameter of the image ROI (e.g., the ROI can be located within 50m of the vehicle), the support point grid step size parameter (e.g., the grid step size parameter is 8*8), and the placeholder grid resolution parameter (e.g., the placeholder grid resolution parameter is that the grid size is set to 0.2m).

[0187] As another example, as shown in Table 6, the target ROI parameter corresponding to the preset ROI parameter switching condition of the third sub-scene may also include at least one of the following auxiliary adjustment parameters: number of image scaling layers (e.g., the number of image scaling layers is 4 layers), support point sparsity parameter (e.g., the support point density indicated by the support point sparsity parameter is not greater than the preset density), and support point distribution mode parameter (e.g., the support point distribution mode parameter is used to indicate the distribution mode dominated by obstacle distribution).

[0188] Of course, the target ROI parameters corresponding to the preset ROI parameter switching conditions of the third sub-scenario mentioned above may also include other parameters, which are not limited in this embodiment.

[0189] It should be understood that if the target ROI parameters include any image preprocessing parameters corresponding to the image preprocessing part, and these image preprocessing parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the image preprocessing part; if the target ROI parameters include any point cloud generation parameters corresponding to the point cloud generation part, and these point cloud generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the point cloud generation part; if the target ROI parameters include any drivable area generation parameters corresponding to the drivable area generation part, and these drivable area generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the drivable area generation part.

[0190] Figure 10 Schematic diagram of an image containing a region of interest (ROI) provided for embodiments of this application. Figure 2 ,like Figure 10 As shown, an obstacle is detected in the current lane of the ROI. As the vehicle moves forward, the obstacle gets closer and closer and reaches the boundary of the ROI. When the obstacle continues to move relative to the vehicle and goes beyond the boundary of the ROI, it will affect the safe driving of the vehicle because the obstacle is in the current lane. The drivable area detection device can move the ROI closer to the vehicle and reproject the drivable area data of the vehicle at the current time to the new ROI corresponding to the target ROI parameters, and then continuously detect and track obstacles.

[0191] In another possible implementation, if the vehicle's driving scenario is detected to meet the preset ROI parameter switching conditions of the fourth sub-scenario, that is, the target ROI parameter switching conditions are the preset ROI parameter switching conditions of the fourth sub-scenario, then the drivable area detection device can adjust the vehicle's current ROI parameters to the target ROI parameters corresponding to the preset ROI parameter switching conditions of the fourth sub-scenario.

[0192] For ease of understanding, the following embodiments of this application provide a schematic table of preset ROI parameter switching conditions for the fourth sub-scene, and the implementation method is introduced in conjunction with the schematic table.

[0193] Table 7 is a schematic diagram of the preset ROI parameter switching conditions for the fourth sub-scene.

[0194]

[0195]

[0196] It should be noted that the values ​​of each parameter in Table 7 are merely illustrative, and the specific values ​​can be determined according to the actual situation. This application embodiment does not impose any limitations on this.

[0197] For example, as shown in Table 7, the target ROI parameters corresponding to the preset ROI parameter switching conditions of the fourth sub-scene mentioned above may include at least one of the following main adjustment parameters: the size parameter of the image ROI (e.g., the ROI can cover the range of 0m-50m), the position parameter of the image ROI (e.g., the ROI can be located within 50m of the vehicle), the support point grid step size parameter (e.g., the grid step size parameter is 8*8), and the placeholder grid resolution parameter (e.g., the placeholder grid resolution parameter is that the grid size is set to 0.2m).

[0198] As another example, as shown in Table 7, the target ROI parameter corresponding to the preset ROI parameter switching condition of the fourth sub-scene may also include at least one of the following auxiliary adjustment parameters: number of image scaling layers (e.g., the number of image scaling layers is 4 layers), support point sparsity parameter (e.g., the support point density indicated by the support point sparsity parameter is not greater than the preset density), and support point distribution mode parameter (e.g., the support point distribution mode parameter is used to indicate the distribution mode mainly based on obstacle distribution).

[0199] Of course, the target ROI parameters corresponding to the preset ROI parameter switching conditions of the fourth sub-scenario mentioned above may also include other parameters, which are not limited in this embodiment.

[0200] It should be understood that if the target ROI parameters include any image preprocessing parameters corresponding to the image preprocessing part, and these image preprocessing parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the image preprocessing part; if the target ROI parameters include any point cloud generation parameters corresponding to the point cloud generation part, and these point cloud generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the point cloud generation part; if the target ROI parameters include any drivable area generation parameters corresponding to the drivable area generation part, and these drivable area generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the drivable area generation part.

[0201] Figure 11 Schematic diagram of an image containing a region of interest (ROI) provided for embodiments of this application. Figure 3 ,like Figure 11As shown, since the turn signal is detected to be on, the drivable area detection device can determine that the vehicle will travel in a different direction than the previous vehicle (i.e., the vehicle is turning). In other words, the scene in front of the vehicle will quickly switch to an unknown scene. Therefore, the ROI is moved closer to the vehicle so that objects in the near distance can be detected in the unknown scene, thereby improving the safety of vehicle driving.

[0202] In summary, when the drivable area detection device detects that the vehicle's driving scenario matches the aforementioned highway driving scenario, it can adaptively adjust the detection area of ​​the ROI from far to near, or from near to far, according to the different ROI parameter switching conditions that the vehicle's driving scenario meets. This can effectively reduce computational energy consumption and improve the detection distance and accuracy of drivable areas at a distance.

[0203] 2) The vehicle's driving scenario meets the preset ROI parameter switching conditions for the above-mentioned congested road driving scenario:

[0204] For example, if the current driving scenario of the vehicle is detected to meet the preset ROI parameter switching conditions of the above-mentioned congested road driving scenario, that is, the target ROI parameter switching conditions are the preset ROI parameter switching conditions of the congested road driving scenario, the drivable area detection device can adjust the current ROI parameters of the vehicle to the target ROI parameters corresponding to the preset ROI parameter switching conditions of the above-mentioned congested road driving scenario.

[0205] For ease of understanding, the following embodiments of this application provide a schematic table of preset ROI parameter switching conditions for congested road driving scenarios, and the implementation method is introduced in conjunction with the schematic table.

[0206] Table 8 is a schematic diagram of the preset ROI parameter switching conditions for driving scenarios in congested roads.

[0207]

[0208] It should be noted that the values ​​of each parameter in Table 8 are only illustrative, and the specific values ​​can be determined according to the actual situation. This application does not limit this.

[0209] For example, as shown in Table 8, the target ROI parameters corresponding to the preset ROI parameter switching conditions for the above-mentioned congested road driving scenario may include at least one of the following main adjustment parameters: the size parameter of the image ROI (e.g., the ROI can cover the range of 0m-50m), the position parameter of the image ROI (e.g., the ROI can be located within 50m of the vehicle), the support point grid step size parameter (e.g., the grid step size parameter is 16*16), and the placeholder grid resolution parameter (e.g., the placeholder grid resolution parameter is that the grid size is set to 0.1m).

[0210] As another example, as shown in Table 8, the target ROI parameters corresponding to the preset ROI parameter switching conditions for the above-mentioned congested road driving scenario may also include at least one of the following auxiliary adjustment parameters: number of image scaling layers (e.g., the number of image scaling layers is 6 layers), support point sparsity parameter (e.g., the support point density indicated by the support point sparsity parameter is not greater than the preset density), and support point distribution mode parameter (e.g., the support point distribution mode parameter is used to indicate a distribution mode that is mainly uniform distribution).

[0211] Of course, the target ROI parameters corresponding to the preset ROI parameter switching conditions in the above-mentioned congested road driving scenario may also include other parameters, which are not limited in this embodiment.

[0212] It should be understood that if the target ROI parameters include any image preprocessing parameters corresponding to the image preprocessing part, and these image preprocessing parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the image preprocessing part; if the target ROI parameters include any point cloud generation parameters corresponding to the point cloud generation part, and these point cloud generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the point cloud generation part; if the target ROI parameters include any drivable area generation parameters corresponding to the drivable area generation part, and these drivable area generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the drivable area generation part.

[0213] Figure 12 Schematic diagram of an image containing a region of interest (ROI) provided for embodiments of this application. Figure 4 ,like Figure 12 As shown, for congested road driving scenarios, this mainly refers to the detection of nearby obstacles. Since the same nearby obstacle occupies most of the image area, a sparser distribution of support points or a uniform distribution method can be used, thereby reducing computational energy consumption and improving the accuracy of nearby obstacle detection.

[0214] 3) The vehicle's driving scenario meets the preset ROI parameter switching conditions for the above-mentioned narrow space uphill and downhill road driving scenario:

[0215] For example, if the current driving scenario of the vehicle is detected to meet the preset ROI parameter switching conditions of the above-mentioned narrow space uphill and downhill road driving scenario, that is, the target ROI parameter switching conditions are the preset ROI parameter switching conditions of the narrow space uphill and downhill road driving scenario, then the drivable area detection device can adjust the current ROI parameters of the vehicle to the target ROI parameters corresponding to the preset ROI parameter switching conditions of the above-mentioned narrow space uphill and downhill road driving scenario.

[0216] For ease of understanding, the following embodiments of this application provide a schematic table of preset ROI parameter switching conditions for driving scenarios on narrow spaces and uphill / downhill roads, and the implementation method is introduced in conjunction with the schematic table.

[0217] Table 9 is a schematic diagram of the preset ROI parameter switching conditions for driving scenarios on narrow, uphill and downhill roads.

[0218]

[0219] It should be noted that the values ​​of each parameter in Table 9 are only illustrative, and the specific values ​​can be determined according to the actual situation. This application does not limit this.

[0220] For example, as shown in Table 9, the target ROI parameters corresponding to the preset ROI parameter switching conditions for the above-mentioned narrow space uphill and downhill road driving scenario may include at least one of the following main adjustment parameters: image ROI size parameter (e.g., the ROI can cover a range of 0m-50m), image ROI location parameter (e.g., the ROI can be located within 50m of the vehicle), support point grid step size parameter (e.g., the grid step size parameter is 8*8), occupancy grid resolution parameter (e.g., the occupancy grid resolution parameter is that the grid size is set to 0.1m), and support point sparsity parameter (e.g., the support point sparsity parameter is used to indicate that the support point distribution density in the two sides of the image is adjusted to twice that in the middle of the image).

[0221] As another example, as shown in Table 9, the target ROI parameters corresponding to the preset ROI parameter switching conditions for the above-mentioned narrow space uphill and downhill road driving scenario may also include at least one of the following auxiliary adjustment parameters: number of image scaling layers (e.g., the number of image scaling layers is 6 layers), support point distribution mode parameter (e.g., the support point distribution mode parameter is used to indicate a distribution mode that is mainly uniform distribution).

[0222] Of course, the target ROI parameters corresponding to the preset ROI parameter switching conditions for the above-mentioned narrow space uphill and downhill road driving scenarios may also include other parameters, which are not limited in this embodiment.

[0223] It should be understood that if the target ROI parameters include any image preprocessing parameters corresponding to the image preprocessing part, and these image preprocessing parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the image preprocessing part; if the target ROI parameters include any point cloud generation parameters corresponding to the point cloud generation part, and these point cloud generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the point cloud generation part; if the target ROI parameters include any drivable area generation parameters corresponding to the drivable area generation part, and these drivable area generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device also needs to reinitialize the drivable area generation part.

[0224] Figure 13 Schematic diagram of an image containing a region of interest (ROI) provided for embodiments of this application. Figure 5 ,like Figure 13 As shown, for driving scenarios on narrow, uphill or downhill roads (e.g., uphill or downhill scenarios in multi-story parking garages), by adjusting the relevant ROI parameters, the detection accuracy of obstacles on the left and right sides of the vehicle is improved. This not only reduces computational energy consumption but also improves the protection of the left and right sides of the vehicle when driving in narrow spaces.

[0225] Step S804: Based on the target ROI parameters, obtain the drivable area data of the vehicle at the next vehicle time.

[0226] In this step, the drivable area detection device can further obtain the drivable area data of the vehicle at the next vehicle time step based on the target ROI parameters corresponding to the ROI parameter switching conditions met by the vehicle's current driving scenario in step S803 above. It is evident that by using different ROI parameters for different road driving scenarios, it is beneficial to reduce the computational energy consumption in the drivable area detection process, thereby not only saving computational resources in the drivable area detection process but also improving local detection accuracy.

[0227] It should be understood that the drivable area detection device obtains the drivable area data of the vehicle at the next vehicle time based on the aforementioned target ROI parameters. This also allows the drivable area detection device to determine whether the vehicle's driving scenario at the next vehicle time meets a certain ROI parameter switching condition based on the vehicle's driving status data and drivable area data at the next vehicle time. When it detects that the vehicle's driving scenario at the next vehicle time meets a certain ROI parameter switching condition, it can adjust the vehicle's ROI parameter at the next vehicle time to the ROI parameter corresponding to the ROI parameter switching condition that the next vehicle time meets.

[0228] The following sections of this application embodiment describe the implementation method of "obtaining the drivable area data of the vehicle at the next vehicle time according to the target ROI parameters" in step S804 above.

[0229] Optionally, the drivable area detection device can acquire the estimated driving state data and point cloud data of the vehicle at the next vehicle time based on the target ROI parameters, the estimated driving state data of the vehicle at the current vehicle time, the drivable area data, and the binocular image data of the vehicle at the next vehicle time.

[0230] It should be noted that the estimated driving state data of the vehicle at the current vehicle moment can be obtained from the ego motion estimation part of the drivable area detection device. For example, the drivable area detection device can perform state estimation processing based on the estimated driving state data of the vehicle at the previous vehicle moment and the binocular image data of the vehicle at the current vehicle moment to obtain the estimated driving state data of the vehicle at the current vehicle moment. The specific state estimation processing method can refer to the state estimation processing methods in related technologies, and is not limited in this embodiment.

[0231] It should be noted that the drivable area data of the vehicle at the current vehicle time can be detected by the drivable area generation part of the drivable area detection device. For example, the drivable area detection device can perform drivable area generation processing based on the estimated driving state data of the vehicle at the current vehicle time and the point cloud data of the vehicle at the current vehicle time to obtain the drivable area data of the vehicle at the current vehicle time. The specific drivable area generation processing can refer to the drivable area generation processing methods in related technologies, and is not limited to this embodiment.

[0232] For example, if the target ROI parameters include image preprocessing parameters and point cloud generation parameters, the drivable area detection device can first perform image preprocessing on the binocular image data of the vehicle at the next vehicle time based on the image preprocessing parameters, the estimated driving state data of the vehicle at the current vehicle time, and the drivable area data of the vehicle at the current vehicle time, thereby obtaining the image processed data of the vehicle at the next vehicle time. The specific image preprocessing can refer to image preprocessing methods in related technologies, and is not limited in this embodiment.

[0233] Secondly, the drivable area detection device can perform state estimation processing based on the estimated driving state data of the vehicle at the current vehicle time and the binocular image data of the vehicle at the next vehicle time, and thus obtain the estimated driving state data of the vehicle at the next vehicle time.

[0234] Then, the drivable area detection device can perform point cloud generation processing on the image processing data of the vehicle at the next vehicle time step and the driving state estimation data of the vehicle at the next vehicle time step according to the aforementioned point cloud generation parameters, thereby obtaining the point cloud data of the vehicle at the next vehicle time step. The specific point cloud generation processing can refer to the point cloud generation processing methods in related technologies, and is not limited to this embodiment.

[0235] It should be understood that if the target ROI parameters do not include image preprocessing parameters, the drivable area detection device can perform image preprocessing based on the image preprocessing parameters included in the current ROI parameters of the vehicle. If the target ROI parameters do not include point cloud generation parameters, the drivable area detection device can perform point cloud generation processing based on the point cloud generation parameters included in the current ROI parameters of the vehicle.

[0236] Of course, the drivable area detection device can also obtain the estimated driving state data and point cloud data of the vehicle at the next vehicle time through other means based on the target ROI parameters, the estimated driving state data of the vehicle at the current vehicle time, the drivable area data, and the binocular image data of the vehicle at the next vehicle time. This application embodiment does not limit this.

[0237] Furthermore, the drivable area detection device can obtain the drivable area data of the vehicle at the next vehicle time based on the estimated driving state data and point cloud data of the vehicle at the next vehicle time.

[0238] For example, if the target ROI parameters further include drivable area generation parameters, the drivable area detection device can perform drivable area generation processing on the estimated driving state data of the vehicle at the next vehicle time and the point cloud data of the vehicle at the next vehicle time based on the drivable area generation parameters, and thus obtain the drivable area data of the vehicle at the next vehicle time.

[0239] It should be understood that if the target ROI parameters do not include drivable area generation parameters, the drivable area detection device can perform drivable area generation processing based on the drivable area generation parameters included in the current ROI parameters of the vehicle.

[0240] Of course, the drivable area detection device can also obtain the drivable area data of the vehicle at the next vehicle time through other means based on the estimated driving status data and point cloud data of the vehicle at the next vehicle time. This application embodiment does not limit this.

[0241] In summary, in this embodiment, when the vehicle's driving scenario meets the target ROI parameter switching conditions, the vehicle's current ROI parameters are adjusted to the target ROI parameters corresponding to the target ROI parameter switching conditions, and then drivable area detection is performed based on the target ROI parameters. Therefore, in this embodiment, different ROI parameters can be used for different road driving scenarios, which helps reduce computational energy consumption, thereby saving computational resources and improving local detection accuracy.

[0242] It should be noted that the binocular vision-based drivable area detection method provided in this application has low computational power consumption, so it can be applied to general-purpose chip systems without the need for dedicated neural network hardware, thereby saving system costs.

[0243] Furthermore, before acquiring the drivable area data of the vehicle at the next vehicle time based on the target ROI parameters, the drivable area detection device can also reproject the drivable area data of the vehicle at the current vehicle time onto the new ROI corresponding to the target ROI parameters, so that in the next vehicle time, not only can computing energy consumption be reduced, but drivable area detection can also be performed accurately.

[0244] For ease of understanding, the following embodiments of this application incorporate... Figure 4 and Figure 5 As shown, the execution timing of the drivable area detection method provided in the embodiments of this application at the current vehicle time is described.

[0245] Figure 14 This is a timing diagram illustrating the execution of a drivable area detection method provided in an embodiment of this application. Based on the above embodiments, as... Figure 14 As shown, the method in this application embodiment may include two main parts: drivable area detection and ROI adaptive adjustment.

[0246] I. Drivable Area Detection

[0247] 1) The drivable area detection device can obtain the binocular image data of the vehicle at the current vehicle time, the drivable area data of the vehicle at the previous vehicle time, and the estimated driving state data of the vehicle at the previous vehicle time from the drivable area detection memory. Next, the drivable area detection device can perform image preprocessing based on the binocular image data of the vehicle at the current vehicle time, the drivable area data of the vehicle at the previous vehicle time, and the estimated driving state data of the vehicle at the previous vehicle time, thus obtaining the image processing data of the vehicle at the current vehicle time. Then, the drivable area detection device can store the image processing data of the vehicle at the current vehicle time into the drivable area detection memory.

[0248] 2) The drivable area detection device can obtain the binocular image data of the vehicle at the current vehicle moment and the estimated driving state data of the vehicle at the previous vehicle moment from the drivable area detection memory. Next, the drivable area detection device can perform state estimation processing based on the binocular image data of the vehicle at the current vehicle moment and the estimated driving state data of the vehicle at the previous vehicle moment to obtain the estimated driving state data of the vehicle at the current vehicle moment. Then, the drivable area detection device can store the estimated driving state data of the vehicle at the current vehicle moment into the drivable area detection memory.

[0249] 3) The drivable area detection device can obtain the image processing data of the vehicle at the current vehicle moment and the estimated driving state data of the vehicle at the current vehicle moment from the drivable area detection memory. Next, the drivable area detection device can perform point cloud generation processing based on the image processing data of the vehicle at the current vehicle moment and the estimated driving state data of the vehicle at the current vehicle moment, thus obtaining the aforementioned point cloud data of the vehicle at the current vehicle moment. Then, the drivable area detection device can store the point cloud data of the vehicle at the current vehicle moment into the drivable area detection memory.

[0250] 4) The drivable area detection device can obtain the point cloud data of the vehicle at the current vehicle time and the estimated driving state data of the vehicle at the current vehicle time from the drivable area detection memory. Next, the drivable area detection device can perform drivable area generation processing based on the point cloud data and the estimated driving state data of the vehicle at the current vehicle time, thus obtaining the aforementioned drivable area data of the vehicle at the current vehicle time. Then, the drivable area detection device can store the drivable area data of the vehicle at the current vehicle time into the drivable area detection memory.

[0251] II. Adaptive Adjustment of ROI

[0252] 1) The drivable area detection device can obtain the vehicle's driving status data (i.e., CAN data) and drivable area data at the current vehicle time from the drivable area detection memory. Secondly, the drivable area detection device can determine whether the vehicle's current driving scenario meets a certain ROI parameter switching condition (or driving scenario judgment) based on the vehicle's driving status data (i.e., CAN data) and drivable area data at the current vehicle time. Then, if the vehicle's current driving scenario meets a certain ROI parameter switching condition (for ease of description, it can be called the target ROI parameter switching condition), the drivable area detection device can adjust the vehicle's current ROI parameters to the target ROI parameters corresponding to the aforementioned target ROI parameter switching condition.

[0253] For example, such as Figure 14 As shown, if the target ROI parameters may include: image preprocessing parameters, point cloud generation parameters, and drivable area generation parameters, the drivable area detection device will adjust the image preprocessing parameters related to ROI adjustment corresponding to the image preprocessing part, adjust the point cloud generation parameters related to ROI adjustment corresponding to the point cloud generation part, and adjust the drivable area generation parameters related to ROI adjustment corresponding to the drivable area generation part.

[0254] It should be noted that, as Figure 14 As shown, if the target ROI parameters include any image preprocessing parameters corresponding to the image preprocessing part, and these image preprocessing parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device needs to reinitialize the image preprocessing part; if the target ROI parameters include any point cloud generation parameters corresponding to the point cloud generation part, and these point cloud generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device needs to reinitialize the point cloud generation part; if the target ROI parameters include any drivable area generation parameters corresponding to the drivable area generation part, and these drivable area generation parameters are adjusted during the process of adjusting the vehicle's current ROI parameters to the target ROI parameters, then the drivable area detection device needs to reinitialize the drivable area generation part.

[0255] It should be understood that if the process of adjusting the vehicle's current ROI parameters to the target ROI parameters causes a change in the ROI, the drivable area detection device can also reproject the drivable area data of the vehicle at the current vehicle time to the new ROI corresponding to the target ROI parameters, so that in the next vehicle time, not only can computing energy consumption be reduced, but also drivable area detection can be performed accurately.

[0256] It should be noted that, Figure 14 This example illustrates how a drivable area detection device first performs the drivable area detection component and then the ROI adaptive adjustment component at the current vehicle time. It should be understood that the drivable area detection device can also first perform the ROI adaptive adjustment component and then the drivable area detection component at the current vehicle time; the drivable area detection component can be referenced from... Figure 14 The relevant drivable area detection section and ROI adaptive adjustment section can be found in the reference section. Figure 14 The relevant ROI adaptive adjustment part (but since the vehicle's drivable area data at the current vehicle time has not yet been detected, the vehicle's drivable area data at the previous vehicle time needs to be used instead of the vehicle's drivable area data at the current vehicle time) can be found in the following section. Figure 14The embodiments shown are not limited to the embodiments described in this application.

[0257] It should be understood that the execution timing of the drivable area detection method provided in the embodiments of this application at the next vehicle time can refer to the execution timing of the drivable area detection method provided in the embodiments of this application at the current vehicle time, and this application does not limit it.

[0258] Figure 15 This is a schematic diagram of the structure of a drivable area detection device provided in one embodiment of this application, as shown below. Figure 15 As shown, the drivable area detection device 150 of this application embodiment may include: a first acquisition module 1501, a judgment module 1502, an adjustment module 1503 and a second acquisition module 1504.

[0259] The first acquisition module 1501 is used to acquire the vehicle's driving status data and drivable area data at the current vehicle time.

[0260] The judgment module 1502 is used to determine whether the driving scenario of the vehicle meets the switching conditions of the region of interest (ROI) parameters based on the driving status data and the driving area data.

[0261] The adjustment module 1503 is used to adjust the vehicle's current ROI parameters to the target ROI parameters if the vehicle's driving scenario meets the ROI parameter switching conditions.

[0262] The second acquisition module 1504 is used to acquire the drivable area data of the vehicle at the next vehicle time based on the target ROI parameters.

[0263] In one possible implementation, the target ROI parameters include at least one of the following: image preprocessing parameters, point cloud generation parameters, or drivable area generation parameters;

[0264] The image preprocessing parameters include at least one of the following: the size parameter of the image ROI, the position parameter of the image ROI, or the number of image scaling layers;

[0265] Point cloud generation parameters include at least one of the following: support point grid step size parameter, support point sparsity parameter, or support point distribution method parameter;

[0266] The parameters for generating the drivable area include: the placeholder grid resolution parameter.

[0267] In one possible implementation, the ROI parameter switching condition includes any of the following:

[0268] Preset ROI parameter switching conditions for highway driving scenarios, preset ROI parameter switching conditions for congested road driving scenarios, or preset ROI parameter switching conditions for narrow space uphill and downhill road driving scenarios.

[0269] Among them, the preset ROI parameter switching conditions for the congested road driving scenario include: the vehicle is traveling in the current lane at a speed less than the first preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the current lane at a first preset distance from the vehicle.

[0270] The preset ROI parameter switching conditions for driving scenarios on narrow, uphill and downhill roads include: the vehicle is traveling in the current lane at a speed less than the first preset speed along the same driving direction as the previous vehicle at that moment; the accelerator pedal state and brake pedal state of the vehicle change intermittently; and the vehicle has a movement distance greater than the second preset distance in the direction perpendicular to the horizontal ground.

[0271] In one possible implementation, the preset ROI parameter switching conditions for the highway driving scenario include any one of the following: the preset ROI parameter switching conditions for the first sub-scenario, the preset ROI parameter switching conditions for the second sub-scenario, the preset ROI parameter switching conditions for the third sub-scenario, or the preset ROI parameter switching conditions for the fourth sub-scenario.

[0272] The preset ROI parameter switching conditions for the first sub-scene include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there are no obstacles within the third preset distance of the vehicle in the adjacent lane of the current lane.

[0273] The preset ROI parameter switching conditions for the second sub-scene include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the adjacent lane of the current lane at a third preset distance from the vehicle.

[0274] The preset ROI parameter switching conditions for the third sub-scene include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the current lane at a third preset distance from the vehicle.

[0275] The preset ROI parameter switching conditions for the fourth sub-scenario include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along a direction different from the previous vehicle's travel direction.

[0276] In one possible implementation, the second acquisition module includes:

[0277] The first acquisition unit is used to acquire the estimated driving state data and point cloud data of the vehicle at the next vehicle time based on the target ROI parameters, the estimated driving state data of the vehicle at the current vehicle time, the drivable area data, and the binocular image data of the vehicle at the next vehicle time.

[0278] The second acquisition unit is used to acquire the drivable area data of the vehicle at the next vehicle time based on the estimated driving state data and point cloud data of the vehicle at the next vehicle time.

[0279] In one possible implementation, if the target ROI parameters include image preprocessing parameters and point cloud generation parameters, the first acquisition unit is specifically used for:

[0280] Based on the image preprocessing parameters, the estimated driving state data of the vehicle at the current vehicle time, and the drivable area data, the binocular image data of the vehicle at the next vehicle time is preprocessed to obtain the image processing data of the vehicle at the next vehicle time.

[0281] The driving state estimation data of the vehicle at the current vehicle time and the binocular image data of the vehicle at the next vehicle time are used to perform state estimation processing to obtain the driving state estimation data of the vehicle at the next vehicle time.

[0282] Based on the point cloud generation parameters, point cloud generation processing is performed on the image processing data and driving state estimation data of the vehicle at the next vehicle time step to obtain the point cloud data of the vehicle at the next vehicle time step.

[0283] In one possible implementation, if the target ROI parameters also include: drivable area generation parameters, the second acquisition unit is specifically used for:

[0284] Based on the drivable area generation parameters, the estimated driving state data and point cloud data of the vehicle at the next vehicle time are processed to generate the drivable area data of the vehicle at the next vehicle time.

[0285] In one possible implementation, the above-mentioned device further includes:

[0286] The projection module is used to project the drivable area data of the vehicle at the current vehicle time onto the ROI corresponding to the target ROI parameter.

[0287] In one possible implementation, the driving status data includes at least one of the following: driving speed, driving direction, accelerator pedal status, and brake pedal status.

[0288] The drivable area detection device provided in this application embodiment can be used to execute the technical solutions in the above-described drivable area detection method embodiments of this application. Its implementation principle and technical effects are similar, and will not be repeated here.

[0289] Figure 16 A schematic diagram of the structure of a drivable area detection device provided in another embodiment of this application is shown below. Figure 16 As shown, the drivable area detection device 160 of this application embodiment may include: a processor 1601, a memory 1602, and a communication interface 1603. The communication interface 1603 is used to acquire data to be processed (e.g., driving status data, and / or, binocular image data, etc.); the memory 1602 is used to store program instructions; the processor 1601 is used to call and execute the program instructions stored in the memory 1602. When the processor 1601 executes the program instructions stored in the memory 1602, the drivable area detection device is used to execute the technical solution of the above-described drivable area detection method embodiment of this application on the data to be processed, obtaining processed data (e.g., drivable area data, etc.), so that the communication interface 1603 is also used to output the processed data. Its implementation principle and technical effect are similar and will not be described again here.

[0290] It should be understood that the memory 1602 in the embodiments of this application can also be used to store intermediate result data of the drivable area detection device in the process of executing the technical solution in the above embodiments of the drivable area detection method of this application.

[0291] For example, the communication interfaces involved in the embodiments of this application may include, but are not limited to: an image data interface, and / or, a CAN data interface.

[0292] This application also provides a chip that may include the drivable area detection device described above, or a chip that supports the drivable area detection device in implementing the functions shown in this application.

[0293] For example, when the above method is implemented by a chip within an electronic device, the chip may include a processing unit, and further, the chip may also include a communication unit. The processing unit may be, for example, a processor. When the chip includes a communication unit, the communication unit may be, for example, an input / output interface, pins, or circuits. The processing unit executes all or part of the actions performed by the various processing modules in the embodiments of this application, and the communication unit may perform corresponding receiving or acquiring actions.

[0294] This application also provides an in-vehicle device that may include the drivable area detection device described above.

[0295] Optionally, the vehicle-mounted terminal in this embodiment may further include: Figure 1 The ECU 12 and controller 13 shown are included; of course, the vehicle terminal in this embodiment may also include other devices, which are not limited in this embodiment.

[0296] This application also provides a computer-readable storage medium for storing a computer program that implements the technical solutions in the above-described embodiments of the drivable area detection method of this application. The implementation principle and technical effects are similar and will not be repeated here.

[0297] This application also provides a chip system, which includes a processor and may also include a memory and a communication interface, for implementing the technical solutions in the above-described embodiments of the drivable area detection method of this application. The implementation principle and technical effects are similar, and will not be repeated here. Exemplarily, the chip system may be composed of chips or may include chips and other discrete devices.

[0298] This application also provides a program that, when executed by a processor, performs the technical solutions described in the above-described embodiments of the drivable area detection method. The implementation principle and technical effects are similar and will not be repeated here.

[0299] This application also provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute the technical solution in the above-described drivable area detection method embodiment of this application. Its implementation principle and technical effect are similar, and will not be repeated here.

[0300] The processors involved in the embodiments of this application can be general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0301] The memory involved in the embodiments of this application can be non-volatile memory, such as hard disk drive (HDD) or solid-state drive (SSD), or it can be volatile memory, such as random-access memory (RAM). Memory is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0302] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0303] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0304] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0305] Those skilled in the art will understand that, in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0306] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

Claims

1. A method for detecting drivable areas, characterized in that, include: Obtain the vehicle's driving status data and drivable area data at the current vehicle time; Based on the driving status data and the drivable area data, it is determined whether the driving scenario of the vehicle meets the ROI parameter switching conditions. Different driving scenarios correspond to different ROI parameter switching conditions. The driving scenarios include highway driving scenarios, congested road driving scenarios, and narrow space uphill and downhill road driving scenarios. If the driving scenario of the vehicle meets the ROI parameter switching conditions, the current ROI parameter of the vehicle is adjusted to the target ROI parameter. The target ROI parameter is the ROI parameter corresponding to the currently met ROI parameter switching conditions. Different driving scenarios correspond to different target ROI parameters. The target ROI parameter includes at least one of the following: image preprocessing parameters, point cloud generation parameters, or drivable area generation parameters. Based on the target ROI parameters, obtain the drivable area data of the vehicle at the next vehicle time.

2. The method according to claim 1, Its features are, The image preprocessing parameters include at least one of the following: the size parameter of the image ROI, the position parameter of the image ROI, or the number of image scaling layers; The point cloud generation parameters include at least one of the following: support point grid step size parameter, support point sparsity parameter, or support point distribution method parameter; The parameters for generating the drivable area include: the placeholder grid resolution parameter.

3. The method according to claim 1 or 2, characterized in that, The ROI parameter switching conditions include any one of the following: Preset ROI parameter switching conditions for highway driving scenarios, preset ROI parameter switching conditions for congested road driving scenarios, or preset ROI parameter switching conditions for narrow space uphill and downhill road driving scenarios. The preset ROI parameter switching conditions for the congested road driving scenario include: the vehicle is traveling in the current lane at a speed less than the first preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the current lane at a first preset distance from the vehicle. The preset ROI parameter switching conditions for the driving scenario on narrow, uphill and downhill roads include: the vehicle is traveling in the current lane at a speed less than the first preset speed along the same driving direction as the previous vehicle at that moment; the accelerator pedal state and brake pedal state of the vehicle change intermittently; and the vehicle has a movement distance greater than the second preset distance in the direction perpendicular to the horizontal ground.

4. The method according to claim 3, characterized in that, The preset ROI parameter switching conditions for the highway driving scenario include any one of the following: preset ROI parameter switching conditions for the first sub-scenario, preset ROI parameter switching conditions for the second sub-scenario, preset ROI parameter switching conditions for the third sub-scenario, or preset ROI parameter switching conditions for the fourth sub-scenario. The preset ROI parameter switching conditions for the first sub-scenario include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there are no obstacles within a third preset distance from the vehicle in the adjacent lanes of the current lane. The preset ROI parameter switching conditions for the second sub-scenario include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the adjacent lane of the current lane at a third preset distance from the vehicle; The preset ROI parameter switching conditions for the third sub-scene include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the current lane at a third preset distance from the vehicle; The preset ROI parameter switching conditions for the fourth sub-scenario include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along a direction different from the previous vehicle's travel direction.

5. The method according to claim 2 or 4, characterized in that, The step of obtaining the drivable area data of the vehicle at the next vehicle time step based on the target ROI parameters includes: Based on the target ROI parameters, the estimated driving state data of the vehicle at the current vehicle time, the drivable area data, and the binocular image data of the vehicle at the next vehicle time, the estimated driving state data and point cloud data of the vehicle at the next vehicle time are obtained. Based on the estimated driving status data and point cloud data of the vehicle at the next vehicle time, the drivable area data of the vehicle at the next vehicle time is obtained.

6. The method according to claim 5, characterized in that, If the target ROI parameters include the image preprocessing parameters and the point cloud generation parameters, the step of obtaining the vehicle's driving state estimation data and point cloud data at the next vehicle time based on the target ROI parameters, the vehicle's driving state estimation data at the current vehicle time, the drivable area data, and the vehicle's binocular image data at the next vehicle time includes: Based on the image preprocessing parameters, the estimated driving state data of the vehicle at the current vehicle time, and the drivable area data, the binocular image data of the vehicle at the next vehicle time is preprocessed to obtain the image processing data of the vehicle at the next vehicle time. Based on the estimated driving state data of the vehicle at the current vehicle time and the binocular image data of the vehicle at the next vehicle time, state estimation processing is performed to obtain the estimated driving state data of the vehicle at the next vehicle time. Based on the point cloud generation parameters, point cloud generation processing is performed on the image processing data and driving state estimation data of the vehicle at the next vehicle time to obtain the point cloud data of the vehicle at the next vehicle time.

7. The method according to claim 6, characterized in that, If the target ROI parameters further include: the drivable area generation parameters, the step of obtaining the drivable area data of the vehicle at the next vehicle time based on the estimated driving state data and point cloud data of the vehicle at the next vehicle time includes: Based on the drivable area generation parameters, the estimated driving state data and point cloud data of the vehicle at the next vehicle time are processed to generate a drivable area, thereby obtaining the drivable area data of the vehicle at the next vehicle time.

8. The method according to any one of claims 1-2, 4, and 6-7, characterized in that, Before obtaining the drivable area data of the vehicle at the next vehicle time step based on the target ROI parameters, the method further includes: The drivable area data of the vehicle at the current vehicle time is projected onto the ROI corresponding to the target ROI parameter.

9. The method according to any one of claims 1-2, 4, and 6-7, characterized in that, The driving status data includes at least one of the following: driving speed, driving direction, accelerator pedal status, and brake pedal status.

10. A drivable area detection device, characterized in that, include: The first acquisition module is used to acquire the vehicle's driving status data and drivable area data at the current vehicle time. The judgment module is used to determine whether the driving scenario of the vehicle meets the switching conditions of the region of interest (ROI) parameters based on the driving status data and the drivable area data. Different driving scenarios correspond to different ROI parameter switching conditions. The driving scenarios include highway driving scenarios, congested road driving scenarios, and narrow space uphill and downhill road driving scenarios. The adjustment module is used to adjust the current ROI parameter of the vehicle to the target ROI parameter if the driving scenario of the vehicle meets the ROI parameter switching condition. The target ROI parameter is the ROI parameter corresponding to the currently met ROI parameter switching condition. Different driving scenarios correspond to different target ROI parameters. The target ROI parameter includes at least one of the following: image preprocessing parameters, point cloud generation parameters, or drivable area generation parameters. The second acquisition module is used to acquire the drivable area data of the vehicle at the next vehicle time based on the target ROI parameters.

11. The apparatus according to claim 10, Its features are, The image preprocessing parameters include at least one of the following: the size parameter of the image ROI, the position parameter of the image ROI, or the number of image scaling layers; The point cloud generation parameters include at least one of the following: support point grid step size parameter, support point sparsity parameter, or support point distribution method parameter; The parameters for generating the drivable area include: the placeholder grid resolution parameter.

12. The apparatus according to claim 10 or 11, characterized in that, The ROI parameter switching conditions include any one of the following: Preset ROI parameter switching conditions for highway driving scenarios, preset ROI parameter switching conditions for congested road driving scenarios, or preset ROI parameter switching conditions for narrow space uphill and downhill road driving scenarios. The preset ROI parameter switching conditions for the congested road driving scenario include: the vehicle is traveling in the current lane at a speed less than the first preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the current lane at a first preset distance from the vehicle. The preset ROI parameter switching conditions for the driving scenario on narrow, uphill and downhill roads include: the vehicle is traveling in the current lane at a speed less than the first preset speed along the same driving direction as the previous vehicle at that moment; the accelerator pedal state and brake pedal state of the vehicle change intermittently; and the vehicle has a movement distance greater than the second preset distance in the direction perpendicular to the horizontal ground.

13. The apparatus according to claim 12, characterized in that, The preset ROI parameter switching conditions for the highway driving scenario include any one of the following: preset ROI parameter switching conditions for the first sub-scenario, preset ROI parameter switching conditions for the second sub-scenario, preset ROI parameter switching conditions for the third sub-scenario, or preset ROI parameter switching conditions for the fourth sub-scenario. The preset ROI parameter switching conditions for the first sub-scenario include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there are no obstacles within a third preset distance from the vehicle in the adjacent lanes of the current lane. The preset ROI parameter switching conditions for the second sub-scenario include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the adjacent lane of the current lane at a third preset distance from the vehicle; The preset ROI parameter switching conditions for the third sub-scene include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along the direction of travel of the previous vehicle, and there is an obstacle in the current lane at a third preset distance from the vehicle; The preset ROI parameter switching conditions for the fourth sub-scenario include: the vehicle is traveling in the current lane at a speed greater than the second preset speed along a direction different from the previous vehicle's travel direction.

14. The apparatus according to claim 11 or 13, characterized in that, The second acquisition module includes: The first acquisition unit is used to acquire the estimated driving state data and point cloud data of the vehicle at the next vehicle time based on the target ROI parameters, the estimated driving state data of the vehicle at the current vehicle time, the drivable area data, and the binocular image data of the vehicle at the next vehicle time. The second acquisition unit is used to acquire the drivable area data of the vehicle at the next vehicle time based on the estimated driving state data and point cloud data of the vehicle at the next vehicle time.

15. The apparatus according to claim 14, characterized in that, If the target ROI parameters include the image preprocessing parameters and the point cloud generation parameters, the first acquisition unit is specifically used for: Based on the image preprocessing parameters, the estimated driving state data of the vehicle at the current vehicle time, and the drivable area data, the binocular image data of the vehicle at the next vehicle time is preprocessed to obtain the image processing data of the vehicle at the next vehicle time. Based on the estimated driving state data of the vehicle at the current vehicle time and the binocular image data of the vehicle at the next vehicle time, state estimation processing is performed to obtain the estimated driving state data of the vehicle at the next vehicle time. Based on the point cloud generation parameters, point cloud generation processing is performed on the image processing data and driving state estimation data of the vehicle at the next vehicle time to obtain the point cloud data of the vehicle at the next vehicle time.

16. The apparatus according to claim 15, characterized in that, If the target ROI parameters further include: the drivable area generation parameters, the second acquisition unit is specifically used for: Based on the drivable area generation parameters, the estimated driving state data and point cloud data of the vehicle at the next vehicle time are processed to generate a drivable area, thereby obtaining the drivable area data of the vehicle at the next vehicle time.

17. The apparatus according to any one of claims 10-11, 13, and 15-16, characterized in that, The device further includes: The projection module is used to project the drivable area data of the vehicle at the current vehicle time onto the ROI corresponding to the target ROI parameter.

18. The apparatus according to any one of claims 10-11, 13, 15-16, characterized in that, The driving status data includes at least one of the following: driving speed, driving direction, accelerator pedal status, and brake pedal status.

19. A drivable area detection device, characterized in that, include: Processor, memory, and communication interface; The communication interface is used to acquire data to be processed. The memory is used to store program instructions; The processor is configured to call and execute program instructions stored in the memory. When the processor executes the program instructions stored in the memory, the drivable area detection device is configured to perform the method as described in any one of claims 1 to 9 on the data to be processed to obtain processed data. The communication interface is also used to output processed data.

20. A chip, characterized in that, Includes the drivable area detection device as described in claim 19.

21. A vehicle-mounted device, characterized in that, Includes the drivable area detection device as described in claim 19.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for implementing the method as described in any one of claims 1-9.

23. A computer program product, characterized in that, The program product includes computer instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-9.

Images