Vehicle turning control method and apparatus based on binocular stereoscopic vision

Through binocular stereoscopic vision sensors generate three-dimensional point clouds on unstructured roads and filter them, the accuracy problem of unstructured road turn path planning is solved, and vehicle turn path planning is realized in the absence of structured information.

WO2024244791A1PCT designated stage expired Publication Date: 2024-12-05BEIJING PHIGENT TECHNOLOGY CO LTD

Patent Information

Application Number
PCT/CN2024/088439
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-30
Filing Date
2024-04-17
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

The prior art is difficult to accurately plan turn paths on unstructured roads, especially because monocular solutions cannot generate spatial point clouds and lidars are difficult to generate dense point clouds, making it difficult to determine the driving paths when turning at intersections lacking structured information.

Method used

The binocular stereoscopic vision sensor is used to obtain the initial image data, perform three-dimensional stereoscopic matching to generate three-dimensional point cloud data, and filter based on preset height thresholds to determine the travelable space, thereby planning the turning path.

Benefits of technology

By acquiring dense three-dimensional point cloud data, it is possible to accurately avoid obstacles in unstructured roads, determine a driving space, and achieve accurate turn and driving of vehicles on unstructured roads without relying on structured road information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024088439_05122024_PF_FP_ABST
    Figure CN2024088439_05122024_PF_FP_ABST
Patent Text Reader

Abstract

The embodiments of the present invention relate to the technical field of vehicle assisted driving. Provided are a vehicle turning control method and apparatus based on binocular stereoscopic vision. The method comprises: when a vehicle turns on an unstructured road, acquiring initial image data, which is collected by means of a binocular stereoscopic vision sensor; performing three-dimensional matching on the initial image data, so as to generate three-dimensional point cloud data; filtering the three-dimensional point cloud data on the basis of a preset height threshold value, so as to obtain a travelable space; and determining a turning path according to the travelable space. By means of the embodiments of the present invention, in the case of turning on an unstructured road, three-dimensional reconstruction can be performed on the unstructured road, such that a travelable space where an obstacle is accurately avoided can be determined in the unstructured road, and then a turning path is determined to make a vehicle accurately drive into a target intersection.
Need to check novelty before this filing date? Find Prior Art

Description

Vehicle turning control method and device based on binocular stereo vision

[0001] This application claims priority to a Chinese patent application filed with the Patent Office of China on May 30, 2023, with application number 202310627877.2 and invention name “A vehicle turning control method and device based on binocular stereo vision”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of prefabricated vehicle assisted driving technology, and in particular to a vehicle turning control method based on binocular stereo vision, a vehicle turning control device based on binocular stereo vision, a vehicle and a storage medium. Background Art

[0003] Currently, many intersections do not have designated lanes for turns. This means that solutions that use structured road information for path planning are not applicable to unstructured roads. For turning scenarios at intersections on unstructured roads, semantic segmentation is required to determine the driving path. However, due to the limitations of monocular solutions that cannot generate spatial point clouds and lidar's difficulty in generating dense point clouds, it is difficult to accurately plan the driving path in this scenario. Therefore, the method of planning turning paths by identifying structured road information (such as lane lines) and semantic segmentation is not accurate.

[0004] Summary of the Invention

[0005] In view of the above problems, embodiments of the present application are proposed to provide a vehicle turning control method based on binocular stereo vision, a vehicle turning control device based on binocular stereo vision, a vehicle and a storage medium that overcome the above problems or at least partially solve the above problems.

[0006] In some embodiments of the present application, a vehicle turning control method based on binocular stereo vision is disclosed. A binocular stereo vision sensor is installed on a vehicle, including:

[0007] When the vehicle turns on an unstructured road, initial image data collected by the binocular stereo vision sensor is obtained;

[0008] Perform three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data;

[0009] Filter the 3D point cloud data based on a preset height threshold to obtain the drivable space;

[0010] Determine the turning path based on the drivable space.

[0011] Optionally, the step of filtering the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space includes:

[0012] Use the flat point cloud in the 3D point cloud data as the reference plane;

[0013] Based on the reference plane and the preset height threshold, the three-dimensional point cloud data is filtered to obtain the drivable space.

[0014] Optionally, the preset height threshold includes a positive height threshold; and the step of filtering the three-dimensional point cloud data based on the reference plane and the preset height threshold to obtain the drivable space includes:

[0015] Based on the reference plane, the point clouds below the positive height threshold in the three-dimensional point cloud data are filtered as the first target point cloud data;

[0016] The space formed by the first target point cloud data is determined as a drivable space.

[0017] Optionally, the preset height threshold includes a negative height threshold; and the step of filtering the three-dimensional point cloud data based on the reference plane and the preset height threshold to obtain the drivable space includes:

[0018] Based on the reference plane, the point clouds in the three-dimensional point cloud data that are higher than the negative height threshold are filtered as the second target point cloud data;

[0019] The space formed by the second target point cloud data is determined as a drivable space.

[0020] Optionally, the step of determining a turning path according to the drivable space includes:

[0021] Get vehicle size information;

[0022] Based on the vehicle body size information, the turning path is planned in the drivable space.

[0023] Optionally, before the step of determining the turning path according to the drivable space, the method further includes:

[0024] When the vehicle drives onto a structured road, the three primary color point cloud information of the road is obtained;

[0025] Determine road structure information based on the road's three primary color point cloud information;

[0026] Road structure information is used to filter the drivable space.

[0027] Optionally, the method further comprises:

[0028] Control the vehicle to turn along the turning path.

[0029] In some embodiments of the present application, a vehicle turning control device based on binocular stereo vision is disclosed. The binocular stereo vision sensor is installed on the vehicle, and the device includes:

[0030] The first acquisition module is configured to acquire initial image data collected by the binocular stereo vision sensor when the vehicle travels on an unstructured road;

[0031] A 3D stereo matching module is used to perform 3D stereo matching on the initial image data to generate 3D point cloud data;

[0032] A filtering module is used to filter the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space;

[0033] Planning module, used to determine the turning path based on the drivable space

[0034] In some embodiments of the present application, the embodiments of the present application also disclose a vehicle, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the steps of the above-mentioned vehicle turning control method based on binocular stereo vision are implemented.

[0035] In some embodiments of the present application, the embodiments of the present application further disclose a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned vehicle turning control method based on binocular stereo vision are implemented.

[0036] The embodiments of the present application include the following advantages:

[0037] The present embodiment of the present invention obtains initial image data collected by a binocular stereo vision sensor when a vehicle turns on an unstructured road; performs 3D stereo matching on the initial image data to generate 3D point cloud data; filters the 3D point cloud data based on a preset height threshold to obtain a drivable space; and determines a turning path based on the drivable space. By using binocular stereo vision to obtain a dense point cloud and perform 3D reconstruction of the unstructured road, it is possible to determine a drivable space on the unstructured road that accurately avoids obstacles. The turning path can be determined independently of structured road information, allowing the vehicle to accurately enter the target intersection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] FIG1 is a flowchart of an embodiment of a vehicle turning control method based on binocular stereo vision of the present application;

[0039] FIG2 is a flowchart of another embodiment of a vehicle turning control method based on binocular stereo vision of the present application;

[0040] FIG3 is a schematic diagram of height screening of an example of a vehicle turning control method based on binocular stereo vision of the present application;

[0041] FIG4 is a schematic diagram of a turning path determination example of a vehicle turning control method based on binocular stereo vision of the present application;

[0042] FIG5 is a structural block diagram of an embodiment of a vehicle turning control device based on binocular stereo vision of the present application;

[0043] FIG6 is a structural block diagram of an electronic device provided in an embodiment of the present application;

[0044] FIG7 is a structural block diagram of a storage medium provided in an embodiment of the present application. Specific embodiments

[0045] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0046] Referring to Figure 1, a flowchart illustrating the steps of an embodiment of a vehicle turning control method based on binocular stereo vision is shown. In this embodiment, a binocular stereo vision sensor is installed on a vehicle. The specific installation location can be determined based on the actual vehicle conditions and is not limited in this embodiment. For example, the binocular stereo vision sensor is installed on the top inside of the vehicle's front windshield.

[0047] The vehicle turning control method based on binocular stereo vision may specifically include the following steps:

[0048] Step 101: When a vehicle turns on an unstructured road, initial image data collected by a binocular stereo vision sensor is acquired.

[0049] In the embodiments of this application, an unstructured road refers to a road that lacks lane markings, lane guide lines, or other ground markings, such as an intersection without ground markings. When a vehicle turns on an unstructured road, it can obtain initial image data of the road captured by the binocular stereo vision sensor.

[0050] Step 102: Perform three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data.

[0051] After acquiring the initial image data collected by the binocular stereo vision sensor, the initial image data can be subjected to 3D stereo matching to perform 3D reconstruction of the unstructured road and obtain dense 3D point cloud data. Due to the wide variety of mature 3D image matching methods, the present embodiment does not specifically limit the 3D image matching method, such as semi-global matching methods and deep learning methods. Those skilled in the art can select an appropriate 3D image matching method based on actual circumstances to perform 3D stereo matching on the initial image data collected by the binocular stereo vision sensor, thereby reconstructing the 3D scene.

[0052] Step 103: Filter the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space.

[0053] After obtaining the 3D point cloud data, the 3D point cloud data is filtered based on a preset height threshold, and the vehicle's drivable space is determined based on the spatial information corresponding to the 3D point cloud data. The drivable space refers to the space in which the vehicle can travel without obstacles. Those skilled in the art can set the preset height threshold as needed, and this embodiment of the application does not limit this.

[0054] Step 104: Determine a turning path based on the drivable space.

[0055] Then, based on the drivable space, the path that the vehicle can travel is determined, and the optimal route is selected as the turning path.

[0056] The present embodiment of the present invention obtains initial image data collected by a binocular stereo vision sensor when a vehicle turns on an unstructured road; performs 3D stereo matching on the initial image data to generate 3D point cloud data; filters the 3D point cloud data based on a preset height threshold to obtain a drivable space; and determines a turning path based on the drivable space. By using binocular stereo vision to obtain a dense point cloud and perform 3D reconstruction of the unstructured road, it is possible to determine a drivable space on the unstructured road that accurately avoids obstacles. The turning path can be determined independently of structured road information, allowing the vehicle to accurately enter the target intersection.

[0057] Referring to Figure 2, a flowchart of another embodiment of a vehicle turning control method based on binocular stereo vision is shown. In this embodiment, a binocular stereo vision sensor is installed on a vehicle. The vehicle turning control method based on binocular stereo vision may specifically include the following steps:

[0058] Step 201 : when a vehicle turns on an unstructured road, initial image data collected by a binocular stereo vision sensor is acquired.

[0059] In an embodiment of the present application, when a vehicle is on an unstructured road, initial image data collected by a binocular stereo vision sensor is acquired in real time, and vehicle operating condition data of the vehicle is acquired in real time.

[0060] Step 202: Perform three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data. The three-dimensional point cloud data may include point cloud data of the road surface and obstacles of the unstructured road.

[0061] In addition, the initial image data is collected in the camera coordinate system of the binocular stereo vision sensor. In order to be in the same coordinate system as the vehicle for data processing, the generated three-dimensional point cloud data can be mapped to the vehicle coordinate system of the vehicle.

[0062] Specifically, an offset matrix between the camera coordinate system of the binocular stereo vision sensor and the vehicle coordinate system may be calculated, and the three-dimensional point cloud data may be mapped to the vehicle coordinate system based on the offset matrix.

[0063] Step 203: Filter the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space.

[0064] After obtaining the three-dimensional point cloud data, it can be filtered based on a preset height threshold to filter out point clouds with obvious height deviations from the ground, such as potholes, curbs, and flower beds, and determine the space corresponding to the remaining point clouds as drivable space.

[0065] In an optional embodiment of the present application, the step of filtering the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space includes:

[0066] Sub-step S2031, using a flat point cloud in the three-dimensional point cloud data as a reference plane;

[0067] In an embodiment of the present application, the flat point cloud in the three-dimensional point cloud data can be used as a reference plane, that is, the space corresponding to the flat point cloud in the three-dimensional point cloud data can be used as a flat road surface to identify obstacles.

[0068] Sub-step S2032: filtering the three-dimensional point cloud data based on the reference plane and a preset height threshold to obtain a drivable space.

[0069] Then, based on the reference plane and combined with the preset height threshold, the space that does not meet the requirements is filtered out to obtain the drivable space.

[0070] Specifically, the preset height threshold includes a positive height threshold; based on the reference plane, combined with the preset height threshold, the three-dimensional point cloud data is filtered to obtain the drivable space, including the following steps: based on the reference plane, filtering the point clouds in the three-dimensional point cloud data that are below the positive height threshold as the first target point cloud data; and determining the space composed of the first target point cloud data as the drivable space.

[0071] In the embodiment of the present application, the preset height threshold is specifically a positive height threshold, i.e., the height threshold is used to determine whether there are obstacles that hinder vehicle travel in the space protruding from the ground, thereby filtering them out. Specifically, based on a reference plane, a positive height threshold is added to this reference plane. Spaces above this positive height threshold are obstacles that hinder vehicle travel, such as flower beds and curbs. Point clouds below the positive height threshold are defined as first target point cloud data, and the space formed by this first target point cloud data is determined as drivable space.

[0072] Specifically, the preset height threshold includes a negative height threshold; based on the reference plane, combined with the preset height threshold, the three-dimensional point cloud data is filtered to obtain the drivable space, including the following steps: based on the reference plane, filtering the point clouds in the three-dimensional point cloud data that are higher than the negative height threshold as second target point cloud data; and determining the space composed of the second target point cloud data as the drivable space.

[0073] In the embodiment of the present application, the preset height threshold is specifically a negative height threshold. That is, the height threshold is used to determine whether there are obstacles below the ground, thereby filtering them out. Specifically, based on a reference plane, a negative height threshold is added to the reference plane. Spaces below this negative height threshold are considered obstacles, such as deep pits, that could hinder vehicle travel. Point clouds above the negative height threshold are defined as second target point cloud data, and the space formed by the second target point cloud data is determined to be drivable space.

[0074] In addition, in an optional embodiment of the present application, when the vehicle travels on a structured road, the drivable space can be further determined in combination with the information of the structured road, so that the turning path can be more consistent with driving regulations. Specifically, the method further includes the following steps:

[0075] Step S1: When the vehicle travels on a structured road, the three primary colors point cloud information of the road is obtained.

[0076] When a vehicle travels on a structured road, lane information is provided, allowing for the acquisition of three-primary-color (RGB) point cloud information. This RGB point cloud information can be collected and generated using the vehicle's onboard RGB vision sensor. Alternatively, the binocular vision sensor can be a RGB sensor.

[0077] Step S2: determining the road structure information based on the road three primary color point cloud information.

[0078] According to the three primary color point cloud information of the road, the specific road structure information of the road is determined in three-dimensional space.

[0079] Step S3: Using the road structure information, filter the drivable space.

[0080] Then, based on the road structure information, the oncoming lane and other spaces that cannot be driven according to the driving regulations are determined to further filter out the above-mentioned drivable space so that the drivable space can meet the actual driving regulations.

[0081] Step 204: Determine a turning path based on the drivable space.

[0082] After the drivable space is obtained, an optimal driving path may be determined based on the drivable space as a turning path.

[0083] In an optional embodiment of the present application, the step of determining a turning path according to the drivable space includes:

[0084] Sub-step S2041, obtaining vehicle body size information;

[0085] In an embodiment of the present application, the vehicle's body size information can be further obtained and integrated into the path planning.

[0086] Sub-step S2042 , planning and generating a turning path in the drivable space based on the vehicle body size information.

[0087] Based on the vehicle body size information, planning can be performed in the drivable space so that the vehicle can pass safely in the drivable space, and then multiple planned drivable spaces can be connected to generate a turning path.

[0088] Step 205: Control the vehicle to turn along the turning path.

[0089] After obtaining the turning path, the vehicle's speed, steering angle and other parameters are controlled to control the vehicle to turn along the turning path and drive to the target intersection.

[0090] The embodiments of the present application utilize binocular stereo vision to obtain initial image data captured by a binocular stereo vision sensor when a vehicle is turning on an unstructured road; perform 3D stereo matching on the initial image data to generate 3D point cloud data; filter the 3D point cloud data based on a preset height threshold to obtain a drivable space; determine a turning path based on the drivable space; and control the vehicle to turn along the turning path. By utilizing binocular stereo vision to obtain a dense point cloud and perform 3D reconstruction of the unstructured road, it is possible to determine a drivable space on the unstructured road that accurately avoids obstacles. This allows the vehicle to accurately navigate the turning path to the target intersection without relying on structured road information.

[0091] In order to enable those skilled in the art to better understand the embodiments of the present application, the embodiments of the present application are described below using an example:

[0092] 1. The binocular stereo vision sensor is installed in the vehicle. The coordinate system of the binocular stereo vision sensor can be mapped to the vehicle coordinate system through methods such as body calibration.

[0093] 2. When a vehicle reaches an unstructured intersection, binocular stereo vision sensors can be used to obtain a precise, dense intersection point cloud (3D point cloud data). Height thresholds can be used to demarcate the drivable area. As shown in Figure 3 below, the ground appears as a flat surface in the point cloud, while objects such as potholes, curbs, and flower beds have significant height deviations from the ground.

[0094] 3. Based on the divided drivable area and the vehicle body size information, the driving path can be planned in the three-dimensional space of the point cloud to obtain the turning path, thereby avoiding obstacles and accurately entering the next adjacent intersection.

[0095] 4. For intersections with structured roads such as lane markings, the two can be combined to obtain RGBD point cloud information and plan a driving path that complies with traffic regulations. For the left turn use case shown in Figure 4, since the turn is required to be a wide turn, when Turn Path A, Turn Path B, and Turn Path C are obtained, Turn Path A is determined to be the driving turning path.

[0096] 5. Use binocular stereo vision cameras to obtain point cloud information in the space and obtain the drivable space at the intersection where the vehicle is to turn.

[0097] 6. Further determine the drivable space (such as double yellow lines) using the structured information from the RGB (three primary colors) camera. This allows for spatial planning of turning paths.

[0098] It should be noted that for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the order of the actions described, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0099] 5 shows a block diagram of an embodiment of a vehicle turning control device based on binocular stereo vision of the present application. In this embodiment of the present application, a binocular stereo vision sensor is installed on a vehicle, and the vehicle turning control device based on binocular stereo vision may specifically include the following modules:

[0100] The first acquisition module 501 is configured to acquire initial image data collected by a binocular stereo vision sensor when the vehicle travels on an unstructured road;

[0101] A 3D stereo matching module 502 is used to perform 3D stereo matching on the initial image data to generate 3D point cloud data;

[0102] A filtering module 503 is used to filter the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space;

[0103] The planning module 504 is used to determine a turning path based on the drivable space.

[0104] In an optional embodiment of the present application, the filtering module 503 includes:

[0105] The reference submodule is used to use the flat point cloud in the 3D point cloud data as the reference plane;

[0106] The filtering submodule is used to filter the three-dimensional point cloud data based on the reference plane and the preset height threshold to obtain the drivable space.

[0107] In an optional embodiment of the present application, the preset height threshold includes a positive height threshold; and the filtering submodule includes:

[0108] A first filtering unit is configured to filter, based on a reference plane, point clouds in the three-dimensional point cloud data that are below a positive height threshold as first target point cloud data;

[0109] The first drivable space determining unit is configured to determine a space formed by first target point cloud data as a drivable space.

[0110] In an optional embodiment of the present application, the preset height threshold includes a negative height threshold; and the filtering submodule includes:

[0111] A second filtering unit is configured to filter point clouds in the three-dimensional point cloud data that are higher than a negative height threshold based on the reference plane as second target point cloud data;

[0112] The first drivable space determining unit is configured to determine the space formed by the second target point cloud data as the drivable space.

[0113] In an optional embodiment of the present application, the planning module 504 includes:

[0114] The size acquisition submodule is used to obtain vehicle body size information;

[0115] The planning submodule is used to plan and generate a turning path in the drivable space based on the vehicle body size information.

[0116] In an optional embodiment of the present application, the device further includes:

[0117] The second acquisition module is used to obtain the three-primary-color point cloud information of the road when the vehicle travels on the structured road;

[0118] A road structure information determination module is used to determine road structure information based on the road three primary color point cloud information;

[0119] The specification module is used to filter the drivable space using road structure information.

[0120] In an optional embodiment of the present application, the device further includes:

[0121] The control module is used to control the vehicle to turn along the turning path.

[0122] The present embodiment of the present invention obtains initial point cloud data collected by a binocular stereo vision sensor when a vehicle turns on an unstructured road; performs three-dimensional stereo matching on the initial point cloud data to generate three-dimensional point cloud data; filters the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space; and determines a turning path based on the drivable space. By using binocular stereo vision to obtain a dense point cloud and perform three-dimensional reconstruction of the unstructured road, it is possible to determine a drivable space on the unstructured road that accurately avoids obstacles. The turning path can be determined independently of structured road information, allowing the vehicle to accurately enter the target intersection.

[0123] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0124] 6 , an embodiment of the present application further provides a vehicle, including:

[0125] The processor 601 and the storage medium 602 store a computer program executable by the processor 601. When the vehicle is running, the processor 601 executes the computer program to perform the vehicle turning control method based on binocular stereo vision as described in any embodiment of the present application. The vehicle turning control method based on binocular stereo vision includes:

[0126] When the vehicle turns on an unstructured road, the initial point cloud data collected by the binocular stereo vision sensor is obtained;

[0127] Perform three-dimensional stereo matching on the initial point cloud data to generate three-dimensional point cloud data;

[0128] Filter the 3D point cloud data based on a preset height threshold to obtain the drivable space;

[0129] Determine the turning path based on the drivable space.

[0130] Optionally, the step of filtering the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space includes:

[0131] Use the flat point cloud in the 3D point cloud data as the reference plane;

[0132] Based on the reference plane and the preset height threshold, the three-dimensional point cloud data is filtered to obtain the drivable space.

[0133] Optionally, the preset height threshold includes a positive height threshold; and the step of filtering the three-dimensional point cloud data based on the reference plane and the preset height threshold to obtain the drivable space includes:

[0134] Based on the reference plane, the point clouds below the positive height threshold in the three-dimensional point cloud data are filtered as the first target point cloud data;

[0135] The space formed by the first target point cloud data is determined as a drivable space.

[0136] Optionally, the preset height threshold includes a negative height threshold; and the step of filtering the three-dimensional point cloud data based on the reference plane and the preset height threshold to obtain the drivable space includes:

[0137] Based on the reference plane, the point clouds in the three-dimensional point cloud data that are higher than the negative height threshold are filtered as the second target point cloud data;

[0138] The space formed by the second target point cloud data is determined as a drivable space.

[0139] Optionally, the step of determining a turning path according to the drivable space includes:

[0140] Get vehicle size information;

[0141] Based on the vehicle body size information, the turning path is planned in the drivable space.

[0142] Optionally, before the step of determining the turning path according to the drivable space, the method further includes:

[0143] When the vehicle drives onto a structured road, the three primary color point cloud information of the road is obtained;

[0144] Determine road structure information based on the road's three primary color point cloud information;

[0145] Road structure information is used to filter the drivable space.

[0146] Optionally, the method further comprises:

[0147] Control the vehicle to turn along the turning path.

[0148] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0149] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0150] The present embodiment of the present invention obtains initial image data collected by a binocular stereo vision sensor when a vehicle turns on an unstructured road; performs 3D stereo matching on the initial image data to generate 3D point cloud data; filters the 3D point cloud data based on a preset height threshold to obtain a drivable space; and determines a turning path based on the drivable space. By using binocular stereo vision to obtain a dense point cloud and perform 3D reconstruction of the unstructured road, it is possible to determine a drivable space on the unstructured road that accurately avoids obstacles. The turning path can be determined independently of structured road information, allowing the vehicle to accurately enter the target intersection.

[0151] 7 , the present invention also provides a computer-readable storage medium 701 , which stores a computer program. When the computer program is executed by a processor, the vehicle turning control method based on binocular stereo vision according to any of the embodiments of the present invention is executed. The vehicle turning control method based on binocular stereo vision includes:

[0152] When the vehicle turns on an unstructured road, initial image data collected by the binocular stereo vision sensor is obtained;

[0153] Perform three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data;

[0154] Filter the 3D point cloud data based on a preset height threshold to obtain the drivable space;

[0155] Determine the turning path based on the drivable space.

[0156] Optionally, the step of filtering the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space includes:

[0157] Use the flat point cloud in the 3D point cloud data as the reference plane;

[0158] Based on the reference plane and the preset height threshold, the three-dimensional point cloud data is filtered to obtain the drivable space.

[0159] Optionally, the preset height threshold includes a positive height threshold; and the step of filtering the three-dimensional point cloud data based on the reference plane and the preset height threshold to obtain the drivable space includes:

[0160] Based on the reference plane, the point clouds below the positive height threshold in the three-dimensional point cloud data are filtered as the first target point cloud data;

[0161] The space formed by the first target point cloud data is determined as a drivable space.

[0162] Optionally, the preset height threshold includes a negative height threshold; and the step of filtering the three-dimensional point cloud data based on the reference plane and the preset height threshold to obtain the drivable space includes:

[0163] Based on the reference plane, the point clouds in the three-dimensional point cloud data that are higher than the negative height threshold are filtered as the second target point cloud data;

[0164] The space formed by the second target point cloud data is determined as a drivable space.

[0165] Optionally, the step of determining a turning path according to the drivable space includes:

[0166] Get vehicle size information;

[0167] Based on the vehicle body size information, the turning path is planned in the drivable space.

[0168] Optionally, before the step of determining the turning path according to the drivable space, the method further includes:

[0169] When the vehicle drives onto a structured road, the three primary color point cloud information of the road is obtained;

[0170] Determine road structure information based on the road's three primary color point cloud information;

[0171] Road structure information is used to filter the drivable space.

[0172] Optionally, the method further comprises:

[0173] Control the vehicle to turn along the turning path.

[0174] The present embodiment of the present invention obtains initial image data collected by a binocular stereo vision sensor when a vehicle turns on an unstructured road; performs 3D stereo matching on the initial image data to generate 3D point cloud data; filters the 3D point cloud data based on a preset height threshold to obtain a drivable space; and determines a turning path based on the drivable space. By using binocular stereo vision to obtain a dense point cloud and perform 3D reconstruction of the unstructured road, it is possible to determine a drivable space on the unstructured road that accurately avoids obstacles. The turning path can be determined independently of structured road information, allowing the vehicle to accurately enter the target intersection.

[0175] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0176] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0177] The present application embodiment is described with reference to the flow chart and / or block diagram of the method, terminal device (system), and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flow chart and / or block diagram and the combination of the process and / or box in the flow chart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a device for realizing the function specified in one process or multiple processes and / or one box or multiple boxes of the flow chart.

[0178] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0179] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0180] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0181] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0182] The above is a detailed introduction to the vehicle turning control method and device based on binocular stereo vision provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A vehicle turning control method based on binocular stereo vision, wherein a binocular stereo vision sensor is installed on a vehicle, characterized in that: include: When the vehicle turns on an unstructured road; acquiring initial image data collected by the binocular stereo vision sensor; Performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data; Filtering the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space; A turning path is determined according to the drivable space.

2. The method according to claim 1, characterized in that The step of filtering the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space comprises: Using the flat point cloud in the three-dimensional point cloud data as a reference plane; Based on the reference plane and in combination with the preset height threshold, the three-dimensional point cloud data is filtered to obtain a drivable space.

3. The method according to claim 2, characterized in that The preset height threshold includes a positive height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and in combination with the preset height threshold to obtain a drivable space includes: Based on the reference plane, filtering point clouds in the three-dimensional point cloud data that are lower than the positive height threshold as first target point cloud data; The space formed by the first target point cloud data is determined as the drivable space.

4. The method according to claim 2, characterized in that: The preset height threshold includes a negative height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and the preset height threshold to obtain a drivable space includes: Based on the reference plane, filtering the point clouds in the three-dimensional point cloud data that are higher than the negative height threshold as second target point cloud data; The space formed by the second target point cloud data is determined as the drivable space.

5. The method according to claim 1, characterized in that The step of determining a turning path according to the drivable space comprises: Get vehicle size information; The turning path is planned and generated in the drivable space according to the vehicle body size information.

6. The method according to claim 1, characterized in that Before the step of determining a turning path according to the drivable space, the method further includes: When the vehicle travels on a structured road, acquiring three-primary-color point cloud information of the road; Determining road structure information based on the road three-primary color point cloud information; The road structure information is used to filter the drivable space.

7. The method according to claim 1, characterized in that The method further comprises: The vehicle is controlled to turn along the turning path.

8. A vehicle turning control device based on binocular stereo vision, wherein a binocular stereo vision sensor is installed on the vehicle, characterized in that: The device comprises: A first acquisition module is used to acquire initial image data collected by the binocular stereo vision sensor when the vehicle travels on an unstructured road; A three-dimensional stereo matching module, used for performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data; A filtering module, used for filtering the three-dimensional point cloud data based on a preset height threshold to obtain a drivable space; A planning module is used to determine a turning path according to the drivable space.

9. A vehicle, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the vehicle turning control method based on binocular stereo vision as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle turning control method based on binocular stereo vision as described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Vehicle-mounted road unevenness recognition system and method based on binocular camera

    CN108151681A

  • Route planning processing method and device

    CN111874006A

  • Driving area detection method and device, electronic equipment and storage medium

    CN112036274A

  • Bend scene sensing method, system and device based on binocular stereo camera

    CN112465831A

  • Space drivable area map construction method and device based on binocular camera

    CN115164919A

Cited By

  • Graph-free crossroad smooth control method and system based on 4D millimeter wave radar

    CN121929155A