Vehicle body height dynamic adjustment method based on binocular vision

Abstract

The invention discloses a binocular vision-based vehicle body height dynamic adjustment method, which is used for solving the problem that the current vehicle body height adjustment only depends on the working condition of a road below a vehicle body and does not consider a height limiting device above the road at the same time, improving the vehicle safety and passing rate and reducing the driving time. The method for dynamically adjusting the height of the vehicle body based on binocular vision comprises the following steps of: acquiring a binocular image, and outputting a corrected left image and a right image which are clearly imaged by utilizing internal parameters and external parameters of a camera which are obtained through calibration; performing visual perception on the left image and the right image, establishing three branch tasks based on a perception result, and realizing upper, middle and lower three-dimensional complete target information detection of a road scene in front of the vehicle through combination of the three branch tasks to obtain three-dimensional point cloud space target information; and fusing the sensing result with the three-dimensional point cloud space target information to obtain a sensing fusion result, and adaptively adjusting the height information of the vehicle body according to the sensing fusion result.

Description

Technical Field

[0001] The present invention relates to the technical field of assisted driving, and in particular to a vehicle body height dynamic adjustment method based on binocular vision. Background Art

[0002] With the rapid development of automobile technology, air suspension has been gradually applied to passenger cars and commercial vehicles. Among them, active air suspension represented by electronically controlled air suspension can increase or decrease the height of air springs by inflating and deflating according to vehicle speed and different road conditions, so as to dynamically adjust the vehicle height and improve the passability and safety of the vehicle during driving. In order to protect the road and the personal safety of the driver, some roads, bridges, tunnels, parking lots, basements, etc. will be equipped with height limit devices to limit the height of vehicle passage. However, there is no unified standard for setting height limit devices. There are a large number of different heights, shapes, and styles. Some height limit devices are redundant or unreasonable. Some height limit devices do not have unlimited height signs to indicate their height, and there is a situation where the actual height of the height limit device from the ground is inconsistent with the nominal value of the height limit sign. Due to the driver's inattention, fatigue driving, or underestimation of the height of the vehicle, the moving vehicle and the height limit device collide, resulting in casualties and damage to the height limit device, causing economic losses.

[0003] The existing disclosed technology mainly considers the vehicle speed, load, and the road conditions below the vehicle to adjust the vehicle height, and does not consider the situation where there is a height limit device above the vehicle road. The current vehicle detects the height of the height limit device during driving by carrying different sensors, such as laser radar, millimeter wave radar, monocular camera, and binocular camera. The patent publication number is CN112498044B. This patent identifies road obstacles through a vehicle-mounted camera, and obtains the distance and height of road obstacles through a distance sensor or an ultrasonic sensor to dynamically adjust the vehicle height. However, this method does not consider the information of the height limit device above the vehicle, and requires the installation of a camera and a distance sensor at the same time, and the sensor is redundant; the patent publication number is CN115366600A. This patent detects the height of the height limit device in front through two radars to determine whether the vehicle height and the minimum height of the vehicle body can be adjusted by the height limit device. However, this method adjusts the vehicle height according to a fixed value, and uses a camera + two radars for detection, which increases the sensor redundancy cost. The binocular camera can not only take advantage of the monocular recognition type of height limit device to identify road conditions, but also measure distance and height through disparity maps, with high efficiency and low cost.

[0004] In view of this, the present invention is proposed. Summary of the invention

[0005] The main purpose of the present invention is to disclose a method for dynamically adjusting the vehicle height based on binocular vision, which is used to solve the problem that the current vehicle height adjustment only depends on the road conditions below the vehicle body without considering the height limit device above the road, thereby improving vehicle safety and pass rate and reducing driving time.

[0006] To achieve the above object, according to one aspect of the present invention, a vehicle height dynamic adjustment method based on binocular vision is provided, and the following technical solutions are adopted:

[0007] Acquire binocular images, and use the calibrated internal and external parameters of the camera to output corrected, clear left and right images; perform visual perception on the left and right images, and establish three branch tasks based on the perception results. Through the combination of the three branch tasks, complete target information detection in the upper, middle and lower dimensions of the road scene in front of the vehicle is achieved, and the three-dimensional point cloud space target information is obtained; the perception result is fused with the three-dimensional point cloud space target information to obtain the perception fusion result, and the vehicle height information is adaptively adjusted according to the perception fusion result.

[0008] Furthermore, the left image and the right image are visually perceived, and three branch tasks are established based on the perception results. Through the combination of the three branch tasks, complete target information detection in the three dimensions of upper, middle and lower of the road scene in front of the vehicle is realized, and the three-dimensional point cloud space target information is obtained, including: first, the left image and the right image are input into the multi-task deep learning model, the feature map is first extracted through the backbone network, and then three branch tasks are established according to the task requirements, branch 1 is responsible for the segmentation semantic results of the road surface target; branch 2 is responsible for the semantic segmentation results of the target above the road, and the lower edge area of ​​the target above the road is determined through strategy screening; branch 3 is responsible for detecting and identifying the position and nominal value of the height limit sign erected above the height limit device or on both sides of the road.

[0009] Furthermore, the perception result is fused with the 3D point cloud spatial target information to obtain a perception fusion result. According to the perception fusion result, the vehicle height information is adaptively adjusted, including: the output corrected left image and the corrected right image are used as two inputs of the binocular point cloud generation module, a disparity map is generated according to the stereo matching algorithm, and rich point cloud data information of the scene in front of the vehicle is obtained through calculation according to the calculation formula between the disparity map and the spatial point cloud. The calculation process and formula of converting the disparity map to the point cloud are as follows: Assuming that any pixel point image coordinates (u, v), disparity represents disparity, which corresponds to the coordinates (Xc, Yc, Zc) in the 3D point cloud coordinate system, in meters; cx and cy are the coordinates of the optical center of the image, in pixels; fx and fy are the focal lengths of the camera, in pixels; the horizontal baseline distance between the left eye and the right eye is baseline, in meters; Zc is calculated according to formula (1)

[0010] (1)

[0011] Calculate Xc, Yc according to formula (2) (3)

[0012] (2)

[0013] (3).

[0014] Furthermore, the visual perception result is fused with the three-dimensional point cloud space target information to obtain a perception fusion result, and the vehicle body height information is adaptively adjusted according to the perception fusion result, which also includes: constructing a two-dimensional grid and a three-dimensional grid based on the point cloud data information, detecting whether there is a target in each two-dimensional and three-dimensional grid, clustering the grids with targets, judging the overall class objects after clustering, and retaining the class objects that meet the preset conditions as the targets of interest in the road scene ahead of the vehicle.

[0015] Furthermore, the visual perception result is fused with the 3D point cloud space target information to obtain a perception fusion result, and the vehicle height information is adaptively adjusted according to the perception fusion result, which also includes: according to the position of the 3D space target, the targets based on binocular point cloud detection are divided into three categories, namely, road targets, road erected targets, and aerial targets; the road targets and road target semantic segmentation results are roughly matched, and a cross-check is performed based on the rough matching results. When a certain target exists in both 2D space and 3D space, it indicates that the target has a high credibility, and the confidence is marked as high. High-confidence targets are preferentially added to the road target output sequence; aerial targets and road overhead targets are segmented. The results are roughly matched, and a cross-check is performed based on the rough matching results. When a target exists in both 2D space and 3D space, it means that the target has a high credibility and is marked as high. High-confidence targets are preferentially added to the output sequence of targets above the road; the detection results of road erected targets and height limit sign targets are roughly matched, and a cross-check is performed based on the rough matching results. When a target exists in both 2D space and 3D space, it means that the target has a high credibility and is marked as high. High-confidence targets are preferentially added to the output sequence of height limit sign targets; the road target output sequence, the road target output sequence, and the height limit sign target output sequence are integrated to obtain the perception fusion result.

[0016] Furthermore, the visual perception result is fused with the three-dimensional point cloud spatial target information to obtain a perception fusion result, and the vehicle body height information is adaptively adjusted according to the perception fusion result, which also includes: the binocular camera detects in real time whether there are height limit signs on both sides or above the road; if so, the height limit sign nominal number h_sign is identified, and then compared with the current vehicle body top height; if h_sign is smaller than the current vehicle body top, a voice prompt is issued "There is a height limit ahead, please pay attention and slow down", and the height limit signs on both sides of the road are detected in advance to remind people to slow down.

[0017] Furthermore, the visual perception result is fused with the three-dimensional point cloud space target information to obtain a perception fusion result, and adaptively adjusting the vehicle height information according to the perception fusion result also includes: the binocular camera detects the front scene information in real time, outputs the target sequence of the height limit device above the road and the target sequence of the road surface, including the three-dimensional x, y, z coordinate information of each target, the length, width and height information of the geometric dimensions of the height limit device, the longitudinal distance information along the driving direction, and the category information; comprehensively judges whether the vehicle height adjustment is to be performed according to the height limit device information above the road and the road surface target information; and according to the final adjustment result of the vehicle height, the vehicle height adjustment scenarios are mainly divided into three categories, namely, the vehicle height cannot be adjusted, the vehicle height is reduced, and the vehicle height is increased.

[0018] Furthermore, the scenario in which the vehicle height cannot be adjusted includes: finding the target limit_min with the minimum ground height from the target output sequence above the road, comparing the limit_min ground height h_limit and the current vehicle top height h_top_current; if h_limit is less than the current vehicle top height h_top_current, setting the vehicle body lowering mark to 1; finding the target road_max with the maximum ground height from the road surface target output sequence, comparing the road_max ground height h_road and the current vehicle body bottom height h_bottom_current; if h_road is greater than the current vehicle body bottom height h_bottom_current, setting the vehicle body raising mark to 1; judging whether the vehicle body lowering mark and the vehicle body raising mark are 1 at the same time, if both are 1, it indicates that the vehicle body height cannot be adjusted and the road ahead cannot be passed safely.

[0019] Furthermore, find the target limit_min with the minimum ground clearance from the target output sequence above the road, compare the limit_min ground clearance h_limit and the current vehicle top height h_top_current; if h_limit is less than the current vehicle top height h_top_current, set the vehicle body lowering flag to 1; find the target road_max with the maximum ground clearance from the road target output sequence, compare the road_max ground clearance h_road and the current vehicle bottom height h_bottom_current, if h_road is less than the current vehicle bottom height h_bottom_current, nt, then set the body raising mark to 0; determine that the body lowering mark and the body raising mark are not 1 at the same time, and the body lowering mark is 1, indicating that the body height lowering condition 1 is satisfied; determine whether h_limit is less than the minimum adjustable height of the body top h_top_min, if it is greater, it indicates that the body height lowering condition 2 is satisfied; determine whether the current vehicle speed is less than the threshold value V1, if it is less than, it indicates that the vehicle speed height lowering condition 3 is satisfied; calculate the body height lowering adjustment range, set the safety height lowering threshold value to Δh_down, and the upper limit Δh1_up and lower limit Δh1_down of the body height lowering adjustment range Δh1 are calculated according to formulas (4) and (5), and the upper limit is greater than the lower limit.

[0020] (4)

[0021] (5).

[0022] Furthermore, the vehicle height increase scenario includes: finding the minimum ground clearance target limit_min from the road target output sequence, comparing the limit_min ground clearance h_limit and the current vehicle top height h_top_current, and if h_limit is greater than the current vehicle top height h_top_current, setting the vehicle body lowering flag to 0; finding the maximum ground clearance target road_max from the road target output sequence, comparing the road_max ground clearance h_road and the current vehicle bottom height h_bottom_current, and if h_road is greater than the current vehicle bottom height h_bottom_current, setting the vehicle body lowering flag to 0; If the vehicle height is lowered by 1, the vehicle height raising flag is set to 1. If the vehicle height lowering flag and the vehicle height raising flag are not both 1 and the vehicle height raising flag is 1, it means that the vehicle height raising condition 1 is satisfied. If h_road is less than the maximum adjustable height of the vehicle bottom h_bottom_max, if so, it means that the vehicle height lowering condition 2 is satisfied. If the vehicle speed is less than the threshold v1, if so, it means that the vehicle height lowering condition 3 is satisfied. The vehicle height raising adjustment range is calculated. The safety height raising threshold is Δh_up. The upper and lower limits of the vehicle height raising Δh2 adjustment range are calculated according to formulas (6) and (7), and the upper limit is greater than the lower limit:

[0023] (6)

[0024] (7).

[0025] The present invention proposes a vehicle body height dynamic adjustment method based on binocular vision, which utilizes the rich and diverse road scene semantic recognition capabilities and a large amount of point cloud data of the binocular camera, and simultaneously outputs standard height limit targets, special-shaped height limit targets, road surface standard targets, and road surface special-shaped targets. According to the dual constraints of the road surface target in front of the vehicle and the height limit target above the road, the vehicle body height is dynamically and adaptively adjusted, which solves or improves the defects and problems of the existing technology that most of the vehicle body height adjustment only recognizes the road conditions, reduces the risk of the vehicle chassis or roof being scratched, and at the same time improves the constraints of the height limit device scene, increases the vehicle passability, shortens the vehicle driving time to the destination, and improves the driving experience of tourism and cargo transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0027] Figure 1This is a flow chart of a vehicle body height dynamic adjustment method based on binocular vision according to an embodiment of the present invention;

[0028] Figure 2 This is an overall flow chart of a vehicle body height dynamic adjustment method based on binocular vision according to an embodiment of the present invention;

[0029] Figure 3 A flowchart of a vehicle height dynamic adjustment method based on binocular vision according to an embodiment of the present invention based on a multi-task deep learning model perception result;

[0030] Figure 4 A schematic diagram of left correction and corresponding disparity map according to an embodiment of the present invention;

[0031] Figure 5 The visual perception result fusion module described in the embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of a height limit sign according to an embodiment of the present invention erected in front of a height limit device;

[0033] Figure 7 It is a schematic diagram of a height limit sign according to an embodiment of the present invention being located on a height limit device;

[0034] Figure 8 A flow chart of a vehicle height adjustment judgment module of a vehicle height dynamic adjustment method based on binocular vision according to an embodiment of the present invention;

[0035] Fig. 9 A flow chart of a vehicle height drop judgment module based on a vehicle height dynamic adjustment method based on binocular vision according to an embodiment of the present invention;

[0036] Fig.10 This is a flow chart of a vehicle height rise judgment module of a vehicle height dynamic adjustment method based on binocular vision described in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0038] Figure 1 The present invention is a flowchart of a binocular camera-based intelligent chassis control method according to an embodiment of the present invention.

[0039] See also Figure 1 As shown, a vehicle height dynamic adjustment method based on binocular vision includes:

[0040] S101: Acquire binocular images, and use camera internal parameters and external parameters obtained through calibration to output corrected left and right images with clear imaging;

[0041] S102: Perform visual perception on the left and right images, establish three molecular tasks based on the perception results, and achieve complete target information detection in three dimensions of the road scene in front of the vehicle, such as top, middle and bottom, through the combination of three branch tasks, to obtain three-dimensional point and spatial target information;

[0042] S103: Fusing the perception result with the three-dimensional point cloud spatial target information to obtain a perception fusion result, and adaptively adjusting the vehicle body height information according to the perception fusion result.

[0043] Specifically, the whole process starts with step S101, where a precisely calibrated binocular stereo camera is horizontally and accurately installed in the center of the vehicle's front windshield. Once powered on, the camera captures the scene in front in real time. Using the camera's internal and external parameters obtained by the previous calibration, the camera can output corrected left and right images with clear images. Next, in step S102, the visual perception module processes the image using a multi-task deep learning model. This model contains three branches, focusing on the detection of road targets, targets above the road, and height limit signs. Through a shared backbone network and independent detection and segmentation models, the model can achieve comprehensive detection of all target information in the road scene ahead. Furthermore, using the calculation formula between the disparity map and the spatial point cloud, we can calculate the rich point cloud data information of the scene in front of the vehicle. Based on these point cloud data, we construct two-dimensional grids and three-dimensional grids, and detect whether there is a target in each grid. For the grids with targets, we perform clustering and judge the overall class objects after clustering, and only retain the class objects that meet the preset conditions as the targets of the road scene in front of the vehicle. In step S103, we divide the targets based on binocular point cloud detection into three categories according to the position of the targets in the three-dimensional space: road targets, road erected targets and aerial targets. Next, we perform a series of matching and verification operations on these targets. Specifically: rough matching and cross-verification are performed on the road targets and their semantic segmentation results. If a target exists in both two-dimensional space and three-dimensional space, its credibility is high, and we mark its confidence as high and give priority to adding it to the road target output sequence. Similar rough matching and cross-verification are also performed on the segmentation results of aerial targets and targets above the road. Similarly, if a target exists in both spaces, its confidence will also be marked as high and given priority to adding it to the road target output sequence. For the detection results of road erected targets and height limit sign targets, we also perform rough matching and cross-verification. If a target exists in both spaces, its confidence will also be marked as high and will be added to the height-limited target output sequence with priority.

[0044] Finally, we integrate the road target output sequence, the road overhead target output sequence, and the height limit sign target output sequence to obtain the final perception fusion result. Based on this fusion result, we can adaptively adjust the height information of the vehicle body to better cope with different roads and driving environments.

[0045] The above technical solution of this embodiment is used to solve the problem that the current vehicle height adjustment only depends on the road conditions below the vehicle body without considering the height limit device above the road, thereby improving vehicle safety and pass rate and reducing driving time.

[0046] Figure 2 The present invention is a flowchart of a method for dynamically adjusting vehicle height based on binocular vision according to an embodiment of the present invention.

[0047] join Figure 2 As shown, a vehicle height dynamic adjustment method based on binocular vision includes the following steps:

[0048] Step 1: Get binocular images. Install the calibrated binocular stereo camera horizontally and correctly in the center of the vehicle's front windshield. After powering on, the camera captures the scene in front in real time. Using the calibrated internal and external parameters of the camera, output the corrected and clear left and right images. The left and right images can be grayscale or color images.

[0049] Step 2: The visual perception module captures the target information of the road scene ahead and performs the first positioning and measurement of the target information. According to different task requirements, various combinations of different task branches can be performed. In combination with the task requirement of dynamic adjustment of vehicle height, the specific process of the visual perception module proposed in the present invention is: first, the color image of the left camera or the right camera is input into the multi-task deep learning model, the feature map is first extracted through the backbone network, and then three branch tasks are established according to the task requirements. Branch 1 is responsible for the segmentation semantic results of road targets, including road areas and speed bumps, manhole covers, road spills, etc.; Branch 2 is responsible for the semantic segmentation results of targets above the road, including height limit devices, tunnels, bridges, etc., and determines the lower edge area of ​​targets above the road through strategy screening; Branch 3 is responsible for detecting and identifying the position of the height limit sign erected above the height limit device or on both sides of the road and the nominal value of the height limit sign. The subjective network shared by the three branches and the detection and segmentation model structures used by each branch are common industry frameworks and will not be specifically expanded here. Through the combination of three perception subtasks, complete target information detection in the upper, middle and lower dimensions of the road scene ahead of the vehicle is achieved. See the specific process. Figure 2 .

[0050] Step 3: First, the corrected left image and the corrected right image are output as two inputs of the binocular point cloud generation module. A disparity map is generated according to the stereo matching algorithm. According to the calculation formula between the disparity map and the spatial point cloud, rich point cloud data information of the scene in front of the vehicle is obtained through calculation. Figure 4 The left image and disparity map of the real scene during vehicle driving are shown. The calculation process and formula for converting the disparity map to the point cloud are as follows. Assume that the image coordinates of any pixel point (u, v), disparity represents the disparity, which corresponds to the coordinates (Xc, Yc, Zc) in the three-dimensional point cloud coordinate system, in meters; cx and cy are the coordinates of the optical center of the image, in pixels; fx and fy are the focal lengths of the camera, in pixels; the horizontal baseline distance between the left and right eye centers is baseline, in meters.

[0051] Step 1: Calculate Zc according to formula (1)

[0052] (1)

[0053] Calculate Xc, Yc according to formula (2) (3)

[0054] (2)

[0055] (3)

[0056] Step 4, in step 2, uses a deep learning method to detect target information in the 2D image space through a training model. The 2D space target lacks 3D information in the spatial coordinate system, such as geometric size information such as target length, width, height, etc. Therefore, based on the 2D space detection, the present invention proposes a detection module for 3D point cloud space target information to make full use of the multiple information sources of the binocular camera. The specific process of the point cloud detection module is as follows: based on the point cloud data generated in step 3, a 2D grid and a 3D grid are constructed, and whether there is a target in each 2D and 3D grid is detected, and the grids with targets are clustered, and the overall class objects after clustering are judged, and the class objects that meet the preset conditions are retained as the target of interest in the road scene ahead of the vehicle. Since the binocular point cloud information is rich and diverse, considering performance and efficiency, the spatial range can be actually set according to the task requirements, and the predetermined target information can be detected within the spatial range.

[0057] Step 5: For step 2 and step 4, the target information is located and measured in 2D space and 3D space respectively, which inevitably leads to problems such as missed detection, false detection, partial detection, etc. The visual fusion module is used to fuse the perception results based on the output of the deep learning model with the perception results based on the binocular point cloud detection. The specific process is as follows: according to the position of the target in 3D space, the targets detected based on binocular point cloud are divided into three categories: road targets, road erected targets, and aerial targets; the road targets and road target semantic segmentation results are roughly matched, and cross-checked based on the rough matching results. When a certain target exists in both 2D space and 3D space, it means that the target has high credibility, and the confidence is marked as high. High-confidence targets are preferentially added to the road target output sequence; aerial targets and road overhead target segmentation results are roughly matched, and cross-checked based on the rough matching results. When a certain target exists in both 2D space and 3D space, it means that the target has high credibility, and the confidence is marked as high. High-confidence targets are preferentially added to the road overhead target output sequence; road erected targets and height limit sign target detection results are roughly matched, and cross-checked based on the rough matching results. When a certain target exists in both 2D space and 3D space, it means that the target has high credibility, and the confidence is marked as high. High-confidence targets are preferentially added to the height limit sign target output sequence. Integrate the road target output sequence, the road target output sequence, and the height limit sign target output sequence to obtain the perception fusion result. By fusing the perception results, the effectiveness and reliability of the output target can be improved, providing more robust target detection results for the subsequent decision module and control module. See the flow chart of the visual fusion module. Figure 5 .

[0058] Step 6: Height limit sign target output sequence judgment result: During vehicle driving, there are two situations of height limit signs: one is erected at a certain distance in front of the height limit device (height limit pole, bridge, tunnel, signboard, etc.), and indicates the height limit device in front to limit the passing height, see Figure 6 Schematic diagram; one is that it appears at the same time as the height limit device, usually located above the height limit device, see Figure 7 Schematic diagram. During the driving process, the binocular camera detects in real time whether there are height limit signs on both sides or above the road. If there are, the height limit sign nominal number h_sign is identified and then compared with the current vehicle top height h_top_current. If h_sign is less than h_top_current, a voice prompt "There is a height limit ahead, please pay attention and slow down" is issued. By detecting the height limit signs on both sides of the road in advance, the driver is reminded to slow down appropriately and prepare for passing the height limit device later.

[0059] Step 7: In view of the defects and shortcomings of the existing invention technology that only unilaterally utilizes road surface target information or height limit device to adjust the vehicle height information, the present invention proposes to integrate road surface target information and road above target information to adaptively adjust the vehicle height information. The specific process is: During vehicle driving, the binocular camera detects the scene information in front in real time, and outputs the target sequence of the height limit device above the road and the road surface target sequence, including the three-dimensional x, y, z coordinate information of each target, the length, width and height information of the geometric dimensions of the height limit device, the longitudinal distance information along the driving direction, category information, etc. Comprehensively judge whether the vehicle height adjustment is to be performed based on the height limit device information above the road and the road surface target information. According to the final adjustment result of the vehicle height, the vehicle height adjustment scenarios are mainly divided into three categories, namely, the vehicle height cannot be adjusted, the vehicle height decreases, and the vehicle height increases. The specific processes of the three types of scene judgment strategies are respectively shown in Figure 8 , Fig. 9 , Fig.10 . Figure 8 The flowchart shows the scenario where the vehicle height cannot be adjusted. The specific process is as follows: find the target limit_min with the minimum ground clearance from the target output sequence above the road, compare the limit_min ground clearance h_limit and the current vehicle top height h_top_current, if h_limit is less than the current vehicle top height h_top_current, then set the vehicle body lowering mark to 1; find the target road_max with the maximum ground clearance from the road target output sequence, compare the road_max ground clearance h_road and the current vehicle bottom height h_bottom_current, if h_road is greater than the current vehicle bottom height h_bottom_current, then set the vehicle body raising mark to 1; determine whether the vehicle body lowering mark and the vehicle body raising mark are 1 at the same time. If both are 1, it means that the vehicle body height cannot be adjusted and the road ahead cannot be passed safely. At this time, the sound and light warning information is triggered to remind the driver, and the vehicle is controlled to decelerate quickly until it stops.

[0060] in Fig. 9The flowchart shows the scene of the vehicle body lowering. The specific process is: find the target limit_min with the minimum ground clearance from the target output sequence above the road, compare the limit_min ground clearance h_limit and the current vehicle body top height h_top_current, if h_limit is less than the current vehicle body top height h_top_current, then set the vehicle body lowering flag to 1; find the target road_max with the maximum ground clearance from the road target output sequence, compare the road_max ground clearance h_road and the current vehicle body bottom height h_bottom_current, if h_road is less than the current vehicle body bottom height h_bottom_curr ent, then set the body raising mark to 0; determine that the body lowering mark and the body raising mark are not 1 at the same time, and the body lowering mark is 1, indicating that the body height lowering condition 1 is satisfied; determine whether h_limit is less than the minimum adjustable height of the body top h_top_min, if it is greater, it indicates that the body height lowering condition 2 is satisfied; determine whether the current vehicle speed is less than the threshold value V1, if it is less than, it indicates that the vehicle speed height lowering condition 3 is satisfied; calculate the body height lowering adjustment range, set the safety height lowering threshold value to Δh_down, and the upper limit Δh1_up and lower limit Δh1_down of the body height lowering adjustment range Δh1 are calculated according to formulas (4) and (5), and the upper limit is greater than the lower limit.

[0061] (4)

[0062] (5).

[0063] further; Fig.10The flowchart shows the scenario of the vehicle height increasing. The specific process is as follows: find the target limit_min with the minimum ground clearance from the target output sequence above the road, compare the limit_min ground clearance h_limit and the current vehicle top height h_top_current, if h_limit is greater than the current vehicle top height h_top_current, set the vehicle body lowering flag to 0; find the target road_max with the maximum ground clearance from the road target output sequence, compare the road_max ground clearance h_road and the current vehicle bottom height h_bottom_current, if h_road is greater than the current vehicle bottom height h_bottom tom_current, then set the body raising mark to 1; determine if the body lowering mark and the body raising mark are not 1 at the same time, and the body raising mark is 1, indicating that the body height raising condition 1 is satisfied; determine whether h_road is less than the maximum adjustable height of the bottom of the body h_bottom_max, if it is less than, it indicates that the body height lowering condition 2 is satisfied; determine whether the current vehicle speed is less than the threshold v1, if it is less than, it indicates that the vehicle speed height lowering condition 3 is satisfied; calculate the body height raising adjustment range, set the safety height raising threshold as Δh_up, and the upper and lower limits of the body height raising Δh2 adjustment range are calculated according to formulas (6) and (7), and the upper limit is greater than the lower limit.

[0064] (6)

[0065] (7).

[0066] The present invention proposes a vehicle body height dynamic adjustment method based on binocular vision, which utilizes the rich and diverse road scene semantic recognition capabilities and a large amount of point cloud data of the binocular camera, and simultaneously outputs standard height limit targets, special-shaped height limit targets, road surface standard targets, and road surface special-shaped targets. According to the dual constraints of the road surface target in front of the vehicle and the height limit target above the road, the vehicle body height is dynamically and adaptively adjusted, which solves or improves the defects and problems of the existing technology that most of the vehicle body height adjustment only recognizes the road conditions, reduces the risk of the vehicle chassis or roof being scratched, and at the same time improves the constraints of the height limit device scene, increases the vehicle passability, shortens the vehicle driving time to the destination, and improves the driving experience of tourism and cargo transportation.

[0067] The above only describes some exemplary embodiments of the present embodiment by way of illustration. It is undoubted that, for those skilled in the art, the described embodiments can be modified in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection claimed for the present invention.

Claims

1. A vehicle height dynamic adjustment method based on binocular vision, characterized in that: include: Acquire binocular images, and use the calibrated camera internal and external parameters to output corrected, clear left and right images; Perform visual perception on the left and right images, and establish three branch tasks based on the perception results. Through the combination of the three branch tasks, complete target information detection in the upper, middle and lower dimensions of the road scene in front of the vehicle is achieved, and the 3D point cloud spatial target information is obtained; The perception result is fused with the 3D point cloud spatial target information to obtain the perception fusion result, and the vehicle height information is adaptively adjusted according to the perception fusion result.

2. The vehicle body height dynamic adjustment method according to claim 1, characterized in that: The left and right images are visually perceived, and three branch tasks are established based on the perception results. Through the combination of the three branch tasks, complete target information detection in the upper, middle and lower dimensions of the road scene in front of the vehicle is achieved, and the three-dimensional point cloud space target information is obtained, including: First, the left and right images are input into the multi-task deep learning model. The feature map is extracted through the backbone network, and then three branch tasks are established according to the task requirements. Branch 1 is responsible for the segmentation semantic results of road targets; Branch 2 is responsible for the semantic segmentation results of targets above the road, and determines the lower edge area of ​​targets above the road through strategy screening; Branch 3 is responsible for detecting and identifying the position and nominal value of the height limit sign above the height limit device or erected on both sides of the road.

3. The vehicle body height dynamic adjustment method according to claim 2, characterized in that: The step of fusing the perception result with the 3D point cloud spatial target information to obtain a perception fusion result, and adaptively adjusting the vehicle height information according to the perception fusion result includes: The corrected left image and the corrected right image are used as two inputs of the binocular point cloud generation module. The disparity map is generated according to the stereo matching algorithm. According to the calculation formula between the disparity map and the spatial point cloud, the rich point cloud data information of the scene in front of the vehicle is obtained through calculation. The calculation process and formula of converting the disparity map to the point cloud are as follows: Assume that the image coordinates of any pixel point are (u, v), disparity represents the disparity, which corresponds to the coordinates (Xc, Yc, Zc) in the three-dimensional point cloud coordinate system, in meters; cx and cy are the coordinates of the optical center of the image, in pixels; fx and fy are the focal lengths of the camera, in pixels; the horizontal baseline distance between the left and right eye centers is in meters; Zc is calculated according to formula (1): (1); Calculate Xc, Yc according to formula (2) (3) （2） (3)。 4. The vehicle body height dynamic adjustment method according to claim 3, characterized in that: The step of fusing the visual perception result with the 3D point cloud spatial target information to obtain a perception fusion result, and adaptively adjusting the vehicle height information according to the perception fusion result further includes: Based on the point cloud data information, two-dimensional grids and three-dimensional grids are constructed. Whether there is a target in each two-dimensional and three-dimensional grid is detected, the grids with targets are clustered, the overall class objects after clustering are judged, and the class objects that meet the preset conditions are retained as the targets of interest in the road scene ahead of the vehicle.

5. The vehicle body height dynamic adjustment method according to claim 4, characterized in that: The step of fusing the visual perception result with the 3D point cloud spatial target information to obtain a perception fusion result, and adaptively adjusting the vehicle height information according to the perception fusion result further includes: According to the location of the 3D space target, the targets detected based on binocular point cloud are divided into three categories: road targets, road vertical targets, and aerial targets; The road targets and the road target semantic segmentation results are roughly matched, and a cross-check is performed based on the rough matching results. When a certain target exists in both 2D space and 3D space, it means that the target has a high credibility, and the confidence is marked as high. The high-confidence target is preferentially added to the road target output sequence; the aerial targets and the road target segmentation results are roughly matched, and a cross-check is performed based on the rough matching results. When a certain target exists in both 2D space and 3D space, it means that the target has a high credibility, and the confidence is marked as high. The high-confidence target is preferentially added to the road target output sequence; the road erected targets and height limit sign target detection results are roughly matched, and a cross-check is performed based on the rough matching results. When a certain target exists in both 2D space and 3D space, it means that the target has a high credibility, and the confidence is marked as high. The high-confidence target is preferentially added to the height limit sign target output sequence; the road target output sequence, the road target output sequence, and the height limit sign target output sequence are integrated to obtain the perception fusion result.

6. The vehicle body height dynamic adjustment method according to claim 5, characterized in that: The step of fusing the visual perception result with the 3D point cloud spatial target information to obtain a perception fusion result, and adaptively adjusting the vehicle height information according to the perception fusion result further includes: The binocular camera detects in real time whether there are height limit signs on both sides or above the road; If it exists, identify the height limit sign nominal number h_sign and then compare it with the current vehicle top height; If h_sign is smaller than the current top of the vehicle, a voice prompt "There is a height limit ahead, please pay attention and slow down" will be issued, and the vehicle will be reminded to slow down by detecting the height limit signs on both sides of the road in advance.

7. The vehicle body height dynamic adjustment method according to claim 6, characterized in that: The step of fusing the visual perception result with the 3D point cloud spatial target information to obtain a perception fusion result, and adaptively adjusting the vehicle height information according to the perception fusion result further includes: The binocular camera detects the scene information ahead in real time and outputs the target sequence of the height limit device above the road and the target sequence on the road surface, including the three-dimensional x, y, z coordinate information of each target, the length, width and height information of the geometric dimensions of the height limit device, the longitudinal distance information along the driving direction, and the category information; Comprehensively determine whether to adjust the vehicle height based on the information of the height limit device above the road and the road target information; According to the final adjustment result of the vehicle height, the vehicle height adjustment scenarios are mainly divided into three categories, namely, the vehicle height cannot be adjusted, the vehicle height decreases, and the vehicle height increases.

8. The vehicle body height dynamic adjustment method according to claim 7, characterized in that: The vehicle height cannot be adjusted in the following scenarios: Find the target limit_min with the minimum ground clearance from the target output sequence above the road, and compare the limit_min ground clearance h_limit with the current vehicle top height h_top_current; If h_limit is less than the current vehicle top height h_top_current, set the vehicle body lowering flag to 1; Find the target with the largest ground clearance h_road from the road target output sequence, and compare the ground clearance h_road of road_max with the current bottom height h_bottom_current of the vehicle body; If h_road is greater than the current vehicle bottom height h_bottom_current, set the vehicle body elevation flag to 1; Determine whether the vehicle body lowering mark and the vehicle body raising mark are 1 at the same time. If both are 1, it means that the vehicle body height cannot be adjusted and the road ahead cannot be passed safely.

9. The vehicle body height dynamic adjustment method according to claim 8, characterized in that: The vehicle height reduction includes: Find the target limit_min with the minimum ground clearance from the target output sequence above the road, and compare the limit_min ground clearance h_limit with the current vehicle top height h_top_current; If h_limit is less than the current vehicle top height h_top_current, set the vehicle body lowering flag to 1; Find the target road_max with the maximum ground clearance from the road target output sequence, compare the ground clearance h_road of road_max with the current vehicle bottom height h_bottom_current, if h_road is less than the current vehicle bottom height h_bottom_current, set the vehicle body elevation flag to 0; It is determined that the body lowering mark and the body raising mark are not 1 at the same time, and the body lowering mark is 1, indicating that the body height lowering condition 1 is satisfied; it is determined whether h_limit is less than the minimum adjustable height h_top_min of the body top height, if it is greater, it indicates that the body height lowering condition 2 is satisfied; Determine whether the current vehicle speed is less than the threshold value V1. If so, it means that the vehicle speed height reduction condition 3 is met; Calculate the vehicle height drop adjustment range. Assume that the safety height drop threshold is Δh_down. The upper limit Δh1_up and lower limit Δh1_down of the vehicle height drop Δh1 adjustment range are calculated according to formulas (4) and (5), and the upper limit is greater than the lower limit: （4） （5） 。 10. The vehicle body height dynamic adjustment method according to claim 8, characterized in that: The vehicle height increase scenario includes: Find the target limit_min with the minimum ground clearance from the target output sequence above the road, compare the limit_min ground clearance h_limit and the current vehicle top height h_top_current, if h_limit is greater than the current vehicle top height h_top_current, set the vehicle body drop flag to 0; Find the target road_max with the maximum ground clearance from the road target output sequence, compare the ground clearance h_road of road_max with the current vehicle bottom height h_bottom_current, if h_road is greater than the current vehicle bottom height h_bottom_current, set the vehicle body elevation flag to 1; It is determined that the vehicle body lowering mark and the vehicle body raising mark are not 1 at the same time, and the vehicle body raising mark is 1, indicating that vehicle body height raising condition 1 is satisfied; Determine whether h_road is less than the maximum adjustable height of the bottom of the vehicle body h_bottom_max. If so, it means that the vehicle body height reduction condition 2 is met. Determine whether the current vehicle speed is less than the threshold value v1. If so, it means that the vehicle speed height reduction condition 3 is met. Calculate the vehicle height increase adjustment range. Assume that the safety height increase threshold is Δh_up. The upper and lower limits of the vehicle height increase Δh2 adjustment range are calculated according to formulas (6) and (7), and the upper limit is greater than the lower limit: （6） （7）。

Images