A vehicle control method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of automatic driving, and provides a vehicle control method and device, an electronic device and a storage medium. The method comprises the following steps: judging a current driving environment, and determining a driving mode based on the driving environment; processing panoramic view and point cloud data by using a trained road perception model to obtain a road graph containing concave-convex objects; calculating a height mean square error of the concave-convex objects in each preset area, and if the height mean square error is less than a first threshold value, obtaining a driving path sequence based on the height mean square error; determining a target driving path from the driving path sequence based on a vehicle control factor and the height mean square error, and controlling the vehicle to drive based on the target driving path; and if the height mean square error exceeds the first threshold value and the concave-convex objects exceed a transverse vehicle control range, controlling the vehicle to decelerate based on a longitudinal vehicle control speed interval corresponding to the driving mode. The application solves the problem that the vehicle driving is not stable when the existing technology faces a complex working condition road surface.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and in particular to a vehicle control method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the development of vehicle technology, automatic driving has become a hot topic and has an impact on the development prospect of vehicle technology. As a disruptive technology, automatic driving is attracting extensive attention and is regarded as one of the important development directions of the vehicle industry. Current automatic driving technology mainly includes three parts: environment perception, decision planning and control execution.

[0003] Current environment perception is mostly to detect pedestrians, obstacles, vehicles and road sign information through time and space synchronization of cameras, and then to recognize lane markings and traffic participant position trajectories through sensor fusion by spatial mapping method. However, the vehicle driving road scene is very complex, especially when the vehicle passes through a complex road or a non-paved road, the bounce of the vehicle will seriously affect the passenger comfort, and when the vehicle is driving at high speed, the driver is difficult to judge the road condition in a short time and make a timely response strategy, resulting in vehicle driving bumping and even possible rollover.

[0004] Therefore, the prior art has the problem of unstable vehicle driving when facing complex road conditions. SUMMARY

[0005] Therefore, the present application provides a vehicle control method and device, an electronic device and a storage medium to solve the problem of unstable vehicle driving when facing complex road conditions in the prior art.

[0006] In a first aspect of the present application, a vehicle control method is provided, comprising: in a driver assistance system, collecting a panoramic view, point cloud data and driving information; determining a current driving environment based on the panoramic view and the driving information, and determining a driving mode based on the driving environment, the driving mode representing a longitudinal control speed interval and a vehicle head turning angle interval in the corresponding driving environment; processing the panoramic view and the point cloud data using a trained road perception model to obtain a road graph containing concave-convex objects; in the road graph, calculating a height variance of the concave-convex objects in each preset area, and if the height variance is less than a first threshold, obtaining a driving path sequence based on the height variance; determining a target driving path from the driving path sequence based on control factors and the height variance, and controlling the vehicle to drive based on the target driving path, the control factors including at least vehicle speed, lateral acceleration, longitudinal acceleration and vehicle head turning angle interval; if the height variance exceeds the first threshold and the concave-convex objects exceed the lateral control range, controlling the vehicle to decelerate based on the longitudinal control speed interval corresponding to the driving mode.

[0007] In a second aspect, the present application provides a vehicle control device, comprising: a collection module configured to collect a panoramic view, point cloud data and driving information in a driver assistance system; a judgment module configured to judge a current driving environment based on the panoramic view and the driving information, and determine a driving mode based on the driving environment, the driving mode representing a longitudinal vehicle control speed range and a vehicle steering angle range in a corresponding driving environment; a processing module configured to process the panoramic view and the point cloud data using a trained road perception model to obtain a road graph containing concave-convex objects; a calculation module configured to calculate a height mean square error of the concave-convex objects in each preset area in the road graph, and obtain a driving path sequence based on the height mean square error if the height mean square error is less than a first threshold; a first vehicle control module configured to determine a target driving path from the driving path sequence based on vehicle control factors and the height mean square error, and control the vehicle to drive based on the target driving path, the vehicle control factors including at least vehicle speed, lateral acceleration, longitudinal acceleration and the vehicle steering angle range; and a second vehicle control module configured to control the vehicle to decelerate based on the longitudinal vehicle control speed range corresponding to the driving mode if the height mean square error exceeds the first threshold and the concave-convex objects exceed a lateral vehicle control range.

[0008] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0009] In a fourth aspect, the present application provides a storage medium, which stores a computer program executable by a processor to implement the steps of the above method.

[0010] The above at least one technical scheme adopted by the present application can achieve the following beneficial effects:

[0011] By collecting real-time panoramic view, point cloud data and driving information, the current driving environment is determined according to the driving information and the panoramic view, and then a predetermined driving mode corresponding to the current driving environment is selected based on the current driving environment. The driving mode can represent the longitudinal control speed interval and the steering angle interval of the vehicle in the current driving environment. Then the trained road perception model is used to process the panoramic view and the point cloud data to obtain a road graph containing the concave and convex objects in the current road, that is, the three-dimensional concave and convex parts are rendered in the road graph. Then the height variance of the concave and convex objects in each preset area is calculated in the road graph. If the height variance is less than a first threshold, the height variances of all areas are analyzed to obtain a plurality of selectable driving path sequences. Then the control factors and the height variances of the corresponding concave and convex objects in each driving path are analyzed to select the most suitable target driving path from the driving path sequences. Based on the target driving path, the vehicle is controlled. Of course, if the height variance of the concave and convex objects in the area at this time exceeds the first threshold, and the area of the concave and convex objects exceeding the first threshold exceeds the lateral control range of the current vehicle, the vehicle is uniformly decelerated based on the predetermined longitudinal control interval, so that the vehicle is parked or gently swept over the concave and convex objects. In this way, the external environmental factors can be obtained by the vehicle's multiple sensors, the current road information can be obtained in real time, and the road graph can be obtained by using the trained road perception model. Then the analysis of the road graph can obtain the driving path sequence to facilitate the vehicle to automatically analyze and select the target driving path that can be gently driven, or to control the vehicle to gently decelerate and sweep over the concave and convex objects. In this way, the vehicle can be gently controlled when facing complex roads, and various environmental factors are also considered in the vehicle control, so as to comprehensively consider the current environment and road conditions, and avoid the problem of excessive shaking of the vehicle affecting the safety and comfort of the vehicle body and passengers. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creating any creative labor.

[0013] Figure 1 is a flowchart of a vehicle control method provided by an embodiment of the present application;

[0014] Figure 2 is an example diagram of a vehicle control method provided by an embodiment of the present application;

[0015] Figure 3 is a flowchart of another vehicle control method provided by an embodiment of the present application;

[0016] Figure 4 Fig. 1 is a structural schematic diagram of a vehicle control device according to an embodiment of the present application.

[0017] Figure 5 Fig. 1 is a structural schematic diagram of a vehicle control device according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as a particular sequence of steps, in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, and circuits are omitted so as not to obscure the description of the present application with unnecessary detail.

[0019] The terms "first", "second", and the like in the description and in the claims of the present application are used for distinguishing between similar objects and not necessarily for describing a specific sequential or chronological order. It is to be understood that the data so designated are interchangeable under appropriate circumstances such that the embodiments of the present application encompasses embodiments where the data performs proprietary functions in other orders than illustrated or otherwise described herein. Furthermore, reference to first, second, etc. objects means at least one of the objects, and thus the use of such terms is not to be interpreted in an "every or each" basis but rather in an "at least one" basis. Also, the sequence of any process is illustrative and should not be deemed to restrict the order of any steps to the exact sequence presented unless such exact sequence is clearly substantiated by the specification.

[0020] Also, it is to be understood that the terms "comprising", "including", or "having" when used herein, specify the presence of stated features, integers, steps, processes, actions, elements, or components but do not preclude the presence or addition of one or more other features, integers, steps, processes, actions, elements, components, or groups thereof.

[0021] A vehicle control method and device according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0022] Figure 1 Fig. 2 is a flowchart of a vehicle control method according to an embodiment of the present application. As shown in Fig. 2, the vehicle control method comprises the following steps. Figure 1

[0023] S101, in a driver assistance system, collecting a panoramic view, point cloud data, and driving information;

[0024] ​S102, determine the driving mode based on the panoramic view and the driving information, and determine the driving mode based on the driving environment, the driving mode representing the longitudinal control speed interval and the steering angle interval in the corresponding driving environment;

[0025] S103, use the trained road perception model to process the panoramic view and the point cloud data to obtain a road graph containing concave and convex objects;

[0026] S104, in the road graph, calculate the height variance of the concave and convex objects in each preset area, and if the height variance is less than a first threshold, obtain a driving path sequence based on the height variance;

[0027] S105, determine the target driving path from the driving path sequence based on the control factors and the height variance, and control the vehicle to drive based on the target driving path, the control factors including at least vehicle speed, lateral acceleration, longitudinal acceleration and steering angle interval;

[0028] S106, if the height variance exceeds the first threshold and the concave and convex objects exceed the lateral control range, control the vehicle to decelerate based on the corresponding longitudinal control speed interval of the driving mode.

[0029] Specifically, the panoramic view, point cloud data and driving information are collected, and the current driving environment is determined based on the panoramic view and the driving information. The driver assistance system automatically determines the appropriate driving mode based on the driving environment. The driving mode can represent the longitudinal control speed interval and the steering angle interval under the current driving environment and the current speed. Then, the trained road perception model is used to automatically process the panoramic view and the point cloud data to obtain a road graph containing concave and convex objects on the road.

[0030] Based on this road graph, the height variance of the concave and convex objects in each preset area in the current road can be analyzed, and the driving path sequence that can be smoothly passed is obtained based on the obtained height variance analysis. The most suitable target driving path is considered in combination with the control factors of each area. Of course, if the calculated height variance exceeds the first threshold and the concave and convex objects cover the lateral control range, the system can decelerate according to the corresponding longitudinal control interval in the driving mode, so that the vehicle can pass through the area where the concave and convex objects are located relatively smoothly without a target driving path. Of course, in this case, the vehicle can also be controlled to decelerate and send a prompt information to the driver to make the driver control the vehicle to make the vehicle drive smoothly. When the driver manually controls, the control service can be exited and the vehicle can be fully controlled by the driver.

[0031] Further, the trained road perception model can accurately identify the concave and convex parts in the road and render them as concave and convex objects to obtain a road graph for subsequent analysis and processing.

[0032] Furthermore, the mean square error of the heights of the concave and convex objects in each preset area is calculated in order to analyze the concave and convex conditions in the road. By calculating the mean square error, the height difference of the concave and convex objects in each preset area can be represented. If the mean square error is larger, the height difference of the concave and convex objects in the area is larger, which means that the shaking of the vehicle will be more serious when passing by. When the mean square error exceeds the first threshold, detour will be considered. However, if the range where the mean square error exceeds the first threshold exceeds the lateral vehicle control range, deceleration and slow passing can be considered. The lateral vehicle control range can be determined according to factors such as the current vehicle speed and driving mode. There is no specific restriction. Different lateral vehicle control ranges for different situations and different models can be determined through pre-debugging.

[0033] The mean square error can be obtained by calculating the average value of the bump objects, then calculating the difference between the height of each bump object and the average value, squaring each difference, adding up all the squared differences, and finally dividing the sum by the number of bump objects.

[0034] Among them, the range of each preset area can be determined according to the width of the vehicle tire and body, and the range size is set mainly based on whether the vehicle tire can pass smoothly within the area or whether it can bypass it. For example, when the vehicle tire is 200mm, an area with a width and height of 220mm can be set to calculate the mean square error of the concave and convex objects in the corresponding area. After that, the mean square error of all areas with a width and height of 220mm in the road map can be traversed to obtain the driving path sequence. Of course, the specific area range needs to be determined in combination with the actual road conditions, vehicle width, wheel width and other influencing factors. There is no restriction here. Therefore, before the implementation of this solution, different models can be debugged on different roads to determine it. It should be understood that the width and height of the preset area can represent a two-dimensional area in the road map.

[0035] It should also be understood that when determining the driving path sequence based on the height mean square deviation, the height mean square deviation of all preset areas in the road map can be comprehensively analyzed to determine the driving path sequence, such as Figure 2 As shown in the figure, if the mean square difference of the heights of areas A and E is less than the first threshold, but the mean square difference of the heights of areas B, C, D, and F are all greater than the first threshold, then the driving path sequence at this time includes the path AE, and no restriction is made here.

[0036] Among them, the first threshold can be determined comprehensively based on the vehicle body and chassis parameters. For example, the mean square deviation that the vehicle can accept without deceleration is 1.0. However, this is not limited here. The specific first threshold can be determined comprehensively based on the specific vehicle body and chassis parameters. For example, the first threshold of the mean square deviation that the vehicle can accept can be comprehensively judged based on the chassis height, chassis spring coefficient, etc.

[0037] Of course, before calculating the height mean square error, it can also be judged whether the height of the protrusion meets the vehicle driving through, for example, there is a large obstacle in front, at this time the first choice can be to bypass, if not, it can be considered to slow down and stop.

[0038] The longitudinal control vehicle speed interval represents the speed change interval that can be smoothly accelerated and decelerated in the current driving environment, for example, in the case of non-paved road and driving speed less than 30km / h, the longitudinal control vehicle speed interval can be ±10km / h, and the specific vehicle speed change can be automatically determined by the system according to the current driving state and driving environment. The steering angle interval of the vehicle head can represent the controllable vehicle head deflection angle that the vehicle can maintain lane driving in the current driving environment, such as in wet and slippery mode in rainy and snowy weather, if the speed is below 60km / h, the steering angle interval of the vehicle head can be set to 10°, if the speed is above 60km / h, the steering angle interval of the vehicle head can be set to 5°. Of course, no restrictions are made here, and the specific numerical determination can be adjusted according to the corresponding chassis parameters, and also needs to be determined according to the specific road type (such as road width, road surface condition, highway, urban road, etc.), weather condition, driving experience, cloud big data, etc.

[0039] According to the method of the above embodiment, the panoramic view, point cloud data and driving information can be collected by multiple sensors, and then the driving environment and driving mode are determined, the road condition is analyzed to obtain a road map containing concave and convex objects, and the target driving path of the vehicle is obtained combined with the road map, or the vehicle is controlled to slow down or stop without directly passing through the path, so that the vehicle is controlled in a gentle form when facing complex road conditions, the vehicle shaking is slowed down, and passengers have a comfortable driving experience, and accidents caused by the driver's inability to react due to excessive speed and too close roadblocks are avoided.

[0040] In some embodiments, before collecting the panoramic view, point cloud data and driving information, it further includes: in response to the vehicle self-checking result, if the vehicle sensor is working normally and the vehicle is in normal working condition, it is judged whether the vehicle starts the speed cruise or the lane keeping system; if the vehicle starts the speed cruise or the lane keeping system, the vehicle control service is started.

[0041] Specifically, before the vehicle enters the automatic vehicle control service, the vehicle also needs to be self-checked, and the vehicle control service can be started only if the current vehicle body meets the premise of starting the vehicle control service, and then the data is collected to enter the vehicle control service process.

[0042] Further, the vehicle self-checking process at least includes checking the vehicle sensors and the vehicle working condition. If the vehicle sensors work normally, for example, the vehicle camera is not blocked, there is no fogging or failure, etc., which indicates that the camera works normally. If the sensor can normally receive data, it works normally. When the sensor works normally, the vehicle working condition is checked. If the vehicle working condition is also normal, it is further judged whether the current vehicle is started with a speed cruise or a lane keeping system. If the vehicle is started with a speed cruise or a lane keeping system, the vehicle control service can be started.

[0043] According to the method of the above embodiment, by vehicle self-checking, the vehicle sensors and the vehicle are checked to see if they are in a normal state. If the sensor is normal and the vehicle working condition is normal, the system will further detect whether the vehicle is started with a speed cruise or a lane keeping system. In the state of activating the speed cruise or the lane keeping system, the vehicle control service is started. After the vehicle control service is started, the vehicle will automatically collect panoramic view, point cloud data and driving information and other data through various sensors.

[0044] In some embodiments, the driving information includes temperature, humidity and wheel speed. Collecting the panoramic view, point cloud data and driving information includes: obtaining the panoramic view by capturing the camera, obtaining the point cloud data according to the laser radar, and spatially aligning the panoramic view and the point cloud data; obtaining the temperature, humidity and wheel speed through the temperature sensor, humidity sensor and wheel speed controller.

[0045] Specifically, the required data can be collected by various sensors and processed for subsequent use, that is, the image data of the surrounding environment can be captured by the camera, and the panoramic view can be captured by multiple cameras (such as front, rear, left front, right front, left rear and right rear cameras) or a panoramic camera. The point cloud data is collected by the laser radar, which can be placed on the top of the vehicle or other suitable positions for omnidirectional scanning of the surrounding environment to capture the most complete and accurate point cloud data to help the system generate a three-dimensional road map. The temperature and humidity outside the vehicle can also be collected by the temperature sensor and the humidity sensor to help the system analyze the environment, and the wheel speed of each wheel can be obtained by the wheel speed controller, so that the real-time driving state of the vehicle, such as speed and acceleration, etc. Information can be obtained.

[0046] After obtaining each data through the sensor, the point cloud data and the panoramic view can be preliminarily processed in space synchronization. The reason is that the installation position and installation angle of the camera and the laser radar on the vehicle can be different, which causes the angle of the point cloud data and the panoramic view received by the system to be different. Therefore, the point cloud data and the panoramic view can be spatially aligned and overlapped to facilitate accurate rendering of three-dimensional concave and convex objects.

[0047] It should be understood that after the corresponding data is acquired by each sensor, the corresponding data can be temporarily stored in a database, and if the system judges that the vehicle control service is started, the corresponding data is collected from the database, that is, a temporary database can be constructed to store the data acquired by each sensor, which is not limited here.

[0048] According to the method of the above embodiment, the corresponding data can be collected by each sensor and integrated into the driver assistance system, which helps the system to more accurately perceive and understand the surrounding environment, and then make more accurate vehicle control decisions and control.

[0049] In some embodiments, the panoramic view and the point cloud data are spatially aligned, including: taking the center of the rear axle of the vehicle as the origin, establishing a front-left-up-right-hand coordinate system with the driving direction as the x-axis, and constructing a virtual coordinate point on the right-hand coordinate system; the time-synchronized point cloud data and the panoramic view are respectively spatially rotated and translated based on the virtual coordinate point to achieve spatial alignment of the point cloud data and the panoramic view.

[0050] Specifically, the point cloud data and the panoramic view can be first time-aligned to obtain time-synchronized point cloud data and panoramic view. The main performance is that the sampling rates of the camera and the laser radar are different, so a smaller sampling rate can be used as a sampling standard for time synchronization.

[0051] Further, when spatially aligning the point cloud data and the panoramic view, the front view and the front point cloud data can be used for alignment. First, the front-left-up-right-hand coordinate system with the driving direction as the x-axis can be established as the driving coordinate system with the center of the rear axle of the vehicle as the origin, and a virtual coordinate point (Xq, 0, Zq) can be constructed as a reference point in the driving coordinate system to provide a reference frame for the panoramic view and the point cloud data. During processing, the internal and external parameters of the camera can be obtained by the checkerboard calibration method, and the panoramic view and the point cloud data can be mapped to the virtual coordinate point by spatial rotation and translation to ensure that they have the same angle and direction to achieve spatial alignment.

[0052] After that, the super-resolution algorithm of the laser radar point cloud data can be used to obtain dense point cloud to improve the density and accuracy of the point cloud data. Then, the 2D image information and the laser radar point cloud data are matched by outputting the target outline in the panoramic view to remove abnormal points and ensure the accuracy and integrity of the data. The spatial synchronization of the laser radar and the visual information can be further realized to provide reliable data support for subsequent perception and decision-making. Of course, the process of further realizing spatial synchronization can be performed when acquiring the panoramic view and the point cloud data, or in the trained road perception model, which is not limited here and can be determined according to the specific system implementation effect and data transmission effect.

[0053] According to the method of the above embodiment, the point cloud data and the panoramic view can be mapped by establishing a coordinate system, realizing data consistency and accuracy between the laser radar and the camera, and providing more reliable environment perception and information processing capability for vehicle control services.

[0054] In some embodiments, the current driving environment is determined based on the panoramic view and the driving information, and the driving mode is determined based on the driving environment, including: fusing temperature, humidity, wheel speed and panoramic view to obtain fusion data; inputting the fusion data into a multi-class discriminator to determine the driving environment, and determining the driving mode based on the driving environment and the vehicle speed, the driving mode including at least non-paved road mode, wet and slippery mode, sunny mode and fog barrier mode.

[0055] Specifically, the current driving environment can be determined by temperature, humidity, wheel speed and panoramic view, the above data can be fused to obtain fusion data, and the fusion data can be input into a multi-class discriminator to determine the current driving environment, and the driving mode can be determined based on the driving environment and the vehicle speed.

[0056] Further, the multi-class discriminator can be a pre-trained module that can select the driving mode based on the driving environment. The multi-class discriminator can classify the input data, i.e., the fusion data of temperature, humidity, wheel speed and panoramic view, to obtain the current driving environment, which can include at least weather conditions and road conditions. Then, the most suitable driving mode, such as non-paved road mode, wet and slippery mode, sunny mode and fog barrier mode, can be obtained based on the driving environment and the current speed. The longitudinal vehicle control interval and the steering angle interval of the driving mode as described above can be determined according to specific conditions and pre-debugging, which is not limited here.

[0057] As an example, the current driving environment is non-paved road and rainy and snowy weather, and the driving mode is non-paved road mode and wet and slippery mode. When the current vehicle speed is less than 30 km / h, the longitudinal vehicle control speed interval of the non-paved road mode is ±10 km / h, and the steering angle is 30°. However, as known from the pre-control variable debugging, the longitudinal vehicle control speed interval of the wet and slippery mode is ±15 km / h, and the steering angle interval is 25° when the speed is less than 30 km / h. Therefore, under the current driving environment, the driving mode can be determined as longitudinal vehicle control speed interval of ±15 km / h and steering angle interval of 25°, which can maximize the stability of the vehicle body while ensuring the safety of the vehicle body and not deviating from the lane.

[0058] It should be understood that the driving mode can represent the whole task when the vehicle control service is started, that is, not only the uneven object is executed when the uneven object appears on the road, but also the driving assistance system can control the vehicle to achieve the most suitable current driving speed and direction according to the driving mode on the flat road. For example, when an obstacle that cannot be crossed is encountered in the road section, the vehicle can be controlled to bypass the obstacle according to the steering angle interval of the vehicle head in the driving mode. Of course, if the obstacle cannot be bypassed, the vehicle can be controlled to stop.

[0059] It should also be understood that the driving mode can set the steering angle interval of the vehicle head, or can be set to control the vehicle in the lane, that is, without setting a specific steering angle interval of the vehicle head, but automatically adjusting according to the width of the current road to keep the vehicle driving in the lane. Of course, the steering angle interval can also represent the steering angle interval considering keeping driving in the lane, which is not expanded here.

[0060] According to the method of the above embodiment, the current driving environment can be judged according to the comprehensive data in the driver assistance system, and the appropriate driving mode can be automatically determined to improve the driving safety and comfort, provide better driving experience and protection for the driver, and also provide reliable technical support for realizing automatic driving.

[0061] In some embodiments, the panoramic view and the point cloud data are processed by using the trained road perception model to obtain a road map containing uneven objects, including: using the trained road perception model to perform three-dimensional modeling on the panoramic view to obtain a modeling result containing pits and protrusions, using dense point cloud to constrain the modeling result, and rendering the constrained modeling result to obtain an initial road map containing all uneven objects, and the dense point cloud is calculated based on the point cloud data by a super-resolution algorithm; the initial road map is smoothed by a window of a predetermined size to obtain a road map.

[0062] Specifically, the spatially synchronized point cloud data and the panoramic view can be input into the trained road perception model for three-dimensional modeling to obtain an initial road map containing all road uneven objects, and the initial road map is smoothed to remove the negligible uneven objects in the road that have no effect on vehicle driving, to obtain the above-mentioned road map. It should be understood that the uneven objects of course include road signs, plants, buildings, etc., but the application focuses on the vehicle control service process based on the unevenness of the road surface, so the uneven objects mainly represent pits and protrusions on the road surface, such as stones, potholes, and crushed road surfaces.

[0063] Further, the processing process using the trained road perception model can be represented as the remapping of the road surface object in the three-dimensional space (such as the Bird's Eye View, BEV space). The 6 RGB images of the multi-view, i.e. the panoramic view, can be used as the input. The picture encoder module is used to complete the feature extraction in the three-dimensional space to obtain the three-dimensional features of the road surface elements. The IPM is used in the transformer to complete the 2D to 3D conversion. The BEV+transfomer and the occupation module are used to enhance the features in the three-dimensional space, and then the three-dimensional modeling is performed to accurately model the concave and convex objects and other elements on the road surface. After that, the initial road map containing all the concave and convex objects is obtained by rendering the three-dimensional modeled concave and convex objects. In this way, when analyzing the road surface elements, not only the convex objects can be identified, but also the concave objects can be accurately identified. When rendering, the concave objects can be rendered downward, i.e. in the negative direction of the Z-axis of the right-handed coordinate system. It should be understood that before rendering, the voxel and camera data, i.e. the three-dimensional modeling features, can be constrained using the dense point cloud to obtain more accurate rendering results.

[0064] After that, the initial road map can be smoothed, i.e. a window of a predetermined size (such as a 3*3 sliding window) can be used to slide in the initial road map, and the data in the window can be smoothed to remove excessive data that has no effect on the vehicle driving. In this way, the road map can be made more clear and continuous, and the subsequent calculation of the height squared difference can not be affected by excessive data. In this way, the pre-trained road perception model can obtain a complete and accurate road map.

[0065] It should be understood that the road perception model can be continuously tested and debugged to obtain a trained road perception model that accurately identifies road elements and concave and convex objects. In model training, the data collection of the concave pits can be performed in a time sequence, i.e. the visual changes of the same target at different distances and angles are recorded, and the depth width with the largest trend of depth change of the same target is recorded by the laser radar, and the concave pits are processed in turn, but no specific limitation is made here.

[0066] According to the method of the above embodiment, the road concave and convex objects can be rendered according to the trained road perception model, the convex objects and the concave objects can be accurately identified, and a complete and accurate road map can be obtained, which provides a more reliable data basis for subsequent data analysis of the vehicle service, so as to ensure that the vehicle can smoothly drive when facing the concave and convex objects, i.e. unpaved roads, potholes and stone obstacles.

[0067] In some embodiments, after obtaining the road map containing the concave-convex object, further comprising: if there is a target concave-convex object higher than the second threshold in the concave-convex object, obtaining the distance between the target concave-convex object and the vehicle; if the distance is less than the preset distance and the vehicle speed is greater than the preset vehicle speed, the vehicle control service is closed and warning information is sent to the driver.

[0068] Specifically, in the process of vehicle control service, if a target concave-convex object higher than the second threshold is identified in the trained road perception model in front of the vehicle, and the distance is too close and the speed is too fast, the vehicle control service can be closed and warning information can be sent to the driver. The warning information can include reminding the driver that there is an obstacle in front of the vehicle that cannot be passed, and prompting to slow down and stop. Of course, if a particularly urgent situation is encountered, the system can also control the vehicle to brake, i.e. automatic emergency braking. This will not be expanded here. It should be understood that the obstacle in front of the vehicle that cannot be passed may be a situation where the vehicle is currently driving and the driving environment, if the vehicle is controlled to turn around the obstacle, it will deviate from the current lane. Therefore, in order to ensure driving safety and avoid traffic accidents, the vehicle can be controlled to slow down and stop. This is not limited here.

[0069] It should also be understood that the second threshold can be set according to the vehicle chassis height or can be determined based on other factors, for example, the second threshold is 300mm, which is not limited here. The preset vehicle speed can be determined based on the current driving state, driving mode and the distance between the obstacle and the vehicle, i.e. the preset vehicle speed needs to be determined according to the specific situation. A large amount of data can be used to train the system so that the system can accurately calculate the preset vehicle speed according to the current specific situation. Therefore, the specific preset vehicle speed is not limited here.

[0070] As an example, in heavy fog weather, the visible distance is 100 meters, and the vehicle identifies an obstacle in front of the vehicle through laser radar, but at this time the obstacle cannot be seen or is very blurred with the naked eye, and at this time the vehicle speed is too fast to slow down to a stop through the current longitudinal vehicle control interval to the obstacle, and it is also impossible to control the vehicle to bypass the obstacle through the vehicle steering angle interval. Therefore, the driver assistance system can prompt the driver to slow down or perform emergency braking. It can be understood that the target concave-convex object at this time is mainly a convex obstacle, but it is not limited. It should also be understood that the preset vehicle speed can be determined based on different driving environments and driving modes, and can be implemented in actual application. This will not be expanded here.

[0071] It should also be noted that according to the present embodiment, if the visible range is small when generating the road map, the infrared or other sensors can be used to assist in generating the road map based on the point cloud data to expand the visible range of the road map. However, this is not limited here.

[0072] Of course, in addition to closing the vehicle control service when facing obstacles, the user can manually exit the vehicle control service, or the user exits the cruise control or lane keeping system, and the vehicle control service will automatically exit. If you want to start the vehicle control service again, you can first start the cruise control or lane keeping system and then manually start the vehicle control service.

[0073] According to the method of the above embodiment, the driver assistance system can help the driver to predict the road conditions in advance, and preventive measures can be taken in the case of excessive speed and small field of view of the driver to avoid potential accidents or damage. This scheme not only ensures the smooth driving of the vehicle on the concave-convex road, but also prevents according to the road conditions, further perfecting the automatic driving system.

[0074] Figure 3 is a flowchart of another vehicle control method provided by the present application. The following will be further described in combination with Figure 3 to further illustrate the present application.

[0075] After the vehicle is powered on, self-checking can be performed during vehicle driving. If the device is abnormal, a maintenance reminder will be issued and the vehicle control service will not be entered. If the sensors and vehicle state are normal, data collection is performed to obtain the data required for the vehicle control service. All collected data is preliminarily clarified, labeled and trained, and then input into the trained road perception model for processing to obtain a road graph containing concave-convex objects.

[0076] At the same time, it is judged whether the vehicle is started with cruise control or lane keeping. If not, the vehicle control service will not be started. If it is started, it is checked whether the vehicle control service is started. If not, the vehicle control service will not be executed. If it is started, it is further entered into the vehicle control service, i.e. the obtained road graph is analyzed.

[0077] The obtained road graph is analyzed to obtain the height mean square error of the concave-convex objects in the preset area, and then the height mean square error is compared with the first threshold value. If it does not exceed the first threshold value, the optimal target driving path is selected according to the vehicle control factors to smoothly drive through.

[0078] If the height mean square error exceeds the first threshold value, it is judged whether it can be bypassed, i.e. whether it exceeds the lateral vehicle control range. If it can be bypassed, the vehicle is controlled to bypass. If it cannot be bypassed, it is decelerated based on the driving mode and the driver is notified. At this time, it can be considered as exiting the vehicle control service.

[0079] After bypassing the obstacles or large potholes, if there are still concave-convex objects, i.e. obstacles, the target driving path can be determined for vehicle control.

[0080] After the driver exits the vehicle control service or the cruise control and the lane keeping are closed, the vehicle control service can be ended. In this way, a complete vehicle control service is realized, which can flexibly cope with complex road conditions and ensure the comfort and safety of the driver.

[0081] All the optional technical solutions described above can be combined to form optional embodiments of the present application, which will not be described one by one here.

[0082] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0083] Figure 4 is a schematic diagram of a vehicle control device provided by an embodiment of the present application. As shown in Figure 4 , the vehicle control device comprises:

[0084] The acquisition module 401 is configured to acquire a panoramic view, point cloud data and driving information in a driver assistance system;

[0085] The judgment module 402 is configured to judge a current driving environment based on the panoramic view and the driving information, and determine a driving mode based on the driving environment, the driving mode representing a longitudinal vehicle control speed interval and a vehicle head turning angle interval in a corresponding driving environment;

[0086] The processing module 403 is configured to process the panoramic view and the point cloud data by using a trained road perception model to obtain a road graph containing concave-convex objects;

[0087] The calculation module 404 is configured to calculate a height variance of the concave-convex objects in each preset area in the road graph, and obtain a driving path sequence based on the height variance if the height variance is less than a first threshold;

[0088] The first vehicle control module 405 is configured to determine a target driving path from the driving path sequence based on a vehicle control factor and the height variance, and control the vehicle to drive based on the target driving path, the vehicle control factor at least including a vehicle speed, a lateral acceleration, a longitudinal acceleration and the vehicle head turning angle interval;

[0089] The second vehicle control module 406 is configured to control the vehicle to decelerate based on a longitudinal vehicle control speed interval corresponding to the driving mode if the height variance exceeds the first threshold and the concave-convex objects exceed a lateral vehicle control range.

[0090] In some embodiments, the acquisition module 401 is specifically configured to, in response to a vehicle self-checking result, judge whether the vehicle starts a cruise control or a lane keeping system if the vehicle sensors are working normally and the vehicle is in a normal working condition; and start the vehicle control service if the vehicle starts the cruise control or the lane keeping system.

[0091] In some embodiments, the acquisition module 401 is specifically configured to capture a panoramic view through a camera, acquire point cloud data through a laser radar, and perform spatial alignment on the panoramic view and the point cloud data; and obtain temperature, humidity and wheel speed through a temperature sensor, a humidity sensor and a wheel speed controller.

[0092] In some embodiments, the acquisition module 401 is specifically configured to establish a front-left-up-right-hand coordinate system with the center of the rear axle of the vehicle as the origin and the driving direction as the x-axis, and construct a virtual coordinate point on the right-hand coordinate system; and perform spatial rotation and translation on the time-synchronized point cloud data and the panoramic view based on the virtual coordinate point respectively, to realize spatial alignment of the point cloud data and the panoramic view.

[0093] In some embodiments, the judgment module 402 is specifically configured to fuse the temperature, humidity, wheel speed and panoramic view to obtain fusion data; input the fusion data into a multi-class discriminator to determine the driving environment, and determine the driving mode based on the driving environment and the vehicle speed, the driving mode at least including a non-paved road mode, a wet and slippery mode, a sunny mode and a fog barrier mode.

[0094] In some embodiments, the processing module 403 is specifically configured to use the trained road perception model to perform three-dimensional modeling on the panoramic view to obtain a modeling result containing pits and protrusions, constrain the modeling result using dense point cloud, and render the constrained modeling result to obtain an initial road map containing all concave-convex objects, the dense point cloud being calculated based on the point cloud data through a super-resolution algorithm; and perform smoothing processing on the initial road map through a window of a preset size to obtain a road map.

[0095] In some embodiments, the processing module 403 is specifically configured to, if there is a target concave-convex object higher than a second threshold value in the concave-convex objects, acquire the distance between the target concave-convex object and the vehicle; and if the distance is less than a preset distance and the vehicle speed is greater than a preset vehicle speed, shut down the vehicle control service and send a warning information to the driver.

[0096] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0097] Figure 5 is a schematic diagram of an electronic device 5 provided by the embodiments of the present application. As shown in Figure 5As shown, the electronic device 5 of this embodiment includes a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. The processor 501 implements the steps in each of the above method embodiments when executing the computer program 503. Alternatively, the processor 501 implements the functions of each module / unit in each of the above apparatus embodiments when executing the computer program 503.

[0098] The electronic device 5 can be a desktop computer, a notebook computer, a palm computer, a cloud server, or the like. The electronic device 5 can include but is not limited to the processor 501 and the memory 502. Those skilled in the art can understand that the electronic device 5 can include more or less components, or different components than those shown. Figure 5 The electronic device 5 shown is merely an example and does not constitute a limitation on the electronic device 5, which can include more or less components, or different components than those shown.

[0099] The processor 501 can be a central processing unit (CPU), or other general purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or the like.

[0100] The memory 502 can be an internal storage unit of the electronic device 5, such as a hard disk or a memory of the electronic device 5. The memory 502 can also be an external storage device of the electronic device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like. The memory 502 can also include both an internal storage unit and an external storage device of the electronic device 5. The memory 502 is used to store computer programs and other programs and data required by the electronic device.

[0101] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above described functions. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0102] The integrated module / unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a storage medium. The computer program can be executed by a processor to implement the steps of each method embodiment. The computer program can include computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0103] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.​

Claims

1. A vehicle control method, characterized in that: include: In the driver assistance system, panoramic views, point cloud data, and driving information are collected, including temperature, humidity, and wheel speed; determining a current driving environment based on the panoramic view and the driving information, and determining a driving mode based on the driving environment, the driving mode representing a longitudinal vehicle control speed range and a vehicle head steering angle range in the corresponding driving environment, the driving mode including at least an unpaved road mode, a slippery mode, a sunny weather mode, and a foggy mode; Processing the panoramic view and the point cloud data using a trained road perception model to obtain a road map containing concave and convex objects; In the road map, calculating the mean square difference of the heights of the concave and convex objects in each preset area, and if the mean square difference of the heights is less than a first threshold, obtaining a driving path sequence based on the mean square difference of the heights; determining a target driving path from the driving path sequence based on vehicle control factors and the height mean square error, and controlling vehicle driving based on the target driving path, the vehicle control factors including at least vehicle speed, lateral acceleration, longitudinal acceleration, and the vehicle head steering angle range; If the height mean square error exceeds the first threshold and the concave-convex object exceeds the lateral vehicle control range, the vehicle is controlled to decelerate based on the longitudinal vehicle control speed range corresponding to the driving mode.

2. The method according to claim 1, characterized in that Before collecting the panoramic view, point cloud data and driving information, the following steps are also included: In response to the vehicle self-test result, if the vehicle sensors are functioning normally and the vehicle is in a normal operating condition, determining whether the vehicle has a cruise control or lane keeping system activated; If the vehicle has the cruise control or lane keeping system turned on, the vehicle control service is turned on.

3. The method according to claim 1, characterized in that The acquisition of panoramic views, point cloud data, and driving information includes: The panoramic view is obtained by capturing the panoramic view with a camera, the point cloud data is collected by a laser radar, and the panoramic view and the point cloud data are spatially aligned; The temperature, humidity and wheel speed are obtained through a temperature sensor, a humidity sensor and a wheel speed controller.

4. The method according to claim 3, characterized in that The spatially aligning the panoramic view and the point cloud data includes: Taking the center of the rear axle of the vehicle as the origin, establishing a front upper left right-hand coordinate system with the driving direction as the x-axis, and constructing a virtual coordinate point on the right-hand coordinate system; The time-synchronized point cloud data and the panoramic view are spatially rotated and translated based on the virtual coordinate points, respectively, to achieve spatial alignment of the point cloud data and the panoramic view.

5. The method according to claim 3, characterized in that The determining of the current driving environment based on the panoramic view and the driving information, and determining the driving mode based on the driving environment, includes: fusing the temperature, the humidity, the wheel speed, and the panoramic view to obtain fused data; The fused data is input into a multi-classification discriminator for judgment, a driving environment is determined, and a driving mode is determined based on the driving environment and the vehicle speed.

6. The method according to claim 1, characterized in that The method of processing the panoramic view and the point cloud data using the trained road perception model to obtain a road map containing concave and convex objects includes: Using the trained road perception model, three-dimensionally modeling the panoramic view to obtain a modeling result including potholes and bumps, constraining the modeling result using a dense point cloud, and rendering the constrained modeling result to obtain an initial road map including all concave and convex objects, wherein the dense point cloud is calculated based on the point cloud data using a super-resolution algorithm; The initial road map is smoothed using a window of a preset size to obtain the road map.

7. The method according to claim 2, characterized in that After obtaining the road map containing the concave-convex objects, the method further includes: If there is a target concave-convex object with a value higher than a second threshold value among the concave-convex objects, obtaining a distance between the target concave-convex object and the vehicle; If the distance is less than the preset distance and the vehicle speed is greater than the preset speed, the vehicle control service is turned off and a warning message is sent to the driver.

8. A vehicle control device, characterized in that: include: an acquisition module configured to acquire panoramic views, point cloud data, and driving information in a driver assistance system; The driving information includes temperature, humidity and wheel speed; a judgment module configured to judge a current driving environment based on the panoramic view and the driving information, and determine a driving mode based on the driving environment, the driving mode representing a longitudinal vehicle control speed range and a vehicle head steering angle range in the corresponding driving environment, the driving mode including at least an unpaved road mode, a slippery mode, a sunny weather mode, and a foggy obstacle mode; a processing module configured to process the panoramic view and the point cloud data using a trained road perception model to obtain a road map containing concave and convex objects; a calculation module configured to calculate a mean square difference of heights of the concave and convex objects in each preset area in the road map, and obtain a driving path sequence based on the mean square difference of heights if the mean square difference of heights is less than a first threshold; a first vehicle control module configured to determine a target driving path from the driving path sequence based on vehicle control factors and the height mean square error, and control vehicle driving based on the target driving path, wherein the vehicle control factors include at least vehicle speed, lateral acceleration, longitudinal acceleration, and the vehicle head steering angle range; The second vehicle control module is configured to control the vehicle to decelerate based on the longitudinal vehicle control speed range corresponding to the driving mode if the height mean square error exceeds the first threshold and the concave-convex object exceeds the lateral vehicle control range.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images