Vehicle control method and device, equipment, storage medium and product

Abstract

The invention discloses a vehicle control method and device, equipment, a storage medium and a product, and belongs to the technical field of automatic driving. Comprising the steps of performing data acquisition on a current road environment of a first vehicle through a plurality of target sensors installed on the first vehicle to obtain a plurality of pieces of first environment information; preprocessing the plurality of pieces of first environment information to obtain a plurality of pieces of second environment information; carrying out key information extraction on the plurality of pieces of second environment information through an information extraction model based on the weather information to obtain a plurality of pieces of third environment information, wherein the third environment information obtained by carrying out key information extraction on the second environment information is effective under the weather condition indicated by the weather information; based on the characteristic information of the plurality of target sensors, fusing the plurality of pieces of third environment information to obtain first target environment information; based on the first target environment information, key features of the road environment are determined through an environment perception model; and determining target control decision information of the first vehicle based on the key characteristics of the road environment.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a vehicle control method, device, equipment, storage medium and product. Background Art

[0002] In autonomous driving technology, the environment is perceived through cameras or lidars, and then corresponding decisions are made for the first vehicle based on the perceived environmental information; for example, adjusting the vehicle speed, changing lanes, or avoiding obstacles. Environmental perception is the key to the first vehicle's safe and stable driving, but the camera or lidar may be affected by the weather, resulting in inaccurate perceived environmental information; for example, the camera may be blurred by water droplets or fog, and the lidar may be affected by the scattering of raindrops or fog, resulting in a decrease in detection distance and accuracy. Therefore, how to improve the accuracy of sensor-based environmental perception in bad weather and make decisions based on perceived environmental information is the focus of industry research. Summary of the invention

[0003] The embodiments of the present application provide a vehicle control method, device, equipment, storage medium and product. The technical solution is as follows:

[0004] In one aspect, a vehicle control method is provided, the method comprising:

[0005] Determining current weather information of the first vehicle, the weather information being used to indicate current weather conditions;

[0006] Collecting data on the current road environment of the first vehicle through multiple target sensors installed on the first vehicle to obtain multiple first environmental information, where different target sensors are used to collect environmental information of different dimensions;

[0007] Preprocessing the plurality of first environmental information to obtain a plurality of second environmental information, wherein the preprocessing includes at least one of filtering processing, denoising processing and data synchronization processing;

[0008] Based on the weather information, extract key information from the plurality of second environmental information through an information extraction model to obtain a plurality of third environmental information, and for any second environmental information, the third environmental information obtained by extracting key information from the second environmental information is valid under the weather conditions indicated by the weather information;

[0009] Based on the characteristic information of the multiple target sensors, the multiple third environment information are fused through a fusion algorithm to obtain first target environment information of the current road environment of the first vehicle;

[0010] Based on the first target environment information, determining key features of the road environment through an environment perception model;

[0011] Based on the key features of the road environment, target control decision information of the first vehicle is determined, and the target control decision information is used to control the first vehicle.

[0012] In a possible implementation, the fusing of the plurality of third environment information by a fusion algorithm based on the characteristic information of the plurality of target sensors to obtain the first target environment information of the current road environment of the first vehicle includes:

[0013] Based on the characteristic information of the multiple target sensors, a part of the third environment information is deleted from the multiple third environment information to obtain multiple second target environment information, where the deleted third environment information includes erroneous environment information and / or redundant environment information; based on the weight of the target sensor to which the second target environment information belongs, a weighted average is performed on the multiple second target environment information to obtain the first target environment information of the current road environment of the first vehicle; or

[0014] Input the plurality of third environmental information into an information fusion model, and output first target environmental information of the current road environment of the first vehicle, wherein the information fusion model is used to fuse the plurality of third environmental information based on characteristic information of the plurality of target sensors; or

[0015] Based on the characteristic information of the multiple target sensors, a state space model of the multiple target sensors is established, the state space model includes a dynamic equation and an observation equation, the dynamic equation is used to predict the environmental information at the next moment, and the observation equation is used to determine the gain information of the sensor; the multiple third environmental information is used as prior information, and the fourth environmental information at the next moment is predicted through the dynamic equation based on the prior information, and the prediction error of the dynamic equation is determined based on the fourth environmental information and the multiple third environmental information. When the prediction error is less than a preset error, the fourth environmental information is determined as the first target environmental information; when the prediction error is not less than a preset error, the gain information of the multiple target sensors is determined based on the prediction error and the observation equation, and the fourth environmental information is corrected based on the gain information to obtain the fifth environmental information, and the fifth environmental information is used as the prior information, and then the step of predicting the multiple fourth environmental information at the next moment through the dynamic equation based on the prior information is performed until the first target environmental information is determined.

[0016] In another possible implementation manner, the process of filtering the first environment information to obtain the second environment information includes:

[0017] The first environment information is a first environment image, and the second environment information is a second environment image, then the first environment image is defogged to obtain the second environment image, and the contrast of the second environment image is higher than the contrast of the first environment image; or

[0018] The first environmental information is first point cloud data, and the second environmental information is second point cloud data. The second point cloud data is obtained by performing point cloud hierarchical filtering on the first point cloud data and removing raindrop noise in the first point cloud data.

[0019] In another possible implementation, determining target control decision information of the first vehicle based on the key features of the road environment includes:

[0020] Performing target detection based on key features of the road environment to obtain a target object included in the current road environment of the first vehicle;

[0021] Tracking the target object to obtain operation information of the target object;

[0022] Classifying the target object to obtain category information and attribute information of the target object;

[0023] Based on the operation information, category information and attribute information of the target object, target control decision information of the first vehicle is determined.

[0024] In another possible implementation, the method further includes:

[0025] Determine optimization information based on historical driving records of the plurality of second vehicles, the optimization information comprising at least one of parameter optimization information of the plurality of target sensors, first model optimization information of the information extraction model, algorithm optimization information of the fusion algorithm, and second model optimization information of the environment perception model;

[0026] Based on the parameter optimization information, optimizing the parameters of the multiple target sensors; and / or,

[0027] optimizing the information extraction model based on the first model optimization information; and / or,

[0028] Based on the algorithm optimization information, optimizing the fusion algorithm; and / or,

[0029] Based on the second model optimization information, the environmental perception model is optimized.

[0030] In another possible implementation, the method further includes:

[0031] Based on the weather information, a plurality of target sensors matching the weather conditions indicated by the weather information are determined from a plurality of sensors installed on the first vehicle, the plurality of target sensors including at least two types of sensors among meteorological sensors, optical sensors and microwave sensors.

[0032] In another aspect, a vehicle control device is provided, the device comprising:

[0033] A first determining module, used to determine current weather information of the first vehicle, wherein the weather information is used to indicate current weather conditions;

[0034] A collection module, used to collect data about the current road environment of the first vehicle through multiple target sensors installed on the first vehicle to obtain multiple first environmental information, where different target sensors are used to collect environmental information of different dimensions;

[0035] A preprocessing module, used for preprocessing the plurality of first environment information to obtain a plurality of second environment information, wherein the preprocessing includes at least one of filtering processing, denoising processing and data synchronization processing;

[0036] an extraction module, configured to extract key information from the plurality of second environmental information by using an information extraction model based on the weather information to obtain a plurality of third environmental information, wherein for any second environmental information, the third environmental information obtained by extracting key information from the second environmental information is valid under the weather condition indicated by the weather information;

[0037] a fusion module, configured to fuse the plurality of third environmental information through a fusion algorithm based on the characteristic information of the plurality of target sensors to obtain first target environmental information of the current road environment of the first vehicle;

[0038] A second determination module is used to determine the key features of the road environment through an environment perception model based on the first target environment information;

[0039] The third determination module is used to determine target control decision information of the first vehicle based on key features of the road environment, and the target control decision information is used to control the first vehicle.

[0040] In a possible implementation, the fusion module is used to delete part of the third environmental information from the plurality of third environmental information based on the characteristic information of the plurality of target sensors to obtain a plurality of second target environmental information, wherein the deleted third environmental information includes erroneous environmental information and / or redundant environmental information; based on the weight of the target sensor to which the second target environmental information belongs, perform weighted averaging on the plurality of second target environmental information to obtain the first target environmental information of the current road environment of the first vehicle; or

[0041] The fusion module is used to input the plurality of third environmental information into an information fusion model, and output first target environmental information of the current road environment of the first vehicle, wherein the information fusion model is used to fuse the plurality of third environmental information based on characteristic information of the plurality of target sensors; or

[0042] The fusion module is used to establish a state space model of the multiple target sensors based on the characteristic information of the multiple target sensors, the state space model includes a dynamic equation and an observation equation, the dynamic equation is used to predict the environmental information at the next moment, and the observation equation is used to determine the gain information of the sensor; taking the multiple third environmental information as prior information, predicting the fourth environmental information at the next moment through the dynamic equation based on the prior information, determining the prediction error of the dynamic equation based on the fourth environmental information and the multiple third environmental information, and determining the fourth environmental information as the first target environmental information when the prediction error is less than a preset error; determining the gain information of the multiple target sensors based on the prediction error and the observation equation when the prediction error is not less than a preset error, correcting the fourth environmental information based on the gain information to obtain the fifth environmental information, taking the fifth environmental information as the prior information, and then executing the step of predicting the multiple fourth environmental information at the next moment through the dynamic equation based on the prior information until the first target environmental information is determined.

[0043] In another possible implementation, the preprocessing module is used for, when the first environmental information is a first environmental image and the second environmental information is a second environmental image, performing defogging on the first environmental image to obtain the second environmental image, wherein the contrast of the second environmental image is higher than that of the first environmental image; or

[0044] The preprocessing module is used when the first environmental information is the first point cloud data and the second environmental information is the second point cloud data. The second point cloud data is obtained by performing point cloud hierarchical filtering on the first point cloud data to remove raindrop noise in the first point cloud data.

[0045] In another possible implementation, the third determination module is used to perform target detection based on key features of the road environment to obtain target objects included in the current road environment of the first vehicle; track the target objects to obtain operating information of the target objects; classify the target objects to obtain category information and attribute information of the target objects; and determine the target control decision information of the first vehicle based on the operating information, category information and attribute information of the target objects.

[0046] In another possible implementation, the device further includes:

[0047] a fourth determination module, configured to determine optimization information based on historical driving records of the plurality of second vehicles, the optimization information comprising at least one of parameter optimization information of the plurality of target sensors, first model optimization information of the information extraction model, algorithm optimization information of the fusion algorithm, and second model optimization information of the environment perception model;

[0048] An optimization module is used to optimize the parameters of the multiple target sensors based on the parameter optimization information; and / or, to optimize the information extraction model based on the first model optimization information; and / or, to optimize the fusion algorithm based on the algorithm optimization information; and / or, to optimize the environmental perception model based on the second model optimization information.

[0049] In another possible implementation, the device further includes:

[0050] A fifth determination module is used to determine, based on the weather information, a plurality of target sensors that match the weather conditions indicated by the weather information from a plurality of sensors installed on the first vehicle, wherein the plurality of target sensors include at least two types of sensors among meteorological sensors, optical sensors and microwave sensors.

[0051] On the other hand, a vehicle controller is provided, which includes a main control module, the main control module includes a processor and a memory, the memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the above-mentioned vehicle control method.

[0052] On the other hand, a computer-readable storage medium is provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor to implement the above-mentioned vehicle control method.

[0053] On the other hand, a computer program product is provided, wherein the product stores at least one program code, and the at least one program code is used to be executed by a processor to implement the above-mentioned vehicle control method.

[0054] In the embodiment of the present application, since rainy and foggy weather often leads to reduced visibility, the traditional single sensor is greatly limited in acquiring environmental information; therefore, multiple target sensors are used to perceive the environment, thereby utilizing the complementarity of multiple target sensors to improve the accuracy and reliability of environmental perception, thereby being able to provide early warning of potential dangers to the first vehicle, helping the first vehicle to perform safe avoidance and emergency handling, thereby significantly improving driving safety.

[0055] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a schematic diagram of an implementation environment of a vehicle control method shown in an exemplary embodiment of the present application;

[0057] Figure 2 is a flow chart of a vehicle control method shown in an exemplary embodiment of the present application;

[0058] Figure 3 is a block diagram of a vehicle control device shown in an exemplary embodiment of the present application;

[0059] Figure 4 It is a block diagram of a vehicle controller shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the technical solutions and advantages of the present application clearer, the implementation methods of the present application are described in further detail below.

[0061] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0062] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, the reference longitudinal control curves involved in this application are all obtained with full authorization.

[0063] Please refer to Figure 1 , which shows a schematic diagram of the implementation environment of the vehicle control method shown in an exemplary embodiment of the present application. The implementation environment includes a first vehicle, a vehicle controller 101 is configured in the first vehicle, and a first terminal is installed with multiple target sensors 102, and the vehicle controller 101 is used to communicate with the multiple target sensors 102 via a CAN (Controller Area Network) bus. In the embodiment of the present application, the multiple target sensors 102 are used to sense the environment, and transmit the sensed multiple first environmental information to the vehicle controller 101 via the CAN bus, and the vehicle controller 101 is used to control the first vehicle based on the multiple first environmental information.

[0064] In some embodiments, the first vehicle is a new energy vehicle, such as a pure electric vehicle, a plug-in hybrid vehicle, a fuel cell electric vehicle, etc. The plurality of target sensors 102 include at least two types of sensors among meteorological sensors, optical sensors, and microwave sensors, and the meteorological sensor includes at least one of a temperature and humidity sensor, an air pressure sensor, and a wind direction and speed sensor, the optical sensor includes at least one of a visible light camera, an infrared camera, and a laser radar, and the microwave sensor includes at least one of a millimeter wave radar and a microwave hygrometer.

[0065] Please refer to Figure 2 , which shows a flow chart of a vehicle control method shown in an exemplary embodiment of the present application. Figure 2 , the method comprising:

[0066] Step 201: The vehicle controller determines the current weather information of the first vehicle, where the weather information is used to indicate the current weather conditions.

[0067] The weather conditions may be rainy, foggy, snowy, sunny, windy, etc. In one possible implementation, a weather sensor is installed in the first vehicle, and the vehicle controller determines the current weather information of the first vehicle through the weather sensor. In another possible implementation, a weather application is installed in the vehicle terminal of the first vehicle, and the vehicle controller determines the current weather information of the first vehicle through the weather application.

[0068] In one possible implementation, regardless of weather conditions, the vehicle controller will use multiple sensors installed on the first vehicle to collect data; accordingly, the vehicle controller determines the multiple sensors as multiple target sensors, thereby using enough sensors to collect data, thereby improving the comprehensiveness of data collection.

[0069] In another possible implementation, different weather information corresponds to different sensors; accordingly, after the vehicle controller determines the weather information, based on the weather information, it determines multiple target sensors that match the weather conditions indicated by the weather information from multiple sensors installed on the first vehicle. The multiple target sensors include at least two types of sensors among meteorological sensors, optical sensors, and microwave sensors.

[0070] The meteorological sensor includes at least one of a temperature and humidity sensor, an air pressure sensor, and a wind direction and speed sensor. The temperature and humidity sensor is used to collect the current ambient temperature and humidity of the first vehicle, the air pressure sensor is used to collect the current ambient air pressure of the first vehicle, and the wind direction and speed sensor is used to collect the current ambient wind direction and ambient wind speed of the first vehicle.

[0071] The optical sensor includes at least one of a visible light camera (high-definition camera), an infrared camera, and a laser radar. The visible light camera is used to capture images within the visible light range, the infrared camera is used to capture images within the infrared light range, and the laser radar is used to determine point cloud data by emitting laser pulses and the emitted laser pulses. The point cloud data includes spatial coordinate information, reflection intensity information, and other auxiliary information. Among them, the spatial coordinate information includes the X-axis coordinate value, Y-axis coordinate value, and Z-axis coordinate value of each point. These coordinate values ​​represent the position of the point in three-dimensional space. By analyzing these coordinate values, the three-dimensional shape of objects such as pedestrians, vehicles, and obstacles can be obtained. Reflection intensity information refers to the light intensity of the laser pulse emitted by the laser radar that is reflected back by the object. Other auxiliary information includes the number of echoes, category, RGB color information, GPS time, scanning angle, and scanning direction.

[0072] Microwave sensors include at least one of millimeter wave radar and microwave hygrometer. Millimeter wave radar uses electromagnetic waves in the millimeter wave band for detection. Microwave hygrometer is used to detect global atmospheric humidity profile and heavy rainfall.

[0073] For example, LiDAR is responsible for detecting obstacles and terrain, and can provide relatively accurate data even in rainy and foggy weather. Millimeter-wave radar has strong penetration ability for rain and fog, and can provide stable detection results in bad weather. High-definition cameras can provide rich visual information to help the system identify road signs, vehicles, pedestrians, etc.

[0074] In an embodiment of the present application, by selecting sensors based on weather information, multiple target sensors adapted to the current weather can be selected for environmental perception, thereby improving the accuracy of environmental perception based on the determined multiple targets.

[0075] Step 202: The vehicle controller collects data on the current road environment of the first vehicle through multiple target sensors installed on the first vehicle to obtain multiple first environmental information. Different target sensors are used to collect environmental information of different dimensions.

[0076] In the embodiment of the present application, multiple target sensors can work effectively in adverse weather conditions such as rainy days and foggy days, and can capture key information of the environment. In addition, multiple target sensors can be selected according to specific application scenarios and requirements.

[0077] In addition, rainy and foggy weather often leads to reduced visibility, which greatly limits the ability of traditional single sensors, such as cameras or lidar, to obtain environmental information. Cameras may be blurred by water droplets or fog, and lidars may be affected by the scattering of raindrops or fog, resulting in reduced detection distance and accuracy. Therefore, multiple sensors are needed to make up for these deficiencies and improve the accuracy and reliability of environmental perception.

[0078] Step 203: The vehicle controller preprocesses the multiple first environmental information to obtain multiple second environmental information, where the preprocessing includes at least one of filtering processing, denoising processing and data synchronization processing.

[0079] In a possible implementation, the preprocessing includes filtering; then this step may be: the vehicle controller filters out interference signals in the multiple first environmental information to obtain multiple second environmental information. For example, the first environmental information is a first environmental image, and the second environmental information is a second environmental image. Then the vehicle controller performs defogging on the first environmental image to obtain a second environmental image. The contrast of the second environmental image is higher than that of the first environmental image, so the first environmental image is defogged to improve the contrast of the first environmental image and eliminate the visual impact of fog. The vehicle controller may perform defogging on the first environmental image through an image defogging algorithm to obtain a second environmental image; the image defogging algorithm may be a dark channel prior defogging algorithm or a defogging algorithm based on deep learning.

[0080] For example, the first environmental information is the first point cloud data, and the second environmental information is the second point cloud data. The raindrop noise in the first point cloud data is removed by performing point cloud hierarchical filtering on the first point cloud data to obtain the second point cloud data, thereby removing the raindrop noise and ensuring the accuracy of the point cloud data.

[0081] In another possible implementation, the preprocessing includes denoising; then this step may be: the vehicle controller removes noise signals from the multiple first environmental information to obtain multiple second environmental information. In another possible implementation, the preprocessing includes data synchronization; then this step may be: the vehicle controller performs time synchronization and spatial alignment on the multiple first environmental information to obtain multiple second environmental information, thereby ensuring the consistency of the multiple first environmental information.

[0082] In an embodiment of the present application, the vehicle controller pre-processes the multiple first environmental information to eliminate noise and interference in the multiple first environmental information, thereby improving data quality and also ensuring the accuracy and consistency of the multiple first environmental information.

[0083] Step 204: Based on the weather information, the vehicle controller extracts key information from multiple second environmental information through an information extraction model to obtain multiple third environmental information. For any second environmental information, the third environmental information obtained by extracting key information from the second environmental information is valid under the weather conditions indicated by the weather information.

[0084] The information extraction model can learn how to extract useful information from multiple second environmental information in rainy and foggy weather, so that subsequent fusion can be performed based on the extracted multiple third environmental information.

[0085] Step 205: The vehicle controller fuses the plurality of third environment information through a fusion algorithm based on the characteristic information of the plurality of target sensors to obtain the first target environment information of the current road environment of the first vehicle.

[0086] The first method: based on the characteristic information of multiple target sensors, the vehicle controller deletes part of the third environmental information from the multiple third environmental information to obtain multiple second target environmental information, where the deleted third environmental information includes erroneous environmental information and / or redundant environmental information; based on the weight of the target sensor to which the second target environmental information belongs, the multiple second target environmental information are weighted averaged to obtain the first target environmental information of the current road environment of the first vehicle.

[0087] The characteristic information of the target sensor is used to indicate the characteristics of the target sensor, which may be specifically the advantages of the target sensor; accordingly, the vehicle controller may delete part of the third environmental information from the plurality of third environmental information to obtain the plurality of second target environmental information based on the characteristic information of the plurality of target sensors as follows: for the plurality of target sensors, when the same type of environmental information exists in the plurality of target sensors, for the sake of convenience of description, the two sensors collecting the same type of environmental information are called the first sensor and the second sensor, and the vehicle controller determines the sensor with high accuracy under the weather conditions indicated by the weather information based on the characteristics of the first sensor and the characteristics of the second sensor, retains the third environmental information collected by the sensor with high accuracy, and deletes the third environmental information collected by the sensor with low accuracy. Alternatively, for each target sensor, the vehicle controller verifies the currently collected third environmental information based on the historical environmental information collected by the target sensor before the current time to determine whether the third environmental information is correct; if the third environmental information is correct, retains the third environmental information; if the third environmental information is incorrect, deletes the third environmental information.

[0088] The second implementation method: the vehicle controller inputs multiple third environment information into the information fusion model, and outputs the first target environment information of the current road environment of the first vehicle. The information fusion model is used to fuse the multiple third environment information based on the characteristic information of multiple target sensors.

[0089] In the embodiment of the present application, the vehicle controller uses an information fusion model to fuse environmental information, thereby improving the accuracy of information fusion.

[0090] The third implementation method: the vehicle controller establishes a state space model of multiple target sensors based on the characteristic information of multiple target sensors, and the state space model includes a dynamic equation and an observation equation. The dynamic equation is used to predict the environmental information at the next moment, and the observation equation is used to determine the gain information of the sensor; the multiple third environmental information is used as prior information, and the fourth environmental information at the next moment is predicted through the dynamic equation based on the prior information, and the prediction error of the dynamic equation is determined based on the fourth environmental information and the multiple third environmental information. When the prediction error is less than the preset error, the fourth environmental information is determined as the first target environmental information; when the prediction error is not less than the preset error, the gain information of the multiple target sensors is determined based on the prediction error and the observation equation, and the fourth environmental information is corrected based on the gain information to obtain the fifth environmental information, and the fifth environmental information is used as prior information, and then the step of predicting the multiple fourth environmental information at the next moment through the dynamic equation based on the prior information is performed until the first target environmental information is determined.

[0091] It should be noted that the vehicle controller can use any of the above three methods to perform information fusion. Alternatively, the vehicle controller uses the above three methods to perform information fusion at the same time, and then fuses the three first target environment information obtained by fusion again to obtain a first target environment information. Alternatively, the vehicle controller selects a fusion method that is suitable for the weather conditions indicated by the weather information from the three methods based on the weather information, and performs information fusion based on the fusion method.

[0092] In the embodiment of the present application, the vehicle controller fully considers the complementarity and redundancy of each target sensor. For example, laser radar and millimeter wave radar have advantages in detecting obstacles, while cameras can provide rich visual information; therefore, by fusing the first environmental information of multiple target sensors, all-round perception of the environment can be achieved, thereby obtaining a more comprehensive and accurate environmental perception result.

[0093] Step 206: The vehicle controller determines key features of the road environment through an environment perception model based on the first target environment information.

[0094] The key features include object features such as the shape and size of the target object; when the target object is a movable object such as a pedestrian or a vehicle, the key features also include object features such as the moving speed and moving direction of the target object. In addition, since the multiple target sensors include meteorological sensors, the key features also include environmental features such as the temperature, humidity and visibility of the environment.

[0095] Step 207: The vehicle controller determines target control decision information of the first vehicle based on the key features of the road environment, and the target control decision information is used to control the first vehicle.

[0096] This step can be implemented by following the steps 2071-2074, including:

[0097] Step 2071: The vehicle controller performs target detection based on key features of the road environment to obtain a target object included in the current road environment of the first vehicle.

[0098] The vehicle controller performs target detection based on the key features of the road environment through a target detection algorithm to obtain a target object; the target object may be a vehicle, a pedestrian, or an obstacle, etc. The target detection algorithm may be YOLO (You Only Look Once, target detection algorithm) or SSD (Single Shot MultiBox Detector, target detector), etc. In the embodiment of the present application, through an advanced target detection algorithm, targets such as vehicles and pedestrians in the scene can be accurately identified.

[0099] Step 2072: The vehicle controller tracks the target object to obtain the operation information of the target object.

[0100] The operation information includes motion trajectory, motion speed, etc. The vehicle controller continuously tracks the target object through the tracking algorithm to obtain the motion information of the target object.

[0101] Step 2073: The vehicle controller classifies the target object to obtain category information and attribute information of the target object.

[0102] For example, if the target object is a vehicle, the category information of the target object is vehicle, and the attribute information is information such as the color and size of the vehicle. For another example, if the target object is a pedestrian, the category of the target object is person, and the attribute information is gender and age range.

[0103] In this step, the vehicle controller classifies the target object through a classification algorithm to obtain category information and attribute information of the target object. The classification algorithm can be a classification model obtained by training a convolutional neural network.

[0104] Step 2074: The vehicle controller determines the target control decision information of the first vehicle based on the operation information, category information and attribute information of the target object.

[0105] The vehicle controller provides a decision information acquisition model based on the motion information, category information and attribute information of the target object to determine the target control decision information of the first vehicle. The target control decision information is used to control the first vehicle so that the first vehicle makes corresponding decisions; for example, adjusting the vehicle speed, changing lanes or avoiding obstacles. In one possible implementation, the first vehicle is an autonomous driving vehicle or an intelligent driving vehicle, and the vehicle controller outputs the target control decision information to the actuator of the first vehicle, and the actuator of the first vehicle executes the corresponding decision based on the target control decision information. The actuator can be a braking system or a steering system, etc.

[0106] In another possible implementation, the first vehicle is a non-autonomous driving vehicle or a non-intelligent driving vehicle, and the vehicle controller outputs target control decision information, and the driver of the first vehicle executes a corresponding decision based on the target control decision information. For example, the vehicle controller prompts the driver to decelerate, and the driver executes a deceleration operation.

[0107] In the embodiment of the present application, in rainy and foggy weather, the vehicle controller can adjust the vehicle's driving state and control strategy in real time to adapt to different road conditions and traffic environments to ensure stable driving of the vehicle. In addition, the method provided in the embodiment of the present application is not only applicable to self-driving cars, but can also be applied to other vehicles or systems that need to perceive the environment in rainy and foggy weather, and has broad application prospects.

[0108] In the embodiment of the present application, in order to continuously improve the perception performance and robustness of the system, the vehicle controller needs to optimize and evaluate the system regularly. The optimization includes target sensor parameters, optimization fusion algorithm, update information extraction model, update environment perception model, update target detection algorithm, etc. At the same time, the performance of the system is evaluated through actual testing and simulation experiments to ensure that it can work stably and reliably in various rainy and foggy weather conditions.

[0109] The process in which the vehicle controller needs to regularly optimize the system can be achieved through the following steps (1) to (5), including:

[0110] (1) The vehicle controller determines optimization information based on historical driving records of multiple second vehicles, where the optimization information includes at least one of parameter optimization information of multiple target sensors, first model optimization information of an information extraction model, algorithm optimization information of a fusion algorithm, and second model optimization information of an environment perception model.

[0111] In a possible implementation, the historical driving record includes a plurality of first environmental information and control responses, and the control response is used to indicate the response of the target control decision information; for example, the control response is used to indicate that the decision corresponding to the target control decision information is accurate; for another example, the control response is used to indicate that the decision corresponding to the target control decision information is wrong. Accordingly, the step of determining the parameter optimization information of the plurality of target sensors based on the historical driving records of the plurality of second vehicles by the vehicle controller may be: the vehicle controller determines the parameter optimization information of the plurality of target sensors through the first optimization model based on the plurality of first environmental information and control responses. The parameter optimization information of the target sensor may be information such as the posture of the target sensor.

[0112] In another possible implementation, the historical driving record includes a plurality of second environmental information and control responses. Accordingly, the step of determining the first model optimization information of the information extraction model based on the historical driving records of the plurality of second vehicles by the vehicle controller may include: the vehicle controller updates the information extraction model based on the plurality of second environmental information and control responses, and determines the updated model parameters as the first model optimization information.

[0113] In another possible implementation, the historical driving record includes a plurality of third environmental information and control responses. Accordingly, the step of determining the first model optimization information of the information extraction model based on the historical driving records of the plurality of second vehicles by the vehicle controller may be: the vehicle controller determines the algorithm optimization information through the second optimization model based on the plurality of third environmental information and control responses. The algorithm optimization information is used to optimize the fusion algorithm; for example, the algorithm optimization information may be the updated weights of the plurality of target sensors; for another example, the algorithm optimization information may be the updated fusion algorithm, etc.

[0114] In another possible implementation, the historical driving record includes key features of the road environment and control responses. Accordingly, the step of determining the second model optimization information of the environment perception model based on the historical driving records of the plurality of second vehicles by the vehicle controller may include: the vehicle controller updates the environment perception model based on the key information of the road environment and the control response, and determines the updated model parameters as the second model optimization information.

[0115] (2) The vehicle controller optimizes the parameters of multiple target sensors based on the parameter optimization information.

[0116] (3) The vehicle controller optimizes the information extraction model based on the first model optimization information.

[0117] The vehicle controller modifies the model parameters of the information extraction model into the model parameters included in the first model optimization information, thereby optimizing the information extraction model.

[0118] (4) The vehicle controller optimizes the fusion algorithm based on the algorithm optimization information.

[0119] For example, if the algorithm optimization information is the updated weights of multiple target sensors, the vehicle controller modifies the weights of multiple target sensors in the fusion algorithm to the updated weights. For another example, if the algorithm optimization information is the updated fusion algorithm, the vehicle controller updates the fusion algorithm to the updated fusion algorithm.

[0120] (5) The vehicle controller optimizes the environmental perception model based on the second model optimization information.

[0121] The vehicle controller modifies the model parameters of the environment perception model into the model parameters included in the second model optimization information, thereby optimizing the environment perception model.

[0122] It should be noted that after the vehicle controller executes step (1), it executes at least one of steps (2)-(5).

[0123] In the embodiment of the present application, since rainy and foggy weather often leads to reduced visibility, the traditional single sensor is greatly limited in acquiring environmental information; therefore, multiple target sensors are used to perceive the environment, thereby utilizing the complementarity of multiple target sensors to improve the accuracy and reliability of environmental perception, thereby being able to provide early warning of potential dangers to the first vehicle, helping the first vehicle to perform safe avoidance and emergency handling, thereby significantly improving driving safety.

[0124] Please refer to Figure 3 , which shows a block diagram of a vehicle control device shown in an exemplary embodiment of the present application. The device includes:

[0125] A first determining module 301 is used to determine current weather information of the first vehicle, where the weather information is used to indicate current weather conditions;

[0126] A collection module 302 is used to collect data about the current road environment of the first vehicle through multiple target sensors installed on the first vehicle to obtain multiple first environmental information, where different target sensors are used to collect environmental information of different dimensions;

[0127] A preprocessing module 303 is used to preprocess the plurality of first environment information to obtain a plurality of second environment information, wherein the preprocessing includes at least one of filtering processing, denoising processing and data synchronization processing;

[0128] An extraction module 304 is used to extract key information from the plurality of second environmental information by using an information extraction model based on the weather information to obtain a plurality of third environmental information, wherein for any second environmental information, the third environmental information obtained by extracting key information from the second environmental information is valid under the weather condition indicated by the weather information;

[0129] A fusion module 305 is used to fuse the plurality of third environment information through a fusion algorithm based on the characteristic information of the plurality of target sensors to obtain first target environment information of the current road environment of the first vehicle;

[0130] A second determination module 306, configured to determine key features of the road environment through an environment perception model based on the first target environment information;

[0131] The third determination module 307 is used to determine target control decision information of the first vehicle based on the key features of the road environment, and the target control decision information is used to control the first vehicle.

[0132] In a possible implementation, the fusion module 305 is used to delete part of the third environmental information from the plurality of third environmental information based on the characteristic information of the plurality of target sensors to obtain a plurality of second target environmental information, wherein the deleted third environmental information includes erroneous environmental information and / or redundant environmental information; based on the weight of the target sensor to which the second target environmental information belongs, perform weighted averaging on the plurality of second target environmental information to obtain the first target environmental information of the current road environment of the first vehicle; or

[0133] The fusion module 305 is used to input the plurality of third environmental information into an information fusion model, and output the first target environmental information of the current road environment of the first vehicle, wherein the information fusion model is used to fuse the plurality of third environmental information based on the characteristic information of the plurality of target sensors; or

[0134] The fusion module 305 is used to establish a state space model of the multiple target sensors based on the characteristic information of the multiple target sensors, the state space model includes a dynamic equation and an observation equation, the dynamic equation is used to predict the environmental information at the next moment, and the observation equation is used to determine the gain information of the sensor; taking the multiple third environmental information as prior information, predicting the fourth environmental information at the next moment through the dynamic equation based on the prior information, determining the prediction error of the dynamic equation based on the fourth environmental information and the multiple third environmental information, and determining the fourth environmental information as the first target environmental information when the prediction error is less than a preset error; determining the gain information of the multiple target sensors based on the prediction error and the observation equation when the prediction error is not less than a preset error, correcting the fourth environmental information based on the gain information to obtain the fifth environmental information, taking the fifth environmental information as the prior information, and then executing the step of predicting the multiple fourth environmental information at the next moment through the dynamic equation based on the prior information until the first target environmental information is determined.

[0135] In another possible implementation, the preprocessing module 303 is used for, when the first environmental information is a first environmental image and the second environmental information is a second environmental image, performing defogging on the first environmental image to obtain the second environmental image, wherein the contrast of the second environmental image is higher than the contrast of the first environmental image; or

[0136] The preprocessing module 303 is used when the first environmental information is first point cloud data and the second environmental information is second point cloud data. The second point cloud data is obtained by performing point cloud hierarchical filtering on the first point cloud data to remove raindrop noise in the first point cloud data.

[0137] In another possible implementation, the third determination module 307 is used to perform target detection based on key features of the road environment to obtain target objects included in the current road environment of the first vehicle; track the target objects to obtain operating information of the target objects; classify the target objects to obtain category information and attribute information of the target objects; and determine the target control decision information of the first vehicle based on the operating information, category information and attribute information of the target objects.

[0138] In another possible implementation, the device further includes:

[0139] a fourth determination module, configured to determine optimization information based on historical driving records of the plurality of second vehicles, the optimization information comprising at least one of parameter optimization information of the plurality of target sensors, first model optimization information of the information extraction model, algorithm optimization information of the fusion algorithm, and second model optimization information of the environment perception model;

[0140] An optimization module is used to optimize the parameters of the multiple target sensors based on the parameter optimization information; and / or, to optimize the information extraction model based on the first model optimization information; and / or, to optimize the fusion algorithm based on the algorithm optimization information; and / or, to optimize the environmental perception model based on the second model optimization information.

[0141] In another possible implementation, the device further includes:

[0142] A fifth determination module is used to determine, based on the weather information, a plurality of target sensors that match the weather conditions indicated by the weather information from a plurality of sensors installed on the first vehicle, wherein the plurality of target sensors include at least two types of sensors among meteorological sensors, optical sensors and microwave sensors.

[0143] In the embodiment of the present application, since rainy and foggy weather often leads to reduced visibility, the traditional single sensor is greatly limited in acquiring environmental information; therefore, multiple target sensors are used to perceive the environment, thereby utilizing the complementarity of multiple target sensors to improve the accuracy and reliability of environmental perception, thereby being able to provide early warning of potential dangers to the first vehicle, helping the first vehicle to perform safe avoidance and emergency handling, thereby significantly improving driving safety.

[0144] It should be noted that the vehicle control device provided in the above embodiment only uses the division of the above functional modules as an example when performing vehicle control. In actual applications, the above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the vehicle controller is divided into different functional modules to complete all or part of the functions described above. In addition, the vehicle control device provided in the above embodiment and the vehicle control method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0145] Figure 4 4 is a structural block diagram of a vehicle controller provided in an embodiment of the present application. Generally, the vehicle controller 400 includes: a main control module 401, a CAN interface 402, a hard-wire input interface 403, and a hard-wire output interface 404. The main control module 401 is connected to the CAN interface 402, the hard-wire input interface 403, and the hard-wire output interface 404, respectively.

[0146] The main control module 401 generally includes a processor and a memory. Among them, the processor may include one or more processing cores, such as a 4-core processor, a 4-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the vehicle display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning. The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory is used to store at least one computer program, and the at least one computer program is used to be executed by the processor to implement the vehicle control method provided by the method embodiment of the present application.

[0147] The CAN interface 402 may include a power CAN interface, a motor CAN interface, and a diagnostic CAN interface, wherein the power CAN interface is used to communicate with a power system module of the vehicle, the motor CAN interface is used to communicate with a motor controller of the vehicle, and the diagnostic CAN interface is used to communicate with a diagnostic device.

[0148] The hard-wire input interface 403 is used to receive hard-wire control signals. The hard-wire output interface 404 is used to send control instructions to the electronic control components of the vehicle so that the electronic control components of the vehicle perform corresponding actions. The electronic control components of the vehicle include a power management system, a motor controller, an on-board charger, a body control system, etc.

[0149] The main control module 401 can communicate with the vehicle's power system module, motor controller and diagnostic equipment through the CAN interface 402, and generate control instructions based on the hard-wired control signal received by the hard-wired input interface 403 to send control instructions to the vehicle's electronic control components through the hard-wired output interface 404.

[0150] Those skilled in the art will understand that Figure 4 The structure shown in the figure does not constitute a limitation on the vehicle controller 400, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0151] The embodiment of the present application also provides a computer-readable storage medium, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to implement the vehicle control method described in any of the above implementations. Optionally, the storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, and an optical data storage device.

[0152] An embodiment of the present application also provides a computer program product, which stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the vehicle control method shown in the above embodiments.

[0153] In some embodiments, the computer program product involved in the embodiments of the present application may be deployed and executed on a vehicle controller, or on multiple vehicle controllers located at one location, or on multiple vehicle controllers distributed at multiple locations and interconnected through a communication network. Multiple vehicle controllers distributed at multiple locations and interconnected through a communication network can form a blockchain system.

[0154] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0155] The above description is only for the purpose of facilitating those skilled in the art to understand the technical solution of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle control method, characterized in that: The method comprises: Determining current weather information of the first vehicle, the weather information being used to indicate current weather conditions; Collecting data on the current road environment of the first vehicle through multiple target sensors installed on the first vehicle to obtain multiple first environmental information, where different target sensors are used to collect environmental information of different dimensions; Preprocessing the plurality of first environmental information to obtain a plurality of second environmental information, wherein the preprocessing includes at least one of filtering processing, denoising processing and data synchronization processing; Based on the weather information, extract key information from the plurality of second environmental information through an information extraction model to obtain a plurality of third environmental information, and for any second environmental information, the third environmental information obtained by extracting key information from the second environmental information is valid under the weather conditions indicated by the weather information; Based on the characteristic information of the multiple target sensors, the multiple third environment information are fused through a fusion algorithm to obtain first target environment information of the current road environment of the first vehicle; Based on the first target environment information, determining key features of the road environment through an environment perception model; Based on the key features of the road environment, target control decision information of the first vehicle is determined, and the target control decision information is used to control the first vehicle.

2. The method according to claim 1, characterized in that The step of fusing the plurality of third environment information based on the characteristic information of the plurality of target sensors through a fusion algorithm to obtain first target environment information of the current road environment of the first vehicle includes: Based on the characteristic information of the multiple target sensors, a part of the third environment information is deleted from the multiple third environment information to obtain multiple second target environment information, where the deleted third environment information includes erroneous environment information and / or redundant environment information; based on the weight of the target sensor to which the second target environment information belongs, a weighted average is performed on the multiple second target environment information to obtain the first target environment information of the current road environment of the first vehicle; or Input the plurality of third environmental information into an information fusion model, and output first target environmental information of the current road environment of the first vehicle, wherein the information fusion model is used to fuse the plurality of third environmental information based on characteristic information of the plurality of target sensors; or Based on the characteristic information of the multiple target sensors, a state space model of the multiple target sensors is established, the state space model includes a dynamic equation and an observation equation, the dynamic equation is used to predict the environmental information at the next moment, and the observation equation is used to determine the gain information of the sensor; the multiple third environmental information is used as prior information, and the fourth environmental information at the next moment is predicted through the dynamic equation based on the prior information, and the prediction error of the dynamic equation is determined based on the fourth environmental information and the multiple third environmental information. When the prediction error is less than a preset error, the fourth environmental information is determined as the first target environmental information; when the prediction error is not less than a preset error, the gain information of the multiple target sensors is determined based on the prediction error and the observation equation, and the fourth environmental information is corrected based on the gain information to obtain the fifth environmental information, and the fifth environmental information is used as the prior information, and then the step of predicting the multiple fourth environmental information at the next moment through the dynamic equation based on the prior information is performed until the first target environmental information is determined.

3. The method according to claim 1, characterized in that The process of filtering the first environment information to obtain the second environment information includes: The first environmental information is a first environmental image, and the second environmental information is a second environmental image, then the first environmental image is defogged to obtain the second environmental image, and the contrast of the second environmental image is higher than the contrast of the first environmental image; or The first environmental information is first point cloud data, and the second environmental information is second point cloud data. The second point cloud data is obtained by performing point cloud hierarchical filtering on the first point cloud data and removing raindrop noise in the first point cloud data.

4. The method according to claim 1, characterized in that: The determining, based on the key features of the road environment, target control decision information of the first vehicle includes: Performing target detection based on key features of the road environment to obtain a target object included in the current road environment of the first vehicle; Tracking the target object to obtain operation information of the target object; Classifying the target object to obtain category information and attribute information of the target object; Based on the operation information, category information and attribute information of the target object, target control decision information of the first vehicle is determined.

5. The method according to claim 1, characterized in that The method further comprises: Determine optimization information based on historical driving records of the plurality of second vehicles, the optimization information comprising at least one of parameter optimization information of the plurality of target sensors, first model optimization information of the information extraction model, algorithm optimization information of the fusion algorithm, and second model optimization information of the environment perception model; Based on the parameter optimization information, optimizing the parameters of the multiple target sensors; and / or, optimizing the information extraction model based on the first model optimization information; and / or, Based on the algorithm optimization information, optimizing the fusion algorithm; and / or, Based on the second model optimization information, the environmental perception model is optimized.

6. The method according to claim 1, characterized in that The method further comprises: Based on the weather information, a plurality of target sensors matching the weather conditions indicated by the weather information are determined from a plurality of sensors installed on the first vehicle, the plurality of target sensors including at least two types of sensors among meteorological sensors, optical sensors and microwave sensors.

7. A vehicle control device, characterized in that: The device comprises: A first determining module, used to determine current weather information of the first vehicle, wherein the weather information is used to indicate current weather conditions; A collection module, used to collect data about the current road environment of the first vehicle through multiple target sensors installed on the first vehicle to obtain multiple first environmental information, where different target sensors are used to collect environmental information of different dimensions; A preprocessing module, used for preprocessing the plurality of first environment information to obtain a plurality of second environment information, wherein the preprocessing includes at least one of filtering processing, denoising processing and data synchronization processing; an extraction module, configured to extract key information from the plurality of second environmental information by using an information extraction model based on the weather information to obtain a plurality of third environmental information, wherein for any second environmental information, the third environmental information obtained by extracting key information from the second environmental information is valid under the weather condition indicated by the weather information; a fusion module, configured to fuse the plurality of third environmental information through a fusion algorithm based on the characteristic information of the plurality of target sensors to obtain first target environmental information of the current road environment of the first vehicle; A second determination module is used to determine the key features of the road environment through an environment perception model based on the first target environment information; The third determination module is used to determine target control decision information of the first vehicle based on key features of the road environment, and the target control decision information is used to control the first vehicle.

8. A vehicle controller, characterized in that: The vehicle controller includes a main control module, which includes a processor and a memory. The memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the vehicle control method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the vehicle control method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The product stores at least one program code, and the at least one program code is used to be executed by a processor to implement the vehicle control method as described in any one of claims 1 to 6.

Images