Control method, device, storage medium and program product of vehicle

Abstract

The present application relates to a kind of control method, device, storage medium and program product of vehicle, the control method includes: determining the trajectory information of mobile object on the current driving road of vehicle and the position information of waterlogged area;Based on the position information of waterlogged area, the first radar data of waterlogged area collected by first radar installed on vehicle is obtained, and the second radar data of waterlogged area collected by second radar;The type of first radar and second radar is different;Based on the first radar data of waterlogged area and the second radar data of waterlogged area, the depth information of waterlogged area is determined;In the case where it is determined that vehicle can safely pass through waterlogged area based on the depth information of waterlogged area, based on the driving speed of vehicle and the trajectory information of mobile object, the risk level that mobile object exists splash risk when passing through road is determined;Based on the risk level of splash risk, the control strategy of vehicle is determined.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobiles, in particular to a vehicle control method and device, a storage medium and a program product. BACKGROUND

[0002] Due to the changeable weather, especially the road surface water problem caused by rainfall, on the one hand, the vehicle drives into the deep water accumulation area, which causes the engine to be flooded, the electrical system to fail and other serious problems, on the other hand, the splashing water when the vehicle passes through the water accumulation road may affect the moving objects on the road, such as pedestrians, cyclists, etc. In the related art, a single type of sensor is usually used to detect the information of the water accumulation area. However, due to the limitation of the single sensor, the accuracy of the information of the water accumulation area obtained is problematic, and it is difficult to effectively protect the moving objects to safely pass through the splashing water area when the vehicle can safely pass through the water accumulation area. SUMMARY

[0003] One of the purposes of the present application is to provide a vehicle control method and device, a storage medium and a program product, so as to effectively protect the moving objects to safely pass through the splashing water area when the vehicle can safely pass through the water accumulation area.

[0004] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0005] A vehicle control method, the control method comprising: determining the trajectory information of the moving object on the current driving road of the vehicle and the position information of the water accumulation area; based on the position information of the water accumulation area, acquiring the first radar data of the water accumulation area collected by the first radar installed on the vehicle, and the second radar data of the water accumulation area collected by the second radar; the types of the first radar and the second radar are different; based on the first radar data of the water accumulation area and the second radar data of the water accumulation area, determining the depth information of the water accumulation area; in the case that the vehicle can safely pass through the water accumulation area based on the depth information of the water accumulation area, based on the driving speed of the vehicle and the trajectory information of the moving object, determining the risk level of the moving object when passing through the road in the splashing water risk; based on the risk level of the splashing water risk, determining the control strategy of the vehicle.

[0006] According to the above technical means, first, the data collected by the at least one device installed on the vehicle can complement each other to make up for the shortcomings of a single device, improve the accuracy of the position information of the water accumulation area and the trajectory information of the moving object, then determine the first radar data and the second radar data based on the position information of the water accumulation area with improved accuracy, which improves the accuracy of the calculation of the depth information of the water accumulation area determined based on the first radar data and the second radar data, and finally, based on the depth information of the water accumulation area with improved accuracy, determine the risk level of the moving object when passing through the road and the corresponding vehicle control strategy. The moving object can pass through the water accumulation area safely, effectively protecting the safety of the moving object passing through the water accumulation area, improving the level of road traffic safety, and reducing the occurrence of road accidents.

[0007] In some embodiments, the control method comprises: in the case that it is determined based on the depth information of the water accumulation area that the vehicle cannot safely pass through the water accumulation area, determining the control strategy of the vehicle as: outputting warning information through the display module of the vehicle and / or controlling the vehicle to brake.

[0008] According to the above technical means, based on the depth information of the water accumulation area with improved accuracy, it is determined that the vehicle cannot safely pass through the water accumulation area, so that in the case of deep water accumulation area, emergency warning prompts or stopping measures can be taken to solve the serious problems of engine water ingress and electrical system failure when the vehicle passes through the water accumulation area, and the life safety of the occupants in the vehicle is protected.

[0009] In some embodiments, based on the first radar data of the water accumulation area and the second radar data of the water accumulation area, the depth information of the water accumulation area is determined, comprising: determining the first depth information of the water accumulation area based on the first radar data; determining the second depth information of the water accumulation area based on the second radar data; and determining the depth information of the water accumulation area based on the first depth information and the second depth information.

[0010] According to the above technical means, the first radar data and the second radar data determined by the data collected by the first radar and the second radar installed on the vehicle can complement each other to make up for the shortcomings of a single radar device, so that the depth information of the water accumulation area determined based on the first radar data and the second radar data improves the calculation accuracy of the depth information of the water accumulation area.

[0011] In some embodiments, based on the driving speed of the vehicle and the trajectory information of the moving object, determining a risk level of the moving object having a splashing risk when passing through the road comprises: determining a splashing area triggered by the vehicle when passing through the water accumulation area based on the depth of the water accumulation area and the driving speed of the vehicle; determining a first time when the moving object enters the splashing area based on the trajectory information of the moving object and the splashing area; determining a second time when the vehicle enters the splashing area based on the driving speed of the vehicle and the splashing area; and determining the risk level of the moving object having a splashing risk when passing through the road based on the first time and the second time.

[0012] According to the above technical means, based on the improved depth information of the water accumulation area, the calculation accuracy of the splashing area of the vehicle passing through the water accumulation area can be improved, and thus the risk level of the moving object having a splashing risk when passing through the road is determined based on the improved splashing area, thereby improving the accuracy of the determination of the risk level and effectively protecting the moving object from passing through the splashing area safely.

[0013] In some embodiments, based on the first time and the second time, determining the risk level of the moving object having a splashing risk when passing through the road comprises: determining that the risk level of the moving object having a splashing risk when passing through the road is high risk based on the first time, the second time and a first time threshold; determining that the risk level of the moving object having a splashing risk when passing through the road is medium risk based on the first time, the second time and a second time threshold; the second time threshold is greater than the first time threshold; determining that the risk level of the moving object having a splashing risk when passing through the road is low risk based on the first time, the second time and a third time threshold; and the third time threshold is greater than or equal to the second time threshold.

[0014] According to the above technical means, the relationship between the first time and the second time under different risk levels is explained, so that the risk level of the moving object having a splashing risk when passing through the water accumulation area can be determined based on the actual first time and the second time, and thus the control strategy corresponding to the risk level can be determined to effectively protect the moving object from passing through the splashing area safely.

[0015] In some embodiments, based on the risk level of the splashing risk, the control strategy of the vehicle is determined, including: in the case that the risk level of the splashing risk is high risk, the control strategy of the vehicle is determined as: issuing a warning information to the driver of the vehicle through the display module and / or the voice module of the vehicle to remind the driver to slow down or brake; in the case that the risk level of the splashing risk is medium risk, the control strategy of the vehicle is determined as: issuing a warning information to the driver of the vehicle through the display module and / or the voice module of the vehicle to remind the driver to slow down; in the case that the risk level of the splashing risk is low risk, the control strategy of the vehicle is determined as: controlling the vehicle not to issue a warning information, so that the vehicle travels according to the current driving state.

[0016] According to the above technical means, the control strategy of the vehicle under different risk levels is explained, so that the control strategy corresponding to the risk level can be determined based on the risk level of the splashing risk of the moving object corresponding to the water accumulation area on the actual driving road of the vehicle, so as to effectively protect the moving object to pass through the water accumulation area safely.

[0017] In some embodiments, the driving mode includes automatic driving, and based on the risk level of the splashing risk, the control strategy of the vehicle is determined, including: in the case that the risk level of the splashing risk is high risk and the driving mode is automatic driving, the control strategy of the vehicle is determined as: controlling the vehicle to perform emergency braking or emergency deceleration operation; in the case that the risk level of the splashing risk is medium risk and the driving mode is automatic driving, the control strategy of the vehicle is determined as: controlling the vehicle to perform gentle deceleration operation; in the case that the risk level of the splashing risk is low risk and the driving mode is automatic driving, the control strategy of the vehicle is determined as: controlling the vehicle to maintain the current vehicle speed.

[0018] According to the above technical means, the control strategy of the vehicle under different risk levels is explained, so that the control strategy corresponding to the risk level can be determined based on the risk level of the splashing risk of the moving object corresponding to the water accumulation area on the actual driving road of the vehicle, so as to effectively protect the moving object to pass through the water accumulation area safely.

[0019] In some embodiments, the position information of the water accumulation area on the current driving road of the vehicle is determined, including: based on the image data collected by the plurality of cameras, first feature information in each image data under the bird's eye view of the road is determined; based on the third radar data collected by the third radar installed on the vehicle, second feature information in the third radar data under the bird's eye view is determined; the types of the first radar, the second radar and the third radar are different; based on the first feature information and the second feature information, the position information of the water accumulation area on the current driving road of the vehicle is determined.

[0020] According to the above technical means, first, the position information of the water accumulation area determined by the third radar and the data collected by the at least one camera installed on the vehicle can complement each other to improve the accuracy of calculating the position information of the water accumulation area.

[0021] In some embodiments, the control method comprises: determining position information of a water accumulation area based on third radar data; correcting the position information of the water accumulation area based on radar data collected by the first radar, radar data collected by the second radar and the third radar data; wherein the position information of the water accumulation area is based on the first radar data collected by the first radar installed on the vehicle and the second radar data collected by the second radar, and the first radar data collected by the first radar installed on the vehicle and the second radar data collected by the second radar are obtained based on the corrected position information of the water accumulation area.

[0022] According to the above technical means, first, the position information of the water accumulation area determined by the third radar, the first radar and the second radar can complement each other to improve the accuracy of calculating the position information of the water accumulation area.

[0023] A control device of a vehicle, the vehicle being provided with a first radar and a second radar; the control device comprising: a first acquisition module for acquiring radar data collected by the first radar installed on the vehicle and radar data collected by the second radar installed on the vehicle; a first determination module for determining trajectory information of a moving object on a current driving road of the vehicle and position information of a water accumulation area; a second acquisition module for acquiring first radar data of the water accumulation area collected by the first radar based on the position information of the water accumulation area and the radar data collected by the first radar, and acquiring second radar data of the water accumulation area collected by the second radar based on the position information of the water accumulation area and the radar data collected by the second radar; the first radar and the second radar are different in type; a second determination module for determining depth information of the water accumulation area based on the first radar data of the water accumulation area and the second radar data of the water accumulation area; a third determination module for determining a risk level of a water splashing risk of the moving object when passing through the road based on the driving speed of the vehicle and the trajectory information of the moving object in the case that the vehicle can safely pass through the water accumulation area based on the depth information of the water accumulation area; and a fourth determination module for determining a control strategy of the vehicle based on the risk level of the water splashing risk.

[0024] A computer readable storage medium having stored thereon a computer program, the computer program being executed by a control device of a vehicle to implement some or all steps of any of the above methods.

[0025] A computer program product comprising computer programs or instructions which, when executed by a control device of a vehicle, implement some or all of the steps of the method of the claims.

[0026] Advantages of the present application:

[0027] (1) The data collected by at least one device installed on the vehicle can complement the shortcomings of a single device, improving the accuracy of the position information of the water accumulation area and the trajectory information of the moving object.

[0028] (2) The present application determines the first radar data and the second radar data based on the position information of the water accumulation area with improved accuracy, thereby improving the accuracy of the calculation of the depth information of the water accumulation area determined based on the first radar data and the second radar data.

[0029] (3) The present application determines the risk level of the moving object when passing through the road based on the depth information of the water accumulation area with improved accuracy, and the corresponding vehicle control strategy, which can effectively protect the moving object to pass through the splash area safely in the case that the vehicle can safely pass through the water accumulation area, improve the level of road traffic safety, and reduce the occurrence of road accidents. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 An implementation flowchart of a vehicle control method provided by the present application is shown in the figure.

[0031] Figure 2 A schematic diagram of a vehicle control system based on multi-sensor fusion provided by the present application is shown in the figure.

[0032] Figure 3 An implementation flowchart of a two-dimensional occupancy grid algorithm provided by the present application is shown in the figure.

[0033] Figure 4 A schematic diagram of the composition structure of a vehicle control device provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0034] The embodiments of the present application will be described below with reference to the accompanying drawings and preferred embodiments, and those skilled in the art can easily understand other advantages and effects of the present application from the disclosure in the specification. The present application can also be implemented or applied by different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustration of the present application, and are not intended to limit the protection scope of the present application.

[0035] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concepts of the present application, and only the components related to the present application are shown in the diagrams, rather than being drawn according to the number, shape and size of the components in actual implementation. The shapes, number and proportions of the components in actual implementation can be arbitrarily changed, and the layout pattern of the components can be more complex.

[0036] The present embodiment provides a control method of a vehicle, as shown in the figure, the method can include steps S101 to S105: Figure 1

[0037] Step S101: determining the trajectory information of the moving object and the position information of the water accumulation area on the current driving road of the vehicle;

[0038] Here, the moving object refers to an object that moves on the road, and its spatial position changes accordingly with time, such as pedestrians, motorcyclists, animals, etc. The trajectory information of the moving object refers to the route information of the moving object to be traveled.

[0039] In some embodiments, the moving object can be a Vulnerable Road User (VRU), wherein the VRU generally refers to those individuals who use vehicles on the road but have less protection, such as pedestrians, cyclists, motorcyclists, electric scooter or skateboard drivers, etc. These groups are more likely to be injured in traffic accidents relative to motor vehicle drivers.

[0040] In some embodiments, the trajectory information of the moving object and the position information of the water accumulation area on the current driving road of the vehicle can be determined by data collected by the device installed on the vehicle.

[0041] Step S102: obtaining the first radar data of the water accumulation area collected by the first radar and the second radar data of the water accumulation area collected by the second radar installed on the vehicle based on the position information of the water accumulation area; the first radar and the second radar are different in type;

[0042] Here, the first radar and the second radar refer to radars that detect the position, speed and direction of objects by emitting and receiving signals, wherein the detection distance of the first radar is shorter, and the detection distance of the second radar is farther than that of the first radar. For example, the first radar can be an ultrasonic radar, and the second radar can be a millimeter wave radar.

[0043] In some embodiments, the collection module sends the radar data collected by the first radar and the second radar installed on the vehicle to the control device of the vehicle, and the control device of the vehicle screens the first radar data of the water accumulation area from the radar data of the first radar, and screens the second radar data of the water accumulation area from the radar data of the second radar.​

[0044] Step S103: determining the depth information of the waterlogging area based on the first radar data of the waterlogging area and the second radar data of the waterlogging area.

[0045] In some embodiments, first, the time difference between the signal reflected by the water surface of the waterlogging area and the signal partially penetrating the water surface can be used to determine the first depth information of the waterlogging area, and the time difference between the signal reflected by the water surface of the waterlogging area and the signal partially penetrating the water surface can be used to determine the second depth information of the waterlogging area; then, the first depth information and the second depth information are weighted to obtain the depth information of the waterlogging area.

[0046] Step S104: in the case that the depth information of the waterlogging area is determined based on the vehicle being able to safely pass through the waterlogging area, based on the driving speed of the vehicle and the trajectory information of the moving object, the risk level of the moving object when passing through the road is determined.

[0047] Here, the risk level is used to represent the degree of the moving object splashing water when the vehicle passes through the waterlogging area, which can include high risk, low risk and medium risk.

[0048] In some embodiments, the safety threshold of the vehicle is obtained, and based on the safety threshold of the vehicle and the depth information of the waterlogging area, it is determined whether the vehicle can safely pass through the waterlogging area, wherein in the case that the safety threshold of the vehicle is greater than the depth information of the waterlogging area, it is determined that the vehicle can safely pass through the waterlogging area. The safety threshold of the vehicle can be determined based on the wading performance of the vehicle and the maximum safe wading depth recommended by the vehicle manufacturer, for example, for ordinary passenger cars, the safety threshold can be 40 cm.

[0049] In some embodiments, first, based on the driving speed of the vehicle and the depth information of the waterlogging area, the splashing water area of the vehicle when passing through the waterlogging area can be determined; then, in the case that the trajectory information of the moving object represents that the moving object will pass through the splashing water area, the first time when the moving object enters the splashing water area and the second time when the vehicle enters the splashing water area are determined respectively; finally, based on the first time and the second time, the risk level of the moving object when passing through the road is determined.

[0050] Step S105: determining the control strategy of the vehicle based on the risk level of the splashing water risk.

[0051] In this embodiment, firstly, the data collected by the at least one device mounted on the vehicle can complement each other to make up for the shortcomings of a single device, thereby improving the accuracy of the position information of the waterlogged area and the trajectory information of the moving object. Then, the first radar data and the second radar data are determined based on the position information of the waterlogged area with improved accuracy, thereby improving the accuracy of the calculation of the depth information of the waterlogged area determined based on the first radar data and the second radar data. Finally, the risk level of the moving object when passing through the waterlogged area and the corresponding control strategy of the vehicle are determined based on the depth information of the waterlogged area with improved accuracy, so as to effectively protect the moving object from passing through the waterlogged area safely, improve the road traffic safety level, and reduce the occurrence of road accidents.

[0052] In some embodiments, the above steps S104 and S105 can be replaced by step S106:

[0053] Step S106: In the case where it is determined that the vehicle cannot safely pass through the waterlogged area based on the depth information of the waterlogged area, the control strategy of the vehicle is determined as follows: outputting warning information through the display module and / or voice module of the vehicle and / or controlling the vehicle to brake.

[0054] In some embodiments, the safety threshold of the vehicle is obtained, and whether the vehicle can safely pass through the waterlogged area is determined based on the safety threshold of the vehicle and the depth information of the waterlogged area. In the case where the safety threshold of the vehicle is less than the depth information of the waterlogged area, it is determined that the vehicle cannot safely pass through the waterlogged area.

[0055] It should be noted that in the case where the safety threshold of the vehicle is less than the depth information of the waterlogged area, it means that the vehicle will cause serious problems such as water entering the engine and electrical system failure when passing through the waterlogged area, which threatens the life safety of the driver and passengers in the vehicle, so it is necessary to warn the driver of the vehicle.

[0056] In some embodiments, in the case where it is determined that the vehicle cannot safely pass through the waterlogged area based on the depth information of the waterlogged area, the control strategy of the vehicle is determined as follows: outputting warning information to the driver of the vehicle through the display module and / or voice module of the vehicle to remind the driver to brake.

[0057] In some embodiments, in the case where the driving mode of the vehicle is automatic driving and it is determined that the vehicle cannot safely pass through the waterlogged area based on the depth information of the waterlogged area, the control strategy of the vehicle is determined as follows: outputting warning information to the driver of the vehicle through the display module and / or voice module of the vehicle, and controlling the vehicle to brake. The warning information output to the driver of the vehicle through the display module and / or voice module of the vehicle is to make the driver understand the automatic driving strategy. In some embodiments, in the case where the driving mode of the vehicle is automatic driving and it is determined that the vehicle cannot safely pass through the waterlogged area based on the depth information of the waterlogged area, the control strategy of the vehicle is determined as follows: outputting warning information to the driver of the vehicle through the display module and / or voice module of the vehicle, and controlling the vehicle to brake. The warning information output to the driver of the vehicle through the display module and / or voice module of the vehicle is to make the driver understand the automatic driving strategy.

[0058] The above technical means determines that the vehicle cannot safely pass through the water accumulation area based on the depth information of the water accumulation area with improved accuracy, so that when the water accumulation area is too deep, an emergency warning or stopping measure can be taken to solve the serious problems of engine water ingress, electrical system failure and the like when the vehicle passes through the water accumulation area, thereby protecting the life safety of the occupants in the vehicle.

[0059] In some embodiments, the step S103 of determining the depth information of the water accumulation area based on the first radar data of the water accumulation area and the second radar data of the water accumulation area can include steps S1031-S1033:

[0060] Step S1031: determining first depth information of the water accumulation area based on the first radar data;

[0061] The first depth information is shown in formula (1):

[0062]

[0063] wherein D u represents the first depth information, v represents the propagation speed of sound waves in air, and △t represents the round-trip time difference of the signal.

[0064] Step S1032: determining second depth information of the water accumulation area based on the second radar data;

[0065] The second depth information is shown in formula (2):

[0066]

[0067] wherein D m represents the second depth information, c represents the propagation speed of electromagnetic waves in air, and △t represents the round-trip time difference of the signal.

[0068] Step S1033: determining the depth information of the water accumulation area based on the first depth information and the second depth information.

[0069] The depth information of the water accumulation area is shown in formula (3):

[0070] D fused =(1-w)D m +wD u (3);

[0071] wherein D fused represents the depth information of the water accumulation area, D u represents the first depth information, D m represents the second depth information, and w represents the weight coefficient of the first radar.

[0072] Further, the weight calculation method is shown in formula (4):

[0073]

[0074] wherein d u represents the distance between the first radar and the water accumulation area, and a represents an adjustment parameter for controlling the influence degree of the distance on the weight. The greater the value of a is, the more significant the influence of the distance on the weight is, and the smaller the value of a is, the more slight the influence of the distance on the weight is.

[0075] The above technical means, the first radar data and the second radar data determined by the data collected by the first radar and the second radar installed on the vehicle, can make up for the deficiency of a single radar device, so that the depth information of the water accumulation area determined based on the first radar data and the second radar data improves the calculation accuracy of the depth information of the water accumulation area.

[0076] In some embodiments, the step S104 of determining the risk level of the moving object in the water splash risk when passing through the road based on the driving speed of the vehicle and the trajectory information of the moving object can include steps S1041 to S1044:

[0077] Step S1041: determining a water splash area triggered by the vehicle when passing through the water accumulation area based on the depth of the water accumulation area and the driving speed of the vehicle.

[0078] Here, the water splash area refers to an area where water splashes caused by tire extrusion when the vehicle passes through the water accumulation area.

[0079] In some embodiments, the water splash area can be a fan-shaped area extending in the driving direction of the vehicle, and the calculation method of the water splash area is shown in formula (5):

[0080]

[0081] wherein R splash represents the water splash area corresponding to the water accumulation area, β represents an empirical coefficient, v represents the driving speed of the vehicle, D fused represents the depth information of the water accumulation area, and k represents the friction coefficient between the tire of the vehicle and the road surface.

[0082] Step S1042: determining a first time when the moving object enters the water splash area based on the trajectory information of the moving object and the water splash area.

[0083] In some embodiments, it can be determined whether the trajectory information of the moving object intersects with the water splash area, so as to determine whether the moving object will enter the water splash area, and then the first time when the moving object enters the water splash area is determined based on the moving speed of the moving object and the moving direction of the moving object.

[0084] Step S1043: determining a second time when the vehicle enters the water splashing area based on the driving speed of the vehicle and the water splashing area;

[0085] In some embodiments, first, the driving route and the driving speed of the vehicle are obtained; then, the second time when the vehicle enters the water splashing area can be determined based on the driving route and the driving speed of the vehicle. The driving speed of the vehicle can be detected by a speed sensor installed on the vehicle. The driving route of the vehicle can be determined based on the destination address of the vehicle, or can be predicted by a deep learning algorithm based on the driving direction and the driving speed of the vehicle.

[0086] Step S1044: determining the risk level of the moving object having a water splashing risk when passing through the road based on the first time and the second time.

[0087] Here, the risk level of the moving object having a water splashing risk when passing through the road can be determined by the relationship between the first time and the second time.

[0088] The above technical means can improve the calculation accuracy of the vehicle passing through the water splashing area of the water accumulation area based on the improved depth information of the water accumulation area, thereby determining the risk level of the moving object having a water splashing risk when passing through the road based on the improved water splashing area, which improves the accuracy of determining the risk level and effectively protects the safety of the moving object passing through the water splashing area.

[0089] In some embodiments, the step S1044 of determining the risk level of the moving object having a water splashing risk when passing through the road based on the first time and the second time can include steps S1144 to S1146:

[0090] Step S1144: determining that the risk level of the moving object having a water splashing risk when passing through the road is high risk based on the first time, the second time, and a first time threshold value;

[0091] In some embodiments, when the absolute value of the difference between the first time and the second time is less than the first time threshold value, it can be determined that the vehicle and the moving object enter the water splashing area at the same time, and therefore the risk level of the moving object having a water splashing risk when passing through the road is high risk.

[0092] Step S1145: determining that the risk level of the moving object having a water splashing risk when passing through the road is medium risk based on the first time, the second time, and a second time threshold value; the second time threshold value is greater than the first time threshold value;

[0093] In some embodiments, in a case that the first time is less than the second time, and an absolute value of a difference between the first time and the second time is greater than the first time threshold and less than the second time threshold, it can be determined that the vehicle and the moving object do not enter the splash zone at the same time but the time difference is close, and thus the risk level of the splash risk of the moving object when passing the road is a medium risk.

[0094] Step S1146: determining, based on the first time, the second time, and the third time threshold, that the risk level of the splash risk of the moving object when passing the road is a low risk; the third time threshold is greater than or equal to the second time threshold.

[0095] In some embodiments, in a case that the absolute value of the difference between the first time and the second time is greater than the third time threshold, it can be determined that the risk level of the splash risk of the moving object when passing the road is a low risk.

[0096] In some other embodiments, in a case that the state information of the moving object can determine that the moving object has obvious avoidance to the splash zone, it is determined that the risk level of the splash risk of the moving object when passing the road is a low risk.

[0097] The above technical means illustrates the relationship between the first time and the second time under different risk levels, so that the risk level of the splash risk of the moving object corresponding to the splash zone when the vehicle passes the splash zone can be determined based on the actual first time and the second time, and thus the control strategy corresponding to the risk level can be determined to effectively protect the moving object to pass the splash zone safely.

[0098] In some embodiments, the step S105 of determining the control strategy of the vehicle based on the risk level of the splash risk can include steps S1051 to S1053:

[0099] Step S1051: in a case that the risk level of the splash risk is a high risk, determining that the control strategy of the vehicle is to send a warning information to the driver of the vehicle through a display module and / or a voice module of the vehicle to remind the driver to perform a speed reduction or brake operation.

[0100] Here, the display module can be a bridge for information interaction between the user and the vehicle, used to display various information of the vehicle, and provide necessary driving reference and vehicle state feedback for the driver, for example, the display module can be a vehicle display or a head-up display (HUD).

[0101] In some embodiments, the warning information can be sent to the driver through the display module of the vehicle, or the warning information can be sent to the driver through the voice module of the vehicle, or the warning information can be sent to the driver through the display module and the voice module of the vehicle.

[0102] In some embodiments, the display module can also display the position of the moving object, the motion trajectory of the moving object, the information of the water accumulation area, and the information of the water splash area, etc. in real time, so that the driver can understand the current water splash risk, wherein the information of the water accumulation area can include the position information of the water accumulation area, the depth information of the water accumulation area, and the depth information of the water accumulation area, etc.

[0103] It should be noted that in the case of high risk of water splash risk, it is indicated that water will soon splash on the moving object, and the driver needs to take emergency avoidance measures, therefore, the driver should take emergency braking or emergency deceleration measures to avoid water splash on the moving object upon receiving the warning information.

[0104] Step S1052: In the case of medium risk of water splash risk, the control strategy of the vehicle is determined as: issuing a warning information to the driver of the vehicle through the display module and / or the voice module of the vehicle to remind the driver to decelerate;

[0105] It should be noted that in the case of medium risk of water splash risk, if the vehicle maintains the current driving state, water will splash on the moving object at a future time, and the driver needs to take avoidance measures, therefore, the driver should take gentle deceleration measures to avoid water splash on the moving object at a future time upon receiving the warning information.

[0106] Step S1053: In the case of low risk of water splash risk, the control strategy of the vehicle is determined as: controlling the vehicle not to issue a warning information, so that the vehicle can drive according to the current driving state.

[0107] Here, the driving state refers to the specific condition or mode of the vehicle when driving on the road, which can be classified and described according to multiple dimensions such as driving power, driving speed, driving direction, driving stability, etc.

[0108] In some embodiments, in the case of low risk of water splash risk, no warning information needs to be issued to the driver, so that the vehicle can continue to drive in the current driving state.

[0109] In other embodiments, although the moving object has a low risk of water splash risk, there may still be some potential water splash areas, in which case the display module can display the potential water splash area. It should be noted that in this risk level, the moving object has a low risk of water splash when passing through the potential water splash area, however, displaying the potential water splash area through the display module can enable the driver to dynamically adjust the driving state of the vehicle at any time according to the displayed information.

[0110] The above technical means illustrates the control strategy of the vehicle under different risk levels, so that the control strategy corresponding to the risk level of the splash risk of the moving object on the water accumulation area on the actual driving road of the vehicle can be determined to effectively protect the moving object from passing through the splash area.

[0111] In some embodiments, the driving mode includes automatic driving, and the step S105 of determining the control strategy of the vehicle based on the risk level of the splash risk can include steps S1054-S1056:

[0112] Step S1054: in the case that the risk level of the splash risk is high risk and the driving mode is automatic driving, the control strategy of the vehicle is determined as controlling the vehicle to perform emergency braking or emergency deceleration operation.

[0113] Here, automatic driving refers to that the vehicle can autonomously perform driving tasks without direct operation of a human driver. When the vehicle is in automatic driving, the vehicle can perceive the surrounding environment through vehicle-mounted sensors, and control the steering and speed of the vehicle according to the road, vehicle position and obstacle information obtained by perception, so that the vehicle can safely and reliably drive on the road.

[0114] In some embodiments, in the case that the driving mode of the vehicle is automatic driving, the vehicle display module and / or the voice module can also issue warning information to the driver of the vehicle to make the driver understand the current control strategy of automatic driving.

[0115] It should be noted that in the case that the risk level of the splash risk is high risk, it means that the water will soon splash onto the moving object, and emergency avoidance measures need to be taken, so the vehicle should be controlled to take emergency braking or emergency deceleration measures to avoid splashing onto the moving object.

[0116] Step S1055: in the case that the risk level of the splash risk is medium risk and the driving mode is automatic driving, the control strategy of the vehicle is determined as controlling the vehicle to perform gentle deceleration operation.

[0117] In some embodiments, in the case that the driving mode of the vehicle is automatic driving, the vehicle display module and / or the voice module can also issue warning information to the driver of the vehicle to make the driver understand the current control strategy of automatic driving.

[0118] It should be noted that in the case that the risk level of the splash risk is medium risk, if the vehicle maintains the current driving state, the water will splash onto the moving object at a future time, so the vehicle should be controlled to take gentle deceleration measures to avoid splashing onto the moving object at a future time.

[0119] Step S1056: In the case that the risk level of the splashing risk is low risk and the driving mode is automatic driving, the control strategy of the vehicle is determined as: controlling the vehicle to maintain the current vehicle speed.

[0120] It should be noted that in the case that the risk level of the splashing risk is low risk, if the vehicle maintains the current driving state, the probability of splashing water to the moving object in the future time is low, and therefore the vehicle can be controlled to maintain the current vehicle speed.

[0121] The above technical means illustrates the control strategy of the vehicle under different risk levels, so that the control strategy corresponding to the risk level can be determined based on the risk level of the splashing risk of the moving object existing in the water accumulation area on the actual driving road of the vehicle, so as to effectively protect the safety of the moving object passing through the water accumulation area.

[0122] In some embodiments, the step S101 of determining the position information of the water accumulation area on the current driving road of the vehicle can include steps S1011 to S1013:

[0123] Step S1011: determining first feature information in each image data in the bird's-eye view of the road based on the image data collected by the plurality of cameras;

[0124] Here, the plurality of cameras refers to at least one camera, for example, 2, 3, 6 and 7. The bird's-eye view (Bird's-Eye-View, BEV) refers to a method of projecting three-dimensional environmental information to a two-dimensional plane to display objects and terrain in the environment from a top-down perspective.

[0125] In some embodiments, the determination method of the first feature information can include image feature extraction, depth estimation and perspective conversion.

[0126] In some embodiments, the implementation of image feature extraction can be: after the control device of the vehicle obtains the image data collected by the plurality of cameras, the control device uses a deep learning algorithm (such as a convolutional neural network CNN) to extract features from the input multiple image data, so as to extract useful feature information such as edges, textures, colors, etc. from the multiple image data, which will be used for subsequent depth estimation and perspective conversion.

[0127] In some embodiments, the implementation of depth estimation can be: first, a depth network is used to predict the depth information of each pixel point in the multiple image data, wherein the depth information is crucial for converting the image features from the perspective projection view to the bird's-eye view. Then, according to the prediction result of the depth network, a corresponding plurality of depth maps are generated, wherein the depth map is a matrix with the same size as the input multiple image data, and the value of each element represents the depth information of the corresponding pixel point.

[0128] In some embodiments, the implementation of the perspective conversion can be: first, using the feature information of the depth map and the image data, the two-dimensional image features are lifted to the three-dimensional space, wherein this step usually involves combining the features of each pixel point with its corresponding depth information to generate three-dimensional feature points or voxels. Then, the three-dimensional feature points or voxels are projected onto the BEV plane, wherein this step involves converting the points or voxels in the three-dimensional space to the two-dimensional BEV plane while retaining their spatial position information. Finally, the projected features are aggregated on the BEV plane, wherein since one ground position may correspond to feature points in multiple camera perspectives, these feature points need to be fused or averaged, etc. to generate the final first feature information.

[0129] Step S1012: determining second feature information in the third radar data in the bird's eye view based on third radar data collected by a third radar installed on the vehicle; the types of the first radar, the second radar and the third radar are different;

[0130] Here, the third radar refers to a radar that can obtain information about a target by transmitting a detection signal to the target, then comparing the received signal reflected from the target with the transmitted signal, and after appropriate processing, the information about the target is obtained. For example, the third radar can be a laser radar.

[0131] In some embodiments, the method for determining the second feature information can include data preprocessing and feature extraction.

[0132] In some embodiments, the implementation of data preprocessing can be: after the control device of the vehicle obtains the third radar data, the third radar data is preprocessed, including steps such as removing noise, filtering, downsampling, etc. to improve the quality and processing efficiency of the third radar data.

[0133] In some embodiments, the implementation of feature extraction can be: first, projecting the preprocessed third radar data onto the XY plane to generate a basic image of radar BEV features, wherein during the data projection process, a suitable resolution needs to be selected, i.e. determining how large a range in the point cloud space corresponds to a pixel point in the image. Then, on the BEV image obtained by projection, according to the specific application requirements, a suitable feature representation method is selected or designed, wherein common features include maximum height value, average height value, point cloud density, reflection intensity, etc. These features can reflect the shape, size and position of the target object, etc. Finally, using a specific algorithm (such as a convolutional neural network, a deep learning model, etc.), the features are extracted from the BEV image to obtain the second feature information.

[0134] Step S1013: determining the position information of the water accumulation area on the current driving road of the vehicle based on the first feature information and the second feature information.

[0135] In some embodiments, first, the first feature information and the second feature information are fused, wherein the fusion process can be point-level fusion or feature-level fusion; then, the fused feature information is subjected to three-dimensional target detection to identify road, vehicle, pedestrian, and water accumulation area target objects; finally, based on the water accumulation area in the identified target objects, the position information of the water accumulation area on the current driving road of the vehicle can be determined.

[0136] The above technical means, first, the position information of the water accumulation area determined by the data collected by the third radar and the at least one camera installed on the vehicle can make up for the shortcomings of a single device and improve the accuracy of calculating the position information of the water accumulation area.

[0137] In some embodiments, after the above step S1013, steps S1014 and S1015 can be included:

[0138] Step S1014: determining the position information of the water accumulation area based on the third radar data;

[0139] Step S1015: correcting the position information of the water accumulation area based on the radar data collected by the first radar, the radar data collected by the second radar, and the third radar data;

[0140] In step S102, the first radar data of the water accumulation area collected by the first radar installed on the vehicle and the second radar data of the water accumulation area collected by the second radar are obtained based on the position information of the water accumulation area, including: based on the corrected position information of the water accumulation area, the first radar data of the water accumulation area collected by the first radar installed on the vehicle and the second radar data of the water accumulation area collected by the second radar are obtained.

[0141] In some embodiments, based on the height information and the reflection intensity information of the radar data of the third radar, the position information and the range information of the water accumulation area are preliminarily judged.

[0142] In some embodiments, first, the first position information of the water accumulation area can be determined based on the radar data collected by the first radar, the second position information of the water accumulation area can be determined based on the radar data collected by the second radar, and the third position information of the water accumulation area can be determined based on the third radar data; then, the position information of the water accumulation area is corrected based on the first position information, the second position information, and the third position information.

[0143] In some embodiments, the implementation of determining the first position information of the water accumulation area based on the radar data collected by the first radar can be: first, data preprocessing is performed on the radar data collected by the first radar; then, the distance value of each measurement point in the preprocessed radar data collected by the first radar is determined; finally, by analyzing the change trend of the distance value of each measurement point, the position of the water accumulation area can be preliminarily judged, for example, if the distance value of a certain area suddenly decreases, it may mean that there is water accumulation in the area.

[0144] In some embodiments, the implementation of determining the second position information of the water accumulation area based on the radar data collected by the second radar can be: first, data preprocessing is performed on the radar data collected by the second radar; then, the distance value of each measurement point in the preprocessed radar data collected by the second radar is determined; finally, by analyzing the change trend of the distance value of each measurement point, the position of the water accumulation area can be preliminarily judged, for example, if the distance value of a certain area suddenly decreases, it may mean that there is water accumulation in the area.

[0145] In some embodiments, the implementation of determining the third position information of the water accumulation area based on the third radar data can be: first, data preprocessing is performed on the third radar data; then, the height information and the reflection intensity information of the preprocessed third radar data are determined; finally, based on the height information and the reflection intensity information of the third radar data, the position and range of the water accumulation area are preliminarily judged.

[0146] The above technical means, first, the position information of the water accumulation area determined based on the third radar is corrected by the data collected by the third radar, the first radar and the second radar installed on the vehicle, which can make up for the shortcomings of a single device and improve the accuracy of calculating the position information of the water accumulation area.

[0147] In some embodiments, the step S101 of determining the trajectory information of the moving object on the current driving road of the vehicle can include steps S1016 and S1017:

[0148] Step S1016: determining the information of the moving object on the current driving road of the vehicle based on the first feature information and the second feature information;

[0149] In some embodiments, first, the first feature information and the second feature information are fused, and the fusion process can be point-level fusion or feature-level fusion; then, the fused feature information is subjected to three-dimensional target detection to identify target objects such as roads, vehicles, pedestrians and water accumulation areas; finally, based on the moving object in the identified target object, the information of the moving object on the current driving road of the vehicle can be determined, for example, the information of the moving object includes the position information of the moving object, the moving speed of the moving object and the moving direction of the moving object, etc.

[0150] Step S1017: Based on the information of the moving object and the image data collected by the plurality of cameras, the trajectory information of the moving object on the current driving road of the vehicle is determined.

[0151] In some embodiments, after obtaining the information of the moving object and the image data collected by the plurality of cameras, the trajectory information of the moving object can be predicted by a Kalman filter or an extended Kalman filter (EKF); wherein the trajectory information of the moving object refers to a set of moving positions of the moving object in the next few seconds.

[0152] In some embodiments, after obtaining the information of the moving object and the image data collected by the plurality of cameras, the trajectory information of the moving object can be predicted by a deep learning trajectory prediction model.

[0153] In some embodiments, first, after detecting the moving object, it can be determined from the storage area of the control device of the vehicle whether the information of the moving trajectory is stored; then, after determining that the information of the moving trajectory is stored in the storage area, the historical moving trajectory information is obtained; finally, based on the historical moving trajectory information, the current information of the moving object and the image data collected by the plurality of cameras.

[0154] The above technical means, first, the trajectory information of the moving object determined by the data collected by the third radar and at least one camera installed on the vehicle can make up for the shortcomings of a single device, and improve the accuracy of calculating the trajectory information of the moving object.

[0155] The above vehicle control method and system will be described below in combination with a specific embodiment. In order to facilitate understanding, the following takes ultrasonic radar as the first radar, millimeter wave radar as the second radar, laser radar as the third radar, and VRU as the moving object as an example to introduce the possible process suitable for the embodiments of the present application. However, it should be noted that the specific embodiment is only used to better illustrate the present application and does not constitute an improper limitation on the present application.

[0156] In the rapidly developing field of transportation today, automatic driving is gradually becoming the mainstream trend, and how to ensure the safe driving of automatic driving vehicles in complex road environments has become a problem to be solved.

[0157] Due to the unpredictable weather, especially the road surface water problem caused by rainfall, it poses an overlooked hidden danger to driving safety. Not only does the water affect the vehicle's handling performance, increasing the risk of skidding and loss of control, but it also hides potholes and obstacles on the road, further increasing the likelihood of accidents. On the one hand, the depth and range of water on different road sections vary, and if the vehicle recklessly drives into an area with too deep water, it may cause serious problems such as engine water ingress, electrical system failure, etc., threatening the life safety of the driver and passengers. On the other hand, the splashing water generated when the vehicle passes through the water-covered road may affect VRUs, such as pedestrians and cyclists.

[0158] Existing detection technologies usually rely on a single type of sensor, such as cameras, radars, or ultrasonic sensors, etc. However, these single sensors have obvious limitations, for example, cameras perform poorly in bad weather and poor lighting conditions; radars may be subject to electromagnetic interference, and their detection accuracy for close-range objects is limited; ultrasonic sensors have a narrow detection range. In addition, most existing systems only focus on the safety of the vehicle itself, and lack comprehensive consideration of VRU protection and road water, lack of effective multi-sensor fusion and interactive visualization means, and cannot provide comprehensive and intuitive road conditions to the driver or automatic driving system, making it difficult to make accurate decisions and timely responses.

[0159] Therefore, there is an urgent need for a multi-sensor fusion vehicle control system that can accurately detect VRUs and road water conditions at the same time. On the one hand, if the water area is too deep, an emergency warning or stop can be made; on the other hand, if the water is shallow and can be safely passed through, the VRU is detected, and whether it needs to slow down to pass through is determined according to the distance between the VRU and the vehicle, the water depth, and the current vehicle speed, to protect the VRU from passing through the water area safely, thereby effectively improving road traffic safety and reducing accidents. In addition, through interactive visualization, the current road conditions, water area information, and VRU information can be clearly and intuitively presented to the user to assist the driver in controlling the vehicle's driving state.

[0160] First, introduce a multi-sensor fusion vehicle control system provided by the embodiment, which can efficiently and accurately detect VRUs and road water, and present the detection results to the driver or automatic driving system in real time through an interactive visualization interface. Through this system, the problems raised in the background art can be effectively solved. As shown in Figure 2 The system mainly includes the following modules:

[0161] 1. Multi-sensor module 1:

[0162] Here, the vehicle control system is installed with six cameras 11, laser radar 12, millimeter wave radar 13 and ultrasonic radar 14 around the vehicle, among which the six cameras 11 are used to collect visual images of the surrounding environment, providing high-resolution visual information; the laser radar 12 is used to detect the position of objects on the road through the collected three-dimensional point cloud data; the millimeter wave radar 13 and ultrasonic radar 14 are respectively used for long-distance and short-distance depth measurement, especially suitable for detecting water accumulation areas. The sensors of this module are described in detail as follows:

[0163] Laser radar 12: used to obtain high-resolution three-dimensional point cloud data, identify uneven areas of the road surface such as the outline of water accumulation areas, and accurately detect the shape and position of VRUs.

[0164] Millimeter wave radar 13: millimeter wave radar sensors are installed on the vehicle, which can emit millimeter wave signals forward and receive reflected millimeter wave signals. The millimeter wave signals can penetrate the water surface to estimate the depth of the water pit, so that the medium and long distance water depth can be measured.

[0165] Ultrasonic radar 14: ultrasonic radar sensors are installed on the vehicle, which can emit ultrasonic signals forward and receive reflected ultrasonic signals, so that the short-distance water depth can be measured.

[0166] Camera 11: six cameras are mounted around the vehicle, covering a 360-degree view around the vehicle, used to obtain high-definition visual images, and further improve the accuracy of detection in combination with radar data.

[0167] 2. Data processing and fusion module 2:

[0168] Here, the data processing and fusion module 2 includes a central processing unit (CPU) and a graphics processing unit (GPU), which is used to receive and process data from various sensors, and generate an environment perception model through data fusion technology. The functional modules of the data processing and fusion module 2 include a VRU and water accumulation area position detection module 21, a water accumulation area accurate position and depth identification module 23, and a VRU motion trajectory prediction module 22, which are used to implement the following functions:

[0169] Data preprocessing unit: performs preliminary processing on the data collected by each sensor in the multi-sensor module, including denoising, correction, synchronization and other operations. Through the denoising operation, the noise in the data collected by each sensor can be removed, i.e. those unrelated or random fluctuations that may interfere with the true information of the data; through the correction operation, the data collected by each sensor device can be adjusted to eliminate system errors or biases, making the data closer to the true value; through the synchronization operation, the data collected by each sensor can be kept consistent in time, which is convenient for subsequent analysis.

[0170] Data fusion unit: using algorithms such as Kalman filtering, Bayesian inference, etc., to fuse data from different sensor devices to generate a more accurate environmental model.

[0171] Deep learning algorithm unit: applying deep learning models to identify and classify VRUs and accumulated water based on the fused data, which can update the detection results in real time.

[0172] The functional modules of the data processing and fusion module 2 also include a dangerous area judgment module 24 for judging whether the VRU is in or about to enter a dangerous area of being splashed by water based on the depth information of the accumulated water area, the predicted VRU trajectory information, and the vehicle kinematic model.

[0173] 3. Interactive visualization and warning module 3:

[0174] Display control unit: real-time display of the processed detection results through the vehicle-mounted display or HUD, including VRU position information, VRU trajectory information, accumulated water area depth information, and area information, etc.

[0175] Warning system: when potential danger is detected, the control system of the vehicle triggers warning signals, including sound alarms, visual reminders, etc., and can automatically prompt the driver to take evasive measures.

[0176] Next, a vehicle control method based on multi-sensor fusion provided by the embodiment is introduced. This method ensures accurate detection of VRUs and accumulated water through the cooperative work of multiple sensors, and improves driving safety through dynamic weight adjustment and automatic warning. The specific implementation method of this embodiment mainly includes: multi-sensor data acquisition, data fusion processing, VRU and accumulated water detection and classification, interactive visualization and warning, and system optimization.

[0177] 1. Multi-sensor data acquisition

[0178] Laser radar: used to collect three-dimensional point cloud data, which is used for target detection through multi-modal input with 6-camera images, mainly detecting the position and area of VRUs and accumulated water areas in the BEV plane; and used for position positioning and depth measurement of accumulated water areas through multi-modal input with millimeter wave radar and ultrasonic radar, which can complement each other and improve the accuracy of water pit and depth measurement.

[0179] Millimeter wave radar: used to estimate water depth by detecting the time difference between water surface reflection signals and partially penetrating water signals. This estimation relies on the ability of millimeter wave radar to capture underwater reflection signals, and is usually suitable for shallow water pits.

[0180] Ultrasonic radar: used for short-range detection, detecting the distance between the ultrasonic radar and the water surface, supplementing the shortcomings of other sensors in the short distance.

[0181] Camera: provides a full range of visual images, and works with laser radar to detect VRUs and water accumulation areas.

[0182] 2. Data fusion processing

[0183] Data preprocessing: denoising, synchronization and spatial alignment of multi-sensor data to ensure the accuracy and consistency of multi-sensor data.

[0184] Data fusion: combining the characteristics of each sensor, dynamically adjusting the weight of each sensor according to the distance, and generating a comprehensive environmental perception model.

[0185] 3. Detection and classification of VRUs and water accumulation

[0186] VRU detection: using a two-dimensional occupancy grid algorithm (2D Occupancy Grid Algorithm, 2DOCC) algorithm based on vision + laser radar to achieve accurate identification of VRUs and water accumulation areas and obtain their positions in the BEV plane.

[0187] Here, as shown in Figure 2 , the VRU and water accumulation area position detection module in the vehicle control system obtains rough position information of VRUs and water accumulation areas based on 6 camera image data and laser radar data. The VRU and water accumulation area position detection module uses images from six cameras and three-dimensional point cloud data from laser radar for target detection of multi-modal input, which can roughly determine the position information and range information of VRUs and water accumulation areas in the BEV plane.

[0188] In implementation, 6 camera image data and laser radar data are combined as multi-modal input, and a multi-modal 2DOCC algorithm (Occupancy task mainly focuses on analyzing the spatial occupancy in the environment, i.e., which areas are occupied and which areas are not occupied, to assist the autonomous driving system in decision-making) is used for identification and positioning of VRUs (e.g., pedestrians, cyclists, motorcyclists, etc.), water accumulation areas and drivable areas. A feasible multi-modal 2DOCC algorithm can be the Bevfusion algorithm, and the last 3D target detection Head is replaced by a 2DOCC Head, as shown in Figure 3 , and the final output result is the category of VRUs, water accumulation areas and drivable areas, as well as the positions of VRUs, water accumulation areas and drivable areas in the BEV space.

[0189] As shown in Figure 3As shown, the application method of the model architecture includes the following steps: step S301: image data encoding; here, the 6-camera image data is encoded, the feature information in the 6-camera image is extracted, fused, and the multi-view image features are obtained. Step S302: view change; here, the two-dimensional multi-view image features are mapped to a three-dimensional space to obtain image BEV features. Step S303: radar data encoding; here, the laser radar data is encoded, the corresponding point cloud image is obtained based on the laser radar data, the feature information in the point cloud image is extracted, fused, and the radar BEV features are obtained. Step S304: feature fusion; here, the radar BEV features and the image BEV features are input into the feature fusion module for feature fusion to obtain the fused BEV features. Step S305: 2D OCC Head; here, the fused BEV features are input into the 2D OCC Head module to respectively output the categories of VRU, water accumulation area and drivable area, and the positions of VRU, water accumulation area and drivable area in the BEV space respectively. Among them, the 2D OCC Head module is used to replace the traditional 3D target detection Head with the 2D OCC Head to adapt to the output demand of the occupancy grid map, and to generate the occupancy grid map according to the fused features to identify the categories of VRU, water accumulation area and drivable area, and the positions of VRU, water accumulation area and drivable area in the BEV space respectively.

[0190] Water accumulation detection: laser radar, millimeter wave radar and ultrasonic radar are used to jointly detect water pits and their depths, and the sensor weight is adaptively distributed to achieve accurate measurement.

[0191] Here, as shown in Figure 2 , the water accumulation area accurate position and depth identification module in the vehicle control system obtains the accurate position information and depth information of the water accumulation area based on the rough position and size information of the water accumulation area.

[0192] In implementation, first, the rough position and size information of the water accumulation area sent by the VRU and water accumulation area position detection module is received, and the rough position and size information of the water accumulation area is used as prior information to determine the depth information and accurate position information of the water accumulation area. Then, the depth information and accurate position information of the water accumulation area are determined. The implementation of this process includes the following steps:

[0193] a) Auxiliary role of laser radar

[0194] The laser radar is mainly used to identify the outline and accurate position information of the water accumulation area, and is not directly used for depth measurement. These outline and accurate position information of the water accumulation area are then transmitted to the millimeter wave radar and ultrasonic radar for further depth measurement.

[0195] b) Depth estimation of millimeter wave radar and ultrasonic radar

[0196] Millimeter wave radar measurement: millimeter wave radar is used at medium to long distances, with strong penetration capability, and can estimate the depth of the puddle by detecting the time difference of the reflected signal. The depth information of the water accumulation area measured by the millimeter wave radar (first depth information), see formula (2).

[0197] Ultrasonic radar measurement: ultrasonic radar is suitable for precise measurement at close range, and its principle is similar to that of millimeter wave radar, but its resolution is higher at close range. The depth information of the water accumulation area measured by the ultrasonic wave (second depth information), see formula (1).

[0198] c) Multi-sensor result fusion

[0199] Fusion of depth measurement results of millimeter wave radar and ultrasonic radar to determine the depth information of the water accumulation area, see formula (3).

[0200] Estimation of VRU trajectory information: for each detected VRU, the system uses a Kalman filter or an extended Kalman filter to predict the motion trajectory. By analyzing the historical position and speed information of the VRU, the control system can predict the future position of the VRU. In this way, based on the predicted trajectory information of the VRU, it can be determined whether the VRU will enter the driving path of the vehicle or the water splash danger area.

[0201] Here, as shown in Figure 2 , the judgment of the trajectory information of the VRU is made by the VRU motion trajectory prediction module in the control system of the vehicle.

[0202] In implementation, the trajectory information of the VRU can be predicted based on the current speed and direction of the VRU, the environmental context, the historical motion data of the VRU, through a Kalman filter or an extended Kalman filter or a trajectory prediction model based on deep learning. Among them, the trajectory information prediction result of the VRU is the possible position trajectory of the VRU in the next few seconds.

[0203] Among them, the current speed and direction: the current motion state of the VRU is detected through the camera and the laser radar. Historical motion data: if the control system of the vehicle can obtain the historical motion data of the VRU, it can be used as input to improve the prediction accuracy of the trajectory information of the VRU. Environmental context: combined with the surrounding environment, the possible behavior of the VRU is predicted, such as avoiding or continuing straight.

[0204] It should be noted that the trajectory prediction model also combines the kinematic model of the vehicle, taking into account the current speed, direction and acceleration of the vehicle, to further improve the accuracy of trajectory prediction.

[0205] Dangerous area judgment: According to the size and depth of the water accumulation area, the VRU motion estimation and the vehicle kinematic model, it is judged whether the VRU is in or about to enter the dangerous area of being splashed.

[0206] Here, as shown in Figure 2 , the dangerous area judgment module in the vehicle control system makes a judgment of the dangerous area based on the VRU information, the information of the water accumulation area and the trajectory information of the VRU.

[0207] In implementation, first, the VRU information and the information of the water accumulation area sent by the water accumulation area accurate position and depth identification module, and the trajectory information of the VRU predicted by the VRU motion trajectory prediction module are received. The VRU information can include the running speed of the VRU, the running direction of the VRU and the like; the information of the water accumulation area can include the depth information of the water accumulation area, the position information of the water accumulation area and the like.

[0208] Then, by comparing the depth information of the water accumulation area with a preset safety threshold (the safety threshold D is usually determined according to the water crossing performance of the vehicle and the maximum safety water crossing depth recommended by the manufacturer, for example, for ordinary passenger cars, the safety threshold can be set to 40 centimeters), it is determined whether there is a safety problem when the vehicle passes through the water accumulation area; if the depth information of the water accumulation area is greater than the safety threshold, it is determined that there is a safety problem when the vehicle passes through the water accumulation area, then the vehicle control system will immediately take emergency response measures such as emergency warning prompt or automatic brake stop; if the depth information of the water accumulation area is less than the safety threshold, it is determined that there is no safety problem when the vehicle passes through the water accumulation area.

[0209] Next, in the case where it is determined that there is no safety problem when the vehicle passes through the water accumulation area, based on the predicted trajectory information of the VRU, the depth information of the water accumulation area and the kinematic model of the vehicle itself, it is determined whether the VRU is in or about to be in the splashing area, and the splashing risk of the VRU is evaluated. If it is determined that the VRU has a splashing risk, the driver is prompted through the interactive visualization and early warning module and may take automatic brake or deceleration operation to avoid danger.

[0210] Among them, the splashing area of the water accumulation area refers to the area where the water splashed by the tire extrusion when the vehicle passes through the water at high speed. The size and shape of the splashing area mainly depend on the following factors:

[0211] Vehicle speed (v): The faster the vehicle speed, the wider the range of splashing.

[0212] Water depth (D fused ): The deeper the water, the higher and wider the water splashed.

[0213] Tire-road friction coefficient (k): The friction coefficient affects the height and direction of the splashing water.

[0214] Here, the splashing water area is modeled as a sector expanding in the direction of vehicle travel, whose shape and size are given by equation (5).

[0215] After obtaining the splashing water area and the trajectory information of the VRU, the dangerous area judgment module assesses the VRU splashing water risk, which includes the following steps:

[0216] 1. Time to Collision (TTC) calculation

[0217] Here, TTC is a key parameter for assessing the likelihood of the VRU entering the splashing water area. By predicting the trajectory information of the VRU, the dangerous area judgment module calculates the time t VRU (i.e., the first time) for the VRU to enter the splashing water area and the time t vehicle (i.e., the second time) for the vehicle to reach the same area:

[0218] t VRU : Estimate the time for the VRU to reach the boundary of the splashing water area based on its speed and direction.

[0219] t vehicle : Calculate the time for the vehicle to reach the corresponding position based on its current speed and route.

[0220] 2. Assess the VRU splashing water risk

[0221] Here, the dangerous area judgment module assesses the VRU splashing water risk based on the following criteria:

[0222] High risk: If t VRU ≈ t vehicle , i.e., the VRU and the vehicle almost simultaneously enter the splashing water area, the system determines it as a high-risk situation.

[0223] Medium risk: If t VRU < t vehicle , but both are close, the system determines it as a medium-risk situation.

[0224] Low risk: If there is a significant gap between t VRU and t vehicle , or the VRU has obvious avoidance behavior, the system determines it as a low-risk situation.

[0225] 4. Interactive visualization and warning

[0226] Visualization interface: Real-time display of VRU position, water pit depth, and dangerous area information, and intuitive presentation on the vehicle display or HUD.

[0227] Safety warning: On the one hand, when the VRU is detected in a dangerous area, the vehicle control system will trigger a warning to prompt the driver or automatic braking to slow down to avoid water splashing accidents. On the other hand, if the water accumulation area is too deep, the vehicle control system will make an emergency warning prompt or directly stop the car to avoid serious problems such as engine water ingress, electrical system failure, etc., which threaten the life safety of the driver and passengers.

[0228] Here, as shown in Figure 2 The interactive visualization and warning module in the vehicle control system determines the control measures of the vehicle based on the evaluation results of the VRU water splashing risk.

[0229] In implementation, first, the evaluation results of the VRU water splashing risk sent by the dangerous area judgment module are received.

[0230] Then, according to the evaluation results of the VRU water splashing risk and the preset vehicle control strategy, the control measures of the vehicle are determined, wherein the vehicle control strategy includes the driver prompt and the control strategy under automatic driving control.

[0231] The control strategy of the driver prompt includes: high risk: the system issues an emergency prompt to the driver through the in-vehicle display screen or audio warning, suggesting to slow down or brake. Medium risk: the system prompts the driver to pay attention to the situation ahead and suggests appropriate speed reduction. Low risk: the system can choose not to issue a prompt or only mark the potential water splashing area on the screen.

[0232] The control strategy under automatic driving control includes: high risk: the vehicle control system automatically performs an emergency brake or speed reduction operation to ensure that the vehicle will not splash water to the VRU. Medium risk: the vehicle control system slows down gently to reduce the possibility of water splashing. Low risk: the vehicle control system maintains the current speed and route and continues to monitor. Wherein, in the automatic driving mode, the vehicle control system can directly control the vehicle to operate according to the above control strategy.

[0233] Finally, the system visualizes all the detected information and risk evaluation results to the driver through the in-vehicle display screen. Wherein, the display content on the display screen includes:

[0234] The position of the VRU and the water splashing area: graphically display the position of the VRU and the water splashing area to help the driver understand the current risk.

[0235] Risk level: clearly identify the current risk level.

[0236] Suggested operation: prompt the driver for appropriate operation suggestions such as speed reduction, avoidance, etc.

[0237] 5. System optimization

[0238] Modular design: modular design supporting independent upgrade, flexible configuration of different sensors and algorithms.

[0239] Adaptive adjustment: the vehicle control system can adjust the detection strategy according to environmental conditions to ensure real-time and accuracy in different scenarios.

[0240] Based on the foregoing embodiments, the present embodiment provides a control device for a vehicle, which comprises the modules included therein and the units included in each module, and can be implemented by a processor in the vehicle; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA) and the like.

[0241] Based on the foregoing embodiments, the present embodiment provides a control device for a vehicle, which comprises the modules included therein and the units included in each module, and can be implemented by a processor in the vehicle; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA) and the like. Figure 4 As shown, the control device 400 comprises: a first acquisition module 410, configured to acquire radar data collected by a first radar installed on the vehicle and radar data collected by a second radar installed on the vehicle; a first determination module 420, configured to determine trajectory information of a moving object on a current driving road of the vehicle and position information of a water accumulation area; a second acquisition module 430, configured to acquire first radar data of the water accumulation area collected by the first radar based on the position information of the water accumulation area and the radar data collected by the first radar, and acquire second radar data of the water accumulation area collected by the second radar based on the position information of the water accumulation area and the radar data collected by the second radar; the first radar and the second radar are different in type; a second determination module 440, configured to determine depth information of the water accumulation area based on the first radar data of the water accumulation area and the second radar data of the water accumulation area; a third determination module 450, configured to determine a risk level of a water splashing risk of the moving object when passing through the road based on the driving speed of the vehicle and the trajectory information of the moving object in a case where it is determined based on the depth information of the water accumulation area that the vehicle can safely pass through the water accumulation area; and a fourth determination module 460, configured to determine a control strategy of the vehicle based on the risk level of the water splashing risk.

[0242] In the present embodiment, the control device for the vehicle comprises a fifth determination module, configured to determine, in a case where it is determined based on the depth information of the water accumulation area that the vehicle cannot safely pass through the water accumulation area, that the control strategy of the vehicle is to output warning information through a display module of the vehicle and / or control the vehicle to perform a braking operation.

[0243] In this embodiment, the second determining module comprises: a first determining unit, configured to determine first depth information of the waterlogging area based on the first radar data; a second determining unit, configured to determine second depth information of the waterlogging area based on the second radar data; and a third determining unit, configured to determine the depth information of the waterlogging area based on the first depth information and the second depth information.

[0244] In this embodiment, the third determining module comprises: a fourth determining unit, configured to determine a splashing area triggered when the vehicle passes through the waterlogging area based on the depth of the waterlogging area and the driving speed of the vehicle; a fifth determining unit, configured to determine a first time when the moving object enters the splashing area based on the trajectory information of the moving object and the splashing area; a sixth determining unit, configured to determine a second time when the vehicle enters the splashing area based on the driving speed of the vehicle and the splashing area; and a seventh determining unit, configured to determine a risk level of the splashing risk of the moving object when passing through the road based on the first time and the second time.

[0245] In this embodiment, the seventh determining unit comprises: a first determining sub-unit, configured to determine that the risk level of the splashing risk of the moving object when passing through the road is high risk based on the first time, the second time and a first time threshold; a second determining sub-unit, configured to determine that the risk level of the splashing risk of the moving object when passing through the road is medium risk based on the first time, the second time and a second time threshold; the second time threshold is greater than the first time threshold; and a third determining sub-unit, configured to determine that the risk level of the splashing risk of the moving object when passing through the road is low risk based on the first time, the second time and a third time threshold; the third time threshold is greater than or equal to the second time threshold.

[0246] In this embodiment, the fourth determining module comprises: an eighth determining unit, configured to determine that the control strategy of the vehicle is to send warning information to the driver of the vehicle through a display module and / or a voice module of the vehicle to remind the driver to perform deceleration or braking operation in the case that the risk level of the splashing risk is high risk; a ninth determining unit, configured to determine that the control strategy of the vehicle is to send warning information to the driver of the vehicle through a display module and / or a voice module of the vehicle to remind the driver to perform deceleration operation in the case that the risk level of the splashing risk is medium risk; and a tenth determining unit, configured to determine that the control strategy of the vehicle is to control the vehicle not to send warning information so as to make the vehicle travel according to the current driving state in the case that the risk level of the splashing risk is low risk.

[0247] In this embodiment, the driving mode includes automatic driving, and the fourth determination module includes: an eleventh determination unit, configured to determine, in a case where the risk level of the water splashing risk is high risk and the driving mode is automatic driving, that the control strategy of the vehicle is to control the vehicle to perform an emergency braking or emergency deceleration operation; a twelfth determination unit, configured to determine, in a case where the risk level of the water splashing risk is medium risk and the driving mode is automatic driving, that the control strategy of the vehicle is to control the vehicle to perform a gentle deceleration operation; and a thirteenth determination unit, configured to determine, in a case where the risk level of the water splashing risk is low risk and the driving mode is automatic driving, that the control strategy of the vehicle is to control the vehicle to maintain the current vehicle speed.

[0248] In this embodiment, the first determination module includes: a fourteenth determination unit, configured to determine first feature information in each of the image data in the bird's eye view of the road based on the image data collected by the plurality of cameras; a fifteenth determination unit, configured to determine second feature information in the third radar data in the bird's eye view based on third radar data collected by the third radar installed on the vehicle; the types of the first radar, the second radar, and the third radar are different; and a sixteenth determination unit, configured to determine position information of a water accumulation area on a current driving road of the vehicle based on the first feature information and the second feature information.

[0249] In this embodiment, the first determination module includes: a seventeenth determination unit, configured to determine position information of a water accumulation area based on the third radar data; and an eighteenth determination unit, configured to correct the position information of the water accumulation area based on radar data collected by the first radar, radar data collected by the second radar, and the third radar data; and the second acquisition module includes: acquiring first radar data of the water accumulation area collected by the first radar and second radar data of the water accumulation area collected by the second radar based on the corrected position information of the water accumulation area.

[0250] The embodiment also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a control device of a vehicle to implement some or all of the steps of the method.

[0251] The embodiment also provides a computer program, which includes computer programs or instructions. The computer programs or instructions are executed by a control device of a vehicle to implement some or all of the steps of the method.

[0252] The embodiment further provides a computer program product. The computer program product comprises a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, some or all of the steps of the above method are implemented. The computer program product can be implemented by hardware, software or a combination thereof. In some embodiments, the computer program product is embodied as a computer storage medium. In other embodiments, the computer program product is embodied as a software product, such as a software development kit (SDK) or the like.

[0253] It should be noted that the above description of the various embodiments tends to emphasize the differences between the various embodiments, and the same or similar parts can be mutually referred to. The above description of the device, storage medium, computer program and computer program product embodiments is similar to the description of the method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the device, storage medium, computer program and computer program product embodiments of the present application, please refer to the description of the method embodiments for understanding.

Claims

1. A control method of a vehicle, characterized by, The method comprises: determining first feature information in each image data under a bird's eye view of a road based on image data collected by multiple cameras; determining second feature information in third radar data under the bird's eye view based on third radar data collected by a third radar installed on a vehicle; determining position information of a waterlogged area on a road currently traveled by the vehicle based on the first feature information and the second feature information; determining trajectory information of a moving object on the road currently traveled by the vehicle; acquiring first radar data of the waterlogged area collected by a first radar and second radar data of the waterlogged area collected by a second radar installed on the vehicle based on the position information of the waterlogged area; the first radar, the second radar and the third radar are different in type; determining depth information of the waterlogged area based on the first radar data of the waterlogged area and the second radar data of the waterlogged area; in a case where it is determined based on the depth information of the waterlogged area that the vehicle can safely pass through the waterlogged area, determining a splash water area triggered when the vehicle passes through the waterlogged area based on the depth information of the waterlogged area and a driving speed of the vehicle; determining a first time when the moving object enters the splash water area based on the trajectory information of the moving object and the splash water area; determining a second time when the vehicle enters the splash water area based on the driving speed of the vehicle and the splash water area; determining a risk level of a splash water risk of the moving object when passing through the road based on the first time and the second time; determining a control strategy of the vehicle based on the risk level of the splash water risk.

2. The method according to claim 1, characterized in that The method comprises: in a case where it is determined based on the depth information of the waterlogged area that the vehicle cannot safely pass through the waterlogged area, determining a control strategy of the vehicle as outputting warning information through a display module and / or a voice module of the vehicle and / or controlling the vehicle to brake.

3. The method according to claim 1, characterized in that The method comprises: determining first depth information of the waterlogged area based on the first radar data; determining second depth information of the waterlogged area based on the second radar data; determining depth information of the waterlogged area based on the first depth information and the second depth information.

4. The method according to claim 1, characterized in that The method comprises: determining that the risk level of the splash water risk of the moving object when passing through the road is high risk based on the first time, the second time and a first time threshold; determining that the risk level of the splash water risk of the moving object when passing through the road is medium risk based on the first time, the second time and a second time threshold; the second time threshold is greater than the first time threshold. determine, based on the first time, the second time, and a third time threshold, that the mobile object has a low risk of splashing water when passing through the road; the third time threshold is greater than or equal to the second time threshold.

5. The method according to any one of claims 1 to 4, characterized in that determine, based on the risk level of the splashing water risk, a control strategy of the vehicle, including: in a case where the risk level of the splashing water risk is high, determine that the control strategy of the vehicle is to send a warning information to a driver of the vehicle through a display module and / or a voice module of the vehicle to remind the driver to perform a deceleration or braking operation; in a case where the risk level of the splashing water risk is medium, determine that the control strategy of the vehicle is to send a warning information to a driver of the vehicle through a display module and / or a voice module of the vehicle to remind the driver to perform a deceleration operation; in a case where the risk level of the splashing water risk is low, determine that the control strategy of the vehicle is to control the vehicle not to send a warning information, so that the vehicle travels according to a current traveling state.

6. The method according to any one of claims 1 to 4, characterized in that the driving mode includes automatic driving, and the control strategy of the vehicle is determined based on the risk level of the splashing water risk, including: in a case where the risk level of the splashing water risk is high and the driving mode is automatic driving, determine that the control strategy of the vehicle is to control the vehicle to perform an emergency braking or emergency deceleration operation; in a case where the risk level of the splashing water risk is medium and the driving mode is automatic driving, determine that the control strategy of the vehicle is to control the vehicle to perform a gentle deceleration operation; in a case where the risk level of the splashing water risk is low and the driving mode is automatic driving, determine that the control strategy of the vehicle is to control the vehicle to maintain a current vehicle speed.

7. The method according to any one of claims 1 to 4, characterized in that The method further includes: determine position information of the water accumulation area based on the third radar data; correct the position information of the water accumulation area based on the radar data collected by the first radar, the radar data collected by the second radar, and the third radar data; wherein, based on the position information of the water accumulation area, the first radar data of the water accumulation area collected by the first radar installed on the vehicle and the second radar data of the water accumulation area collected by the second radar are obtained, including: based on the corrected position information of the water accumulation area, the first radar data of the water accumulation area collected by the first radar installed on the vehicle and the second radar data of the water accumulation area collected by the second radar are obtained.

8. A control device of a vehicle characterized by comprising: The vehicle is provided with a first radar, a second radar, and a third radar; the control device includes: a first acquisition module for acquiring radar data collected by the first radar installed on the vehicle, radar data collected by the second radar installed on the vehicle, and third radar data collected by the third radar installed on the vehicle; The first determining module is configured to determine first feature information in each image data under a bird's eye view of a road based on image data collected by a plurality of cameras; determine second feature information in third radar data under the bird's eye view based on third radar data collected by a third radar installed on the vehicle; determine position information of a waterlogged area on the road currently traveled by the vehicle based on the first feature information and the second feature information; and determine trajectory information of a moving object on the road currently traveled by the vehicle. The second obtaining module is configured to obtain first radar data of the waterlogged area collected by the first radar based on the position information of the waterlogged area and radar data collected by the first radar, and obtain second radar data of the waterlogged area collected by the second radar based on the position information of the waterlogged area and radar data collected by the second radar; and the first radar, the second radar, and the third radar are of different types. The second determining module is configured to determine depth information of the waterlogged area based on the first radar data of the waterlogged area and the second radar data of the waterlogged area. The third determining module is configured to, in a case where it is determined based on the depth information of the waterlogged area that the vehicle can safely pass through the waterlogged area, determine a water splashing area triggered when the vehicle passes through the waterlogged area based on the depth information of the waterlogged area and a driving speed of the vehicle; determine a first time when the moving object enters the water splashing area based on the trajectory information of the moving object and the water splashing area; determine a second time when the vehicle enters the water splashing area based on the driving speed of the vehicle and the water splashing area; and determine a risk level of a water splashing risk of the moving object when passing through the road based on the first time and the second time. The fourth determining module is configured to determine a control strategy of the vehicle based on the risk level of the water splashing risk.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the control device of the vehicle to implement the steps in the method of any one of claims 1 to 7.

10. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instructions are executed by the control device of the vehicle to implement the steps in the method of any one of claims 1 to 7.

Images