Vehicle control method, system and equipment in severe weather and medium

Abstract

The invention discloses a severe weather vehicle control method, system and device and a medium. The method comprises the steps of obtaining multi-modal data of a target vehicle; performing weather risk prediction processing on the multi-modal data to obtain weather risk information; performing pavement condition analysis processing on the multi-modal data to obtain pavement condition information; performing vehicle state analysis processing on the multi-modal data to obtain vehicle state information; performing feature fusion processing on the weather risk information, the road condition information and the vehicle state information to obtain fused feature data; inputting the fused feature data into a pre-trained strategy matching model to obtain a target control strategy; and performing vehicle control on the target vehicle according to the target control strategy. According to the embodiment of the invention, the corresponding equipment can be automatically started in severe weather, the safety of the vehicle and the driving experience of a user are improved, and the method can be widely applied to the technical field of vehicle control.

Description

Technical Field

[0001] The present application relates to the field of vehicle safety technology, and in particular to a method, system, device and medium for controlling a vehicle in severe weather. Background Art

[0002] In related technologies, driving in inclement weather primarily relies on the driver's manual judgment of weather conditions to activate corresponding devices. For example, fog lights and windshield wipers are manually activated during heavy rain. When encountering inclement weather, manual operation is required, which distracts the driver and easily leads to operational errors, compromising driving safety.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to provide a method, system, device and medium for controlling a vehicle in severe weather, which can improve the safety of the vehicle.

[0005] To achieve the above objectives, one aspect of an embodiment of the present application provides a method for controlling a vehicle in severe weather, the method comprising:

[0006] Acquire multimodal data of the target vehicle;

[0007] Performing weather risk forecasting processing on the multimodal data to obtain weather risk information;

[0008] Performing road condition analysis on the multimodal data to obtain road condition information;

[0009] Performing vehicle status analysis on the multimodal data to obtain vehicle status information;

[0010] Performing feature fusion processing on the weather risk information, the road condition information, and the vehicle status information to obtain fused feature data;

[0011] Inputting the fused feature data into a pre-trained strategy matching model to obtain a target control strategy;

[0012] The target vehicle is controlled according to the target control strategy.

[0013] In some embodiments, obtaining multimodal data of the target vehicle includes:

[0014] The surrounding weather conditions of the target vehicle are monitored and processed through rain sensors, humidity sensors, and temperature sensors to obtain weather data;

[0015] Obtaining the positioning data of the target vehicle through a positioning module;

[0016] Perform three-dimensional modeling of the target vehicle and its surrounding environment through cameras and lidar to obtain a three-dimensional model;

[0017] The weather data, the positioning data and the three-dimensional model are subjected to data fusion processing to obtain the multimodal data.

[0018] In some embodiments, performing weather risk prediction processing on the multimodal data to obtain weather risk information includes:

[0019] Obtaining geographical location weather information of the target vehicle based on the multimodal data;

[0020] Performing weather feature extraction processing on the multimodal data to obtain weather features;

[0021] Classify the weather characteristics according to the meteorological information of the geographical location to obtain a weather type;

[0022] Performing weather grade classification processing on the multimodal data according to the weather type to obtain a weather grade;

[0023] Risk prediction processing is performed according to the weather type and the weather level to obtain the weather risk information.

[0024] In some embodiments, performing road condition analysis on the multimodal data to obtain road condition information includes:

[0025] Analyzing and processing the target vehicle's driving path and ground conditions based on the multimodal data to obtain path ground information;

[0026] Performing three-dimensional scene analysis on the multimodal data to obtain traffic information;

[0027] The path ground information and the traffic information are subjected to data fusion processing to obtain the road condition information.

[0028] In some embodiments, performing vehicle status analysis on the multimodal data to obtain vehicle status data includes:

[0029] Performing detection processing on a control module of a target vehicle according to the multimodal data to obtain control module detection data;

[0030] Performing detection and processing on the entire body of the target vehicle according to the multimodal data to obtain body detection data;

[0031] The control module detection data and the vehicle body detection data are spliced ​​to obtain the vehicle status data.

[0032] In some embodiments, before inputting the fused feature data into a pre-trained policy matching model, the method further includes pre-training the policy matching model, wherein pre-training the policy matching model includes:

[0033] Obtain weather characteristic data, road characteristic data, and vehicle characteristic data of the test vehicle, and determine the corresponding control strategy label based on manual annotation;

[0034] Performing data fusion processing on the weather characteristic data, the road surface characteristic data, and the vehicle characteristic data to obtain a training data set;

[0035] Input the training data set into the pre-built neural network model for strategy matching to obtain the matching results;

[0036] Calculate a loss value based on the matching result and the control strategy label;

[0037] The parameters of the neural network model are updated according to the loss value to obtain the trained strategy matching model.

[0038] In some embodiments, controlling the target vehicle according to the target control strategy includes:

[0039] Generate corresponding control instructions according to the target control strategy, and send the control instructions to the electronic control unit;

[0040] The target module of the target vehicle is controlled by the electronic control unit according to the control instruction.

[0041] To achieve the above objectives, another aspect of the present application provides a severe weather vehicle control system, the system comprising:

[0042] A data acquisition module, used to acquire multimodal data of the target vehicle;

[0043] A risk prediction module, configured to perform weather risk prediction processing on the multimodal data to obtain weather risk information;

[0044] a road surface analysis module, configured to perform road surface condition analysis on the multimodal data to obtain road surface condition information;

[0045] A vehicle analysis module, configured to perform vehicle status analysis on the multimodal data to obtain vehicle status information;

[0046] a feature fusion module, configured to perform feature fusion processing on the weather risk information, the road condition information, and the vehicle status information to obtain fused feature data;

[0047] A strategy matching module is used to input the fusion feature data into a pre-trained strategy matching model to obtain a target control strategy;

[0048] A vehicle control module is used to control the target vehicle according to the target control strategy.

[0049] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned method when executing the computer program.

[0050] To achieve the above objectives, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0051] To achieve the above object, another aspect of the present application provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0052] Embodiments of the present application include at least the following beneficial effects: The present application provides a method, system, device, and medium for controlling a vehicle in severe weather. This solution obtains multimodal data from a target vehicle; performs weather risk prediction processing on the multimodal data to obtain weather risk information; performs road condition analysis processing on the multimodal data to obtain road condition information; performs vehicle state analysis processing on the multimodal data to obtain vehicle state information; performs feature fusion processing on the weather risk information, road condition information, and vehicle state information to obtain fused feature data; inputs the fused feature data into a pre-trained policy matching model to obtain a target control policy; and controls the target vehicle according to the target control policy. By processing the multimodal data and inputting it into a policy matching model to obtain a control policy, the present application can automatically activate corresponding devices in severe weather, thereby improving vehicle safety and the user's driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of a method for controlling a vehicle in severe weather provided by an embodiment of the present application;

[0054] Figure 2 This is a schematic structural diagram of a severe weather vehicle control system provided by an embodiment of the present application;

[0055] Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of systems and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0057] It will be appreciated that the terms "first", "second", etc. used herein may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are merely used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, first data may also be referred to as second data, and similarly, second data may also be referred to as first data. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0058] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0060] In the related art, when driving in the rain, road visibility is poor, and due to the large temperature and humidity difference between the inside and outside of the vehicle on rainy days, the vehicle windshield is prone to fogging and frosting. Simply scraping rain off the vehicle's front windshield with wipers is not enough to ensure driving safety. When the driver needs to turn on other equipment to assist driving in the rain, insufficient preparation or improper operation may result in an inability to accurately turn on the auxiliary equipment, or even cause a safety accident. For example, due to the large temperature difference between indoors and outdoors and high humidity on rainy days, the windshield may fog up while driving, thereby obstructing the driver's vision. In this case, the driver may not be able to remove the fog in time due to the delay in turning on the air conditioner to defog, affecting driving safety. For example, the driver needs to turn on the fog lights to assist driving, but due to panic, the hazard lights are mistakenly turned on as fog lights, thus affecting driving safety.

[0061] In light of this, embodiments of the present application provide a method, system, device, and medium for controlling vehicles in severe weather. This solution obtains multimodal data from a target vehicle; performs weather risk prediction processing on the multimodal data to obtain weather risk information; performs road condition analysis processing on the multimodal data to obtain road condition information; performs vehicle state analysis processing on the multimodal data to obtain vehicle state information; performs feature fusion processing on the weather risk information, road condition information, and vehicle state information to obtain fused feature data; inputs the fused feature data into a pre-trained policy matching model to obtain a target control policy; and controls the target vehicle according to the target control policy. By processing multimodal data and inputting it into a policy matching model to obtain a control policy, the present application can automatically activate corresponding devices in severe weather, improving vehicle safety and the user's driving experience.

[0062] The embodiment of the present application provides a method for controlling a vehicle in severe weather, which relates to the field of vehicle safety technology. The embodiment of the present application provides a method for controlling a vehicle in severe weather, which can be applied to a vehicle controller, or to a server, or can be software running in a vehicle controller or a server. The server side can be configured as an independent physical server, or as a server cluster or distributed system consisting of multiple physical servers. It can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements a method for controlling a vehicle in severe weather, etc., but is not limited to the above forms.

[0063] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0064] Figure 1This is an optional flowchart of a method for controlling a vehicle in severe weather provided by an embodiment of the present application. Figure 1 The method may include, but is not limited to, the following steps:

[0065] Step S101, obtaining multimodal data of a target vehicle;

[0066] Step S102, performing weather risk prediction processing on the multimodal data to obtain weather risk information;

[0067] Step S103, performing road condition analysis on the multimodal data to obtain road condition information;

[0068] Step S104, performing vehicle status analysis on the multimodal data to obtain vehicle status information;

[0069] Step S105, performing feature fusion processing on the weather risk information, road condition information and vehicle status information to obtain fused feature data;

[0070] Step S106, inputting the fused feature data into a pre-trained strategy matching model to obtain a target control strategy;

[0071] Step S107: Control the target vehicle according to the target control strategy.

[0072] In steps S101 to S107, as shown in the embodiment of the present application, by acquiring multimodal data of a target vehicle, the multimodal data can be analyzed and processed. Weather risk prediction and identification are performed on the multimodal data to obtain the target vehicle's weather environment. Road condition analysis and identification are performed to obtain the target vehicle's driving environment. Furthermore, the target vehicle's vehicle state is analyzed to obtain weather risk information, road condition information, and vehicle state information. The identified information is then subjected to feature fusion to obtain fused feature data, which is then input into a pre-trained policy matching model. The policy matching model then matches a corresponding control strategy based on the current target vehicle's weather environment. The control strategy is then adjusted based on the road condition and vehicle state, thereby obtaining a target control strategy. Vehicle auxiliary control of the target vehicle is then performed based on the target control strategy. For example, when the weather is rainy, the target control strategy can be used to automatically turn on the target vehicle's windshield wipers and adjust the wiper frequency. When the road surface is slippery and has a significant light scattering effect, the target control strategy can be used to turn on the target vehicle's anti-skid mode and adjust the vehicle's lights.

[0073] In some embodiments, obtaining multimodal data of the target vehicle includes:

[0074] The surrounding weather conditions of the target vehicle are monitored and processed through rain sensors, humidity sensors, and temperature sensors to obtain weather data;

[0075] Obtaining the positioning data of the target vehicle through a positioning module;

[0076] Perform three-dimensional modeling of the target vehicle and its surrounding environment through cameras and lidar to obtain a three-dimensional model;

[0077] The weather data, the positioning data and the three-dimensional model are subjected to data fusion processing to obtain the multimodal data.

[0078] In an embodiment of the present application, the target vehicle's surrounding weather conditions can be monitored in real time by means of on-board sensors installed on the target vehicle. Data can be collected from the target vehicle through rain sensors, humidity sensors, temperature sensors, etc., and a data basis can be provided for subsequent weather status identification. For example, the rain sensing data obtained by the rain sensor can provide a data basis for the subsequent neural network model to determine whether it is a rainy day, and the neural network model can adjust the wiper's swing frequency according to the amount of rain, thereby matching the corresponding control strategy. In addition, the embodiment of the present application performs three-dimensional modeling of the target vehicle and the surrounding environment through cameras and lidars. The image obtained by the camera can be processed by extracting a two-dimensional feature map, and then fused with the point cloud features obtained by the lidar. The spatial attention weight is calculated based on the two-dimensional feature map for weighted fusion processing to construct a three-dimensional model. The embodiment of the present application also obtains the positioning data of the target vehicle according to the positioning module, thereby fusing the weather data, positioning data and three-dimensional model to obtain multimodal data.

[0079] In some embodiments, performing weather risk prediction processing on the multimodal data to obtain weather risk information includes:

[0080] Obtaining geographical location weather information of the target vehicle based on the multimodal data;

[0081] Performing weather feature extraction processing on the multimodal data to obtain weather features;

[0082] Classify the weather characteristics according to the meteorological information of the geographical location to obtain a weather type;

[0083] Performing weather grade classification processing on the multimodal data according to the weather type to obtain a weather grade;

[0084] Risk prediction processing is performed according to the weather type and the weather level to obtain the weather risk information.

[0085] In an embodiment of the present application, the geographical location of the target vehicle can be obtained based on the positioning data in the multimodal data. That is, the meteorological information of the corresponding location can be obtained through network search or other methods based on the positioning data. This meteorological information can be used as reference data to provide a data basis for weather classification. By classifying and identifying weather characteristics, a weather type can be obtained. In an embodiment of the present application, a convolutional neural network can be used to extract features from the multimodal data to obtain weather characteristics. The weather characteristics can be rainfall characteristics, temperature characteristics, etc., so as to classify the weather type and obtain the weather type. Based on the classified weather type, the multimodal data is classified into weather levels to obtain weather grades. For example, the grades can be classified according to the amount of rainfall, the temperature, etc., and the grades can be classified according to preset grades. Finally, a risk prediction is performed based on the weather type and weather grade to obtain weather risk information. For example, the weather type and weather grade can be used to determine whether the weather is in severe weather such as heavy rain or high temperature. In an embodiment of the present application, corresponding risk predictions can be made based on severe weather, such as the risk level of slippery roads and the risk level of heatstroke.

[0086] In some embodiments, performing road condition analysis on the multimodal data to obtain road condition information includes:

[0087] Analyzing and processing the target vehicle's driving path and ground conditions based on the multimodal data to obtain path ground information;

[0088] Performing three-dimensional scene analysis on the multimodal data to obtain traffic information;

[0089] The path ground information and the traffic information are subjected to data fusion processing to obtain the road condition information.

[0090] In an embodiment of the present application, the target vehicle's travel path can be predicted based on a three-dimensional model in the multimodal data, and the corresponding road surface conditions can be analyzed and processed, such as detecting the slipperiness of the road surface and analyzing the light scattering effect on the road surface. Furthermore, the embodiment of the present application also performs three-dimensional scene analysis on the multimodal data, identifying pedestrians, vehicles, or obstacles by processing the images captured by the camera, and also identifying obstacles using radar sensors, thereby constructing three-dimensional traffic information. Finally, the path ground information and traffic information are fused to obtain road surface condition information for the target vehicle.

[0091] In some embodiments, performing vehicle status analysis on the multimodal data to obtain vehicle status data includes:

[0092] Performing detection processing on a control module of a target vehicle according to the multimodal data to obtain control module detection data;

[0093] Performing detection and processing on the entire body of the target vehicle according to the multimodal data to obtain body detection data;

[0094] The control module detection data and the vehicle body detection data are spliced ​​to obtain the vehicle status data.

[0095] In an embodiment of the present application, by performing control detection processing on the control module of the target vehicle, control module detection data can be obtained, that is, by detecting the control module of the target vehicle, such as detecting the zone controller, the vehicle's position, speed, posture and other data can be obtained, and the operating status of each controller can also be obtained. The embodiment of the present application also performs detection processing on the entire body of the target vehicle based on multimodal data to obtain body detection data, such as detecting all the glass of the vehicle, such as the door glass, front windshield, etc. Finally, the control module detection data and the body detection data are spliced ​​to obtain vehicle status data. The embodiment of the present application can use the sensor values ​​and the pictures taken by the camera to process through the on-board neural network to determine whether the vehicle needs to turn on the rearview mirror heating, defog the front windshield and other door glasses, turn on the heating function of the front and rear electronic rearview mirrors, remove related water mist and other related functions, so that the driver has a clear enough operating field of view.

[0096] In some embodiments, before inputting the fused feature data into a pre-trained policy matching model, the method further includes pre-training the policy matching model, wherein pre-training the policy matching model includes:

[0097] Obtain weather characteristic data, road characteristic data, and vehicle characteristic data of the test vehicle, and determine the corresponding control strategy label based on manual annotation;

[0098] Performing data fusion processing on the weather characteristic data, the road surface characteristic data, and the vehicle characteristic data to obtain a training data set;

[0099] Input the training data set into the pre-built neural network model for strategy matching to obtain the matching results;

[0100] Calculate a loss value based on the matching result and the control strategy label;

[0101] The parameters of the neural network model are updated according to the loss value to obtain the trained strategy matching model.

[0102] In an embodiment of the present application, the vehicle characteristic data of the test vehicle is obtained, and the weather characteristic data and road characteristic data of the test vehicle under the preset environment are manually labeled to determine the corresponding control strategy label. For example, for rainy day features, control strategy labels such as turning on the wipers can be labeled. Specifically, by inputting the training data set into the initialized deep learning neural network, the matching result of the model output can be obtained, that is, the control strategy matching result. The accuracy of the model matching can be evaluated based on the control strategy matching result and the aforementioned control strategy label, thereby updating the parameters of the model. For the strategy matching model, the accuracy of the model matching result can be measured by the loss function (LossFunction). The loss function is defined on a single training data and is used to measure the prediction error of a training data. Specifically, the loss value of the training data is determined by the label of a single training data and the matching result of the model on the training data. During actual training, a training data set has a lot of training data, so a cost function (CostFunction) is generally used to measure the overall error of the training data set. The cost function is defined on the entire training data set and is used to calculate the average value of the matching error of all training data, which can better measure the matching effect of the model. For general machine learning models, the aforementioned cost function, plus a regularization term that measures the complexity of the model, can be used as the objective function for training, and the loss value of the entire training data set can be calculated based on the objective function. There are many types of commonly used loss functions, such as 0-1 loss function, square loss function, absolute loss function, logarithmic loss function, cross entropy loss function, etc., which can all be used as loss functions of machine learning models, which will not be elaborated one by one here. In an embodiment of the present invention, any one of the loss functions can be selected to determine the loss value of training. Based on the loss value of training, the back propagation algorithm is used to update the parameters of the model, and a trained strategy matching model can be obtained after several rounds of iteration. The specific number of iterations can be set in advance, or the training is considered to be completed when the test set meets the accuracy requirements.

[0103] In some embodiments, controlling the target vehicle according to the target control strategy includes:

[0104] Generate corresponding control instructions according to the target control strategy, and send the control instructions to the electronic control unit;

[0105] The target module of the target vehicle is controlled by the electronic control unit according to the control instruction.

[0106] In an embodiment of the present application, corresponding control instructions can be generated according to the target control strategy. For example, if the target control strategy is to turn on the wiper and adjust the swing frequency of the wiper, a corresponding control instruction will be generated and sent to the electronic control unit that controls the wiper. The electronic control unit will turn on the wiper of the target vehicle according to the control instruction, and adjust and control the swing frequency of the wiper according to the swing frequency in the control strategy.

[0107] The following describes the embodiments of the present invention in detail with reference to specific application examples:

[0108] The embodiments of the present application can be applied to the field of vehicle technology, and are suitable for application scenarios in which auxiliary control of vehicles is performed in bad weather. The embodiments of the present application can be equipped with rain sensors, air humidity sensors, temperature sensors, inside and outside vehicle cameras, active mufflers, etc. inside and outside the vehicle, so as to collect data based on these devices, and then perform multi-sensor information fusion on each sensor through a neural network to obtain multimodal data. GPS timestamps can be used to time-align each sensor, and the error can be controlled at the millisecond level. In addition, the embodiments of the present application can also use software to align the data time axis through Kalman filtering, perform spatial alignment processing, and confirm the coordinate transformation matrix between each sensor through an automatic calibration algorithm. The embodiments of the present application can provide a data basis for subsequent automatic function operation judgments by detecting rainfall, air humidity, internal and external temperature differences and humidity. The present embodiment of the present invention performs weather risk prediction, road condition analysis, and vehicle status analysis and feature fusion processing on multimodal data to obtain fused feature data. This data is then input into a pre-trained strategy matching model. The model output generates a corresponding target control strategy, which can be used to perform auxiliary control on the target vehicle. For example, if the vehicle's ESP system (electronic stability program) is turned off, the ESP system will be automatically restarted according to the target control to maintain vehicle driving stability on slippery roads in rainy weather. By analyzing multimodal data for weather visibility and other factors, it can be determined whether to automatically turn on the front and rear fog lights, thereby performing auxiliary control such as turning on the front and rear fog lights according to the target control strategy. The present embodiment of the present invention can also use a neural network to combine and judge the vehicle's location information, current weather conditions, surrounding vehicle conditions, and risk avoidance conditions in real time, thereby performing auxiliary control on the target vehicle according to the target control strategy. The present embodiment of the present invention can determine the weather conditions of the target vehicle and simultaneously perform weather risk prediction based on the vehicle's built-in positioning system and weather warning device, thereby invoking relevant systems to execute relevant commands. For example, when it is identified as light rain or light fog, according to the target control strategy, the vehicle is mainly driven by the camera, assisted by the millimeter-wave radar, and the fog lights are turned on, and the ESP system is turned on for auxiliary control.

[0109] See also Figure 2 The present application also provides a severe weather vehicle control system that can implement the above method. The system includes:

[0110] Data acquisition module 201, used to acquire multimodal data of the target vehicle;

[0111] The risk prediction module 202 is configured to perform weather risk prediction processing on the multimodal data to obtain weather risk information;

[0112] A road surface analysis module 203 is configured to perform road surface condition analysis on the multimodal data to obtain road surface condition information;

[0113] The vehicle analysis module 204 is configured to perform vehicle status analysis on the multimodal data to obtain vehicle status information;

[0114] A feature fusion module 205 is configured to perform feature fusion processing on the weather risk information, the road condition information, and the vehicle status information to obtain fused feature data;

[0115] A strategy matching module 206 is configured to input the fused feature data into a pre-trained strategy matching model to obtain a target control strategy;

[0116] The vehicle control module 207 is configured to control the target vehicle according to the target control strategy.

[0117] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0118] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above method when executing the computer program. The electronic device can be any smart terminal including a tablet computer, an in-vehicle computer, or the like.

[0119] It can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0120] See also Figure 3 , Figure 3 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0121] The processor 301 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0122] The memory 302 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 302 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 302 and is called by the processor 301 to execute the above-mentioned methods of the embodiments of this application.

[0123] Input / output interface 303, used to implement data input and output;

[0124] Communication interface 304, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0125] bus 305 , which transmits data between the various components of the device (e.g., processor 301 , memory 302 , input / output interface 303 , and communication interface 304 );

[0126] The processor 301 , the memory 302 , the input / output interface 303 and the communication interface 304 are connected to each other in communication within the device via the bus 305 .

[0127] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above method is implemented.

[0128] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0129] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0130] It is understandable that the contents of the above method embodiments are all applicable to the present program product embodiments, the functions specifically implemented by the present program product embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0131] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0132] Embodiments of the present application provide a method, system, device, and medium for controlling a vehicle in severe weather. This solution obtains multimodal data from a target vehicle; performs weather risk prediction processing on the multimodal data to obtain weather risk information; performs road condition analysis processing on the multimodal data to obtain road condition information; performs vehicle state analysis processing on the multimodal data to obtain vehicle state information; performs feature fusion processing on the weather risk information, road condition information, and vehicle state information to obtain fused feature data; inputs the fused feature data into a pre-trained policy matching model to obtain a target control policy; and controls the target vehicle according to the target control policy. By processing multimodal data and inputting it into a policy matching model to obtain a control policy, the present application can automatically activate corresponding devices in severe weather, improving vehicle safety and the user's driving experience.

[0133] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0134] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0135] The system embodiment described above is merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0136] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0137] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0138] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the above units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0140] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0142] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0143] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A method for controlling a vehicle in severe weather, characterized in that: The method comprises the following steps: Acquire multimodal data of the target vehicle; Performing weather risk forecasting processing on the multimodal data to obtain weather risk information; Performing road condition analysis on the multimodal data to obtain road condition information; Performing vehicle status analysis on the multimodal data to obtain vehicle status information; Performing feature fusion processing on the weather risk information, the road condition information, and the vehicle status information to obtain fused feature data; Inputting the fused feature data into a pre-trained strategy matching model to obtain a target control strategy; The target vehicle is controlled according to the target control strategy.

2. The method according to claim 1, characterized in that The acquiring of multimodal data of the target vehicle includes: The surrounding weather conditions of the target vehicle are monitored and processed through rain sensors, humidity sensors, and temperature sensors to obtain weather data; Obtaining the positioning data of the target vehicle through a positioning module; Perform three-dimensional modeling of the target vehicle and its surrounding environment through cameras and lidar to obtain a three-dimensional model; The weather data, the positioning data and the three-dimensional model are subjected to data fusion processing to obtain the multimodal data.

3. The method according to claim 1, characterized in that The performing weather risk prediction processing on the multimodal data to obtain weather risk information includes: Obtaining geographical location weather information of the target vehicle based on the multimodal data; Performing weather feature extraction processing on the multimodal data to obtain weather features; Classify the weather characteristics according to the meteorological information of the geographical location to obtain a weather type; Performing weather grade classification processing on the multimodal data according to the weather type to obtain a weather grade; Risk prediction processing is performed according to the weather type and the weather level to obtain the weather risk information.

4. The method according to claim 1, wherein The performing road condition analysis on the multimodal data to obtain road condition information includes: Analyzing and processing the target vehicle's driving path and ground conditions based on the multimodal data to obtain path ground information; Performing three-dimensional scene analysis on the multimodal data to obtain traffic information; The path ground information and the traffic information are subjected to data fusion processing to obtain the road condition information.

5. The method according to claim 1, wherein The performing vehicle status analysis on the multimodal data to obtain vehicle status data includes: Performing detection processing on a control module of a target vehicle according to the multimodal data to obtain control module detection data; Performing detection and processing on the entire body of the target vehicle according to the multimodal data to obtain body detection data; The control module detection data and the vehicle body detection data are spliced ​​to obtain the vehicle status data.

6. The method according to claim 1, characterized in that Before inputting the fused feature data into the pre-trained strategy matching model, the method further includes pre-training the strategy matching model, wherein the pre-training of the strategy matching model includes: Obtain weather characteristic data, road characteristic data, and vehicle characteristic data of the test vehicle, and determine the corresponding control strategy label based on manual annotation; Performing data fusion processing on the weather characteristic data, the road surface characteristic data, and the vehicle characteristic data to obtain a training data set; Input the training data set into the pre-built neural network model for strategy matching to obtain the matching results; Calculate a loss value based on the matching result and the control strategy label; The parameters of the neural network model are updated according to the loss value to obtain the trained strategy matching model.

7. The method according to any one of claims 1 to 6, characterized in that The controlling the target vehicle according to the target control strategy includes: Generate corresponding control instructions according to the target control strategy, and send the control instructions to the electronic control unit; The target module of the target vehicle is controlled by the electronic control unit according to the control instruction.

8. A severe weather vehicle control system, characterized in that: The system comprises: A data acquisition module, used to acquire multimodal data of the target vehicle; A risk prediction module, configured to perform weather risk prediction processing on the multimodal data to obtain weather risk information; a road surface analysis module, configured to perform road surface condition analysis on the multimodal data to obtain road surface condition information; A vehicle analysis module, configured to perform vehicle status analysis on the multimodal data to obtain vehicle status information; a feature fusion module, configured to perform feature fusion processing on the weather risk information, the road condition information, and the vehicle status information to obtain fused feature data; A strategy matching module is used to input the fusion feature data into a pre-trained strategy matching model to obtain a target control strategy; A vehicle control module is used to control the target vehicle according to the target control strategy.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images