Vehicle state prediction method and related equipment

Abstract

The invention relates to the field of new energy vehicles, in particular to a vehicle state prediction method and related equipment. The vehicle is provided with a plurality of sensors, and the method comprises the following steps: collecting a plurality of sensor data from the plurality of sensors respectively; carrying out data fusion on various sensor data to obtain multi-modal fusion data; and predicting the vehicle state of the vehicle based on the multi-modal fusion data and a preset vehicle state prediction model to obtain a predicted vehicle state. According to the method and the device, the vehicle sensor data of different sources and different types can be integrated, so that the vehicle state prediction model can fully utilize the multi-modal fusion data to carry out operation and analysis so as to adapt to different driving environments and working conditions to carry out vehicle state prediction, and the accuracy of vehicle state prediction is improved.

Description

Technical Field

[0001] The present application relates to the automotive field, and in particular to a vehicle state prediction method and related equipment. Background Art

[0002] As the transportation industry rapidly develops towards intelligence and automation, vehicle motion state prediction becomes increasingly important. It plays a key role in many application scenarios such as autonomous driving and intelligent driving assistance systems.

[0003] However, the traffic conditions and driver intentions faced by vehicles are relatively complex, so the difficulty of vehicle state prediction is greatly increased, and it is difficult to achieve accurate prediction of vehicle state. Summary of the invention In view of the above, an embodiment of the present application provides a vehicle state prediction method and related equipment, aiming to improve the accuracy of vehicle state prediction.

[0004] In a first aspect, a vehicle state prediction method is provided, wherein the vehicle is equipped with a plurality of sensors, and the method comprises: collecting a plurality of sensor data from the plurality of sensors respectively; Performing data fusion on the multiple sensor data to obtain multimodal fusion data; The vehicle state of the vehicle is predicted based on the multimodal fusion data and a preset vehicle state prediction model to obtain a predicted vehicle state.

[0005] The embodiments of the present application can integrate information of different sources and types by collecting multiple sensor data from multiple sensors of the vehicle, so that subsequent analysis can be carried out based on more comprehensive and accurate information, which helps to grasp the actual condition of the vehicle more accurately. Data of different modes can complement each other in the fusion process, which is conducive to eliminating the ambiguity and uncertainty that may exist in single modal data, improving the overall data quality, and providing a more reliable basis for vehicle state prediction, so that the vehicle state prediction model can make full use of multi-modal fusion data for calculation and analysis, so that the embodiments of the present application can adapt to different driving environments and working conditions to predict the vehicle state, and improve the accuracy of vehicle state prediction.

[0006] In some embodiments, after predicting the vehicle state of the vehicle based on the multimodal fusion data and a preset vehicle state prediction model to obtain the predicted vehicle state, the method further includes: obtaining an actual vehicle state of the vehicle; Model parameters of the vehicle state prediction model are adjusted based on the actual vehicle state and the predicted vehicle state.

[0007] In some embodiments, data fusion is performed on the multiple sensor data to obtain multimodal fusion data, including: Acquire generation time of the multiple sensor data; The multiple sensor data are aligned according to the generation time to obtain the multimodal fusion data.

[0008] In some embodiments, before fusing the multiple sensor data to obtain multimodal fusion data, the method includes: Performing a preprocessing operation on the multiple sensor data to obtain preprocessed sensor data; wherein the preprocessing operation includes at least one of a denoising operation, a principal component analysis, and a normalization operation; The step of fusing the multiple sensor data to obtain multimodal fusion data includes: The preprocessed sensor data is subjected to data fusion to obtain the multimodal fusion data.

[0009] In some embodiments, the vehicle state prediction model is trained using a deep belief network.

[0010] In some embodiments, the predicted vehicle state is used to indicate a risk condition of abnormal vehicle speed of the vehicle; After predicting the vehicle state of the vehicle based on the multimodal fusion data and the preset vehicle state prediction model to obtain the predicted vehicle state, the method further includes: determining a power control strategy for the vehicle based on the predicted vehicle state; The vehicle is controlled based on the power control strategy.

[0011] In some embodiments, the power control strategy includes a motor control strategy and a battery control strategy, and determining the power control strategy of the vehicle based on the predicted vehicle state includes: determining a speed abnormality type of the vehicle based on the predicted vehicle state; The motor control strategy and the battery control strategy are determined based on the abnormal vehicle speed type.

[0012] In a second aspect, an embodiment of the present application further provides a vehicle state prediction device, wherein the vehicle is equipped with a plurality of sensors, and the device comprises: A collection module, used for collecting a plurality of sensor data from the plurality of sensors respectively; A fusion module, used for fusing the data of the multiple sensors to obtain multi-modal fusion data; A prediction module is used to predict the vehicle state of the vehicle based on the multimodal fusion data and a preset vehicle state prediction model to obtain a predicted vehicle state.

[0013] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a processor and a memory, wherein the memory is used to store instructions, and the processor is used to call the instructions in the memory so that the electronic device executes the vehicle state prediction method described in the first aspect.

[0014] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the vehicle state prediction method described in the first aspect.

[0015] The technical effects obtained by the above-mentioned second, third and fourth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of an implementation environment of a vehicle state prediction method provided in an embodiment of the present application.

[0017] Figure 2 A flowchart of the steps of a vehicle state prediction method provided in one embodiment of the present application.

[0018] Figure 3 A flowchart of the steps of a vehicle state prediction method provided in another embodiment of the present application.

[0019] Figure 4 A schematic diagram of the structure of a vehicle state prediction model provided in one embodiment of the present application.

[0020] Figure 5 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the implementation methods of the present application and the features in the implementation methods can be combined with each other without conflict.

[0022] In the following description, many specific details are set forth to facilitate a full understanding of the present application. The described implementations are only part of the implementations of the present application, rather than all the implementations.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0024] It should be further noted that, in this article, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0025] In this application, "at least one" means one or more, and "more" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0026] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0027] Vehicle motion state prediction plays a key role in many application scenarios such as autonomous driving and intelligent driving assistance systems. Electronic devices can predict vehicle states and control the vehicle in a timely manner based on the predicted vehicle states to improve driving safety. However, the traffic conditions and driver intentions faced by vehicles are relatively complex, resulting in a high degree of difficulty in predicting vehicle states.

[0028] Therefore, in some embodiments, the electronic device may collect the current or voltage signal of the vehicle battery, etc., to predict whether the vehicle status (eg, vehicle speed) is abnormal.

[0029] However, the above prediction process only considers a single data source, and it is difficult to make accurate predictions in complex environments.

[0030] In view of the above, embodiments of the present application provide a vehicle state prediction method, device, electronic device and computer-readable storage medium.

[0031] The vehicle state prediction method includes: collecting multiple sensor data from multiple sensors of the vehicle respectively; performing data fusion on the multiple sensor data to obtain multimodal fusion data, and then predicting the vehicle state of the vehicle based on the multimodal fusion data and a preset vehicle state prediction model to obtain a predicted vehicle state.

[0032] The embodiments of the present application can integrate information of different sources and types by collecting multiple sensor data from multiple sensors of the vehicle, so that subsequent analysis can be carried out based on more comprehensive and accurate information, which helps to grasp the actual condition of the vehicle more accurately. Data of different modes can complement each other in the fusion process, which is conducive to eliminating the ambiguity and uncertainty that may exist in single modal data, improving the overall data quality, and providing a more reliable basis for vehicle state prediction, so that the vehicle state prediction model can make full use of multi-modal fusion data for calculation and analysis, so that the embodiments of the present application can adapt to different driving environments and working conditions to predict the vehicle state, and improve the accuracy of vehicle state prediction.

[0033] The above-mentioned vehicle state prediction method can be executed on a vehicle state prediction device. The vehicle state prediction device can be integrated into an electronic device. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to processors, microprogrammed control units (MCU), application specific integrated circuits (ASIC), programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc. The electronic device can be a vehicle controller, etc., but is not limited to this.

[0034] refer to Figure 1 As shown, the electronic device can be connected to communicate with various sensors configured in the vehicle to obtain various sensor data. For example, the vehicle can be configured with a driving speed sensor, an image acquisition sensor, a motor voltage and current sensor, and a power supply parameter sensor, but is not limited thereto.

[0035] Figure 2 This is a flowchart of the steps of an embodiment of the vehicle prediction method of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0036] See also Figure 2 As shown, the vehicle prediction method may include the following steps.

[0037] Step 201 : Collect a variety of sensor data from a variety of sensors respectively.

[0038] A vehicle may be equipped with a variety of sensors, and the electronic device may collect sensor data from the various sensors respectively. The various sensor data may include images of the vehicle's driving environment, motor voltage and current, battery parameters, and driving speed data.

[0039] For example, the sensors configured in the vehicle and their corresponding sensor data can be described as follows: 1. Image acquisition sensor: The image acquisition sensor is used to collect images of the vehicle's driving environment. The image acquisition sensor can use a camera, such as a CMOS camera, which can provide clear images in complex driving environments. The camera can use a 120-degree wide-angle lens to cover a wider viewing angle, thereby effectively capturing environmental information in front of and to the sides of the vehicle.

[0040] The camera can also be equipped with low-light enhancement technology and anti-glare functions to ensure stable image data even at night and in strong light environments.

[0041] The camera can capture images at preset intervals, for example, 30 frames per second, to ensure the real-time nature of image data and meet the system's requirements for rapid response and accurate analysis.

[0042] Image acquisition sensors can clearly capture real-time images of road signs, traffic signals, pedestrians and other vehicles. Vehicles can parse and analyze the collected image data through image processing technology to obtain key information such as road type, lane lines, driving direction and obstacles, thereby controlling the vehicle.

[0043] For example, when the camera detects a slippery or uneven road surface, the electronic device can automatically adjust the vehicle speed to prevent stalling or overspeeding due to changes in road resistance; when the camera recognizes traffic signs and signal lights, the electronic device can automatically adjust the vehicle speed according to the traffic signs and signal lights to ensure that the vehicle travels within different speed limit areas and avoid speeding; when the camera detects an obstacle on the road, the electronic device can adjust the vehicle speed according to the distance and relative speed to avoid sudden deceleration or acceleration, thereby avoiding stalling or overspeeding caused by sudden acceleration or deceleration.

[0044] As can be seen from the above, image acquisition sensors can not only provide clear real-time image data, but also help to quickly analyze and judge road conditions. These data are crucial for the vehicle to adjust speed and power supply strategy during driving, and can effectively prevent stalling and overspeeding problems, and improve driving safety and comfort.

[0045] 2. Motor voltage and current sensor: The motor voltage and current sensor is used to monitor the current and voltage of the vehicle motor. The motor voltage and current sensor can use a Hall effect sensor to monitor the voltage and current of the motor. The Hall effect sensor can work stably in the temperature range of -40°C to 85°C, with a measurement accuracy of ±0.5%. Its voltage measurement range is 0-800V and the current measurement range is 0-300A, which is sufficient to cover the operating status of the motor under various working conditions. The motor voltage sensor can collect data a preset number of times per second (such as 100 times), ensuring real-time monitoring and timely response to the working status of the motor.

[0046] The Hall Effect Sensor uses the Hall Effect principle to efficiently and accurately measure the voltage and current changes of the motor. The high-frequency data acquisition capability of the Hall Effect Sensor enables the electronic equipment to quickly identify small fluctuations in the operation of the motor and make corresponding adjustments in time to ensure the stable operation of the motor. The built-in noise filtering function can effectively eliminate electromagnetic interference, ensure the high accuracy and reliability of the collected data, and provide accurate data support even in complex electromagnetic environments.

[0047] Electronic devices can evaluate the load and operating efficiency of the motor by analyzing the voltage and current. For example, when the vehicle accelerates, decelerates, or encounters a steep slope, the voltage and current of the motor will change significantly. Sensors can capture these changes in real time. Electronic devices can dynamically adjust the distribution of power based on the data provided by the sensors, optimize the operating status of the motor, and prevent stalling due to insufficient power supply or overspeed due to excessive power supply.

[0048] Motor voltage and current sensors can provide important support for optimizing motor operation and stabilizing vehicle speed. Electronic devices can use sensor data collected by motor voltage and current sensors to accurately control and manage motor operation, improving the overall performance and safety of new energy vehicles.

[0049] 3. Driving speed sensor: The driving speed sensor is used to monitor the driving speed of the vehicle. The driving speed sensor can use a high-precision wheel speed sensor and a positioning module (such as a GPS module) to monitor the vehicle's driving speed. The wheel speed sensor uses magnetoresistive effect technology, and the measurement accuracy can reach ±0.1 km / h, which can accurately calculate the rotation speed of the wheel. The GPS module can update 10 times per second, the position accuracy can reach ±2 meters, and the speed measurement accuracy can reach ±0.05 m / s, which can provide stable speed data in various geographical environments.

[0050] The dual monitoring method of wheel speed sensor and positioning module can not only improve the accuracy of speed measurement, but also supplement the data by GPS module when the wheel speed sensor fails, thus ensuring the continuity and reliability of vehicle speed monitoring.

[0051] 4. Power parameter sensor: The power parameter sensor is used to monitor the power parameters of the vehicle battery, such as voltage, current, temperature and remaining power. The power parameter sensor can adopt a battery management system (BMS) sensor, which can monitor the battery voltage, current, temperature and remaining power in real time.

[0052] The voltage measurement accuracy can reach ±0.1V, the current measurement range can reach 0 to 500A, and the temperature measurement range can reach -40°C to 125°C.

[0053] The BMS sensor can use multi-point temperature monitoring to accurately detect the temperature distribution within the battery pack. The data acquisition frequency can be set to 1000 times / second to ensure that the system can obtain detailed status information of the battery in real time.

[0054] The BMS sensor also has a battery fault warning function, which can detect and prevent abnormal conditions such as battery overheating and overcharging in advance to ensure the safe operation of the vehicle.

[0055] Through the selection of these high-precision and high-reliability sensors, the system can achieve comprehensive monitoring and accurate analysis of the driving status of new energy vehicles. The image acquisition sensor provides environmental perception data, the motor voltage and current sensor monitors the motor working status, the power parameter sensor ensures the stability of the battery power supply, and the driving speed sensor provides real-time feedback of the vehicle's speed information.

[0056] The fusion of these sensor data enables electronic devices to effectively predict and control the vehicle's power supply and speed, prevent stalling and overspeeding problems, and improve the safety and driving experience of new energy vehicles.

[0057] In some embodiments, after acquiring the above-mentioned various sensor data, these sensor data may be preprocessed to obtain preprocessed sensor data for subsequent processing.

[0058] The preprocessing operation includes at least one of denoising, principal component analysis and normalization.

[0059] For example, electronic devices can filter data using a Kalman filter to remove noise from sensor data and improve data accuracy. Then, electronic devices can normalize the data using a Min-Max scaling method to convert sensor data to a uniform scale for subsequent processing. Next, electronic devices can also use techniques such as principal component analysis (PCA) to extract key features, reduce data dimensions, and improve computing efficiency.

[0060] In addition to the above preprocessing operations, electronic devices can also perform operations such as de-meaning, cross-modal normalization, and regularization on sensor data.

[0061] Step 202: fusing the data of various sensors to obtain multi-modal fusion data.

[0062] In some embodiments, step 202 may include: aligning multiple sensor data, such as pre-processed images, motor voltage and current, battery parameters, and driving speed data, to form multi-modal fusion data.

[0063] For example, the electronic device obtains the generation time of the multiple sensor data; aligns the multiple sensor data according to the generation time to obtain the multimodal fusion data.

[0064] The purpose of aligning the data by the generation time of the sensor is to ensure that the sensor data of different modes can correspond to each other in dimensions such as time.

[0065] That is, if sensors of different modes have accurate timestamps when collecting data, electronic devices can use the timestamps as the time when the sensor data was generated and align the data based on these timestamps. For example, in a vehicle system, the data collected by the camera and radar have their own time records. By finding data points with the same timestamp or within a certain time interval, they are combined as multimodal data at the same time to obtain multimodal fusion data.

[0066] The above embodiments align the various sensor data based on time. In other embodiments, the various sensor data may also be aligned based on space.

[0067] For example, different sensors may have different coordinate systems, such as the pixel coordinate system of camera images and the polar coordinate system of radar. Electronic devices can convert these different coordinate systems into a unified reference coordinate system.

[0068] Taking the camera and forward radar on the vehicle as an example, assuming that a three-dimensional spatial coordinate system is established with the vehicle as the reference center, for the target object in the camera image, the pixel coordinates can be converted into physical coordinates in the vehicle coordinate system through camera calibration (using known camera intrinsic and extrinsic parameters); for the target detected by the radar, its own data can be converted to the vehicle coordinate system through a certain conversion formula (taking into account factors such as the radar installation position and angle), so that the two can correspond in space.

[0069] In some embodiments, a particle filter algorithm may be used for filtering during the fusion process of multiple sensor data. The embodiment of the present application uses particle filtering to process nonlinear and non-Gaussian distribution state estimation, which can enhance the adaptability of vehicle state prediction to environmental changes. Step 203 , predicting the vehicle state of the vehicle based on the multimodal fusion data and a preset vehicle state prediction model to obtain a predicted vehicle state.

[0070] The predicted vehicle state can be used to indicate whether the vehicle has a driving safety risk.

[0071] For example, the predicted vehicle state can be used to indicate whether there is an abnormality in the vehicle speed and the type of abnormality, which can be vehicle stalling or vehicle overspeeding.

[0072] The vehicle state prediction model can be created based on the Deep Belief Network (DBN). The Deep Belief Network is a generative model of deep learning. It is composed of multiple Restricted Boltzmann Machines (RBMs) stacked together. Each RBM has a visible layer and a hidden layer, and the layers are connected by weights. During the training process, these RBMs are trained layer by layer through unsupervised learning. After pre-training, supervised learning can be used for fine-tuning.

[0073] That is, the deep belief network has a multi-layer structure, and each layer can be regarded as a feature extractor. The training process of the deep belief network can adopt unsupervised pre-training (for example, using the contrastive divergence algorithm to train each RBM in the DBN), and then supervised fine-tuning can be used to complete the training of the entire network. This training method helps to alleviate the gradient vanishing problem in the training of deep neural networks.

[0074] In the embodiment of the present application, DBN establishes an association model between various sensor data and predicted vehicle status through unsupervised learning and layer-by-layer pre-training.

[0075] For example, during vehicle driving, the DBN model analyzes the input multimodal fusion data in real time to predict possible stalling or overspeeding.

[0076] When the predicted vehicle status indicates that the vehicle is at risk of stalling or overspeeding, the vehicle may issue a warning prompt, for example, using an on-board display or an on-board speaker.

[0077] In some embodiments, reference Figure 3 As shown, after obtaining the predicted vehicle state, the electronic device may also execute the following steps 204 to 205.

[0078] Step 204 , determining a power control strategy for the vehicle based on the predicted vehicle state.

[0079] In some embodiments, step 204 may be implemented as follows: the electronic device determines the abnormal speed type of the vehicle based on the predicted vehicle state; and determines the motor control strategy and the battery control strategy based on the abnormal speed type.

[0080] The electronic device may generate a motor control strategy and the battery control strategy (referred to as a combined control strategy) based on the predicted vehicle state.

[0081] Furthermore, the electronic device can also dynamically adjust the vehicle's joint control strategy to regulate the vehicle speed based on the current vehicle status and environmental data (such as vehicle speed, battery power, traffic signs, road conditions, etc.).

[0082] For example, if the predicted vehicle state indicates that the vehicle is at risk of stalling, the electronics can boost the motor power and accelerate appropriately.

[0083] In situations where the predicted vehicle state indicates that the vehicle is at risk of overspeeding, the electronics can ensure that the vehicle travels within a safe range by limiting battery output and decelerating the vehicle.

[0084] Environmental feedback will also affect the optimization of control strategies. The system further adjusts the vehicle's driving speed and trajectory through real-time collected traffic information, weather conditions, road conditions and other environmental data to ensure safe and smooth driving. Through continuous feedback and adaptive adjustment, the system can optimize control strategies in a constantly changing environment, improve vehicle operating efficiency and safety, and achieve intelligent adaptive control.

[0085] In an embodiment of the present application, the electronic device can generate a joint control strategy for the motor and the battery based on the output of the multimodal fusion data and the intelligent algorithm, so that the motor and the battery can work in coordination to achieve fast and precise control. The joint control strategy can adapt to different driving conditions and optimize vehicle performance.

[0086] Step 205, controlling the vehicle based on the power control strategy.

[0087] For example, electronic devices can combine power control strategies and use proportional-integral-derivative (PID) control to control the battery or motor of the vehicle. For example, the PID algorithm can be used to control the motor torque, battery output current, voltage or power, etc.

[0088] The embodiment of the present application comprehensively analyzes data from sensors such as images, motor voltage and current, power supply parameters, and driving speed, and uses a deep belief network to perform regression prediction to achieve a comprehensive assessment of the driving status of new energy vehicles.

[0089] After being collected, various sensor data are preprocessed, formatted in a unified format, and noise removed to ensure data accuracy and consistency. The DBN model learns these preprocessed multimodal data to establish a correlation model between the vehicle's operating status and stalling and overspeeding problems (i.e., the vehicle status prediction model), thereby predicting and judging potential risks in real time during actual operation, and dynamically adjusting the vehicle's power supply and speed strategy to prevent stalling or overspeeding problems caused by unstable power supply, thereby improving the vehicle's operating safety and stability.

[0090] In some embodiments, after the electronic device predicts the vehicle state of the vehicle based on the multimodal fusion data and a preset vehicle state prediction model and obtains the predicted vehicle state, it can also obtain the actual vehicle state of the vehicle; and adjust the model parameters of the vehicle state prediction model based on the actual vehicle state and the predicted vehicle state.

[0091] For example, this embodiment can compare the actual vehicle state with the predicted state, and automatically adjust and optimize the parameters of the DBN model according to the comparison results to improve the adaptability to complex driving environments and long-term stability.

[0092] This embodiment can also adjust the DBN model through a self-learning mechanism. For example, during operation, by continuously analyzing newly collected data, the system uses an incremental learning method (such as an online support vector machine (SVM)) to update the DBN model in real time to reflect the changing driving environment and vehicle status.

[0093] The above process enables the system to automatically optimize its control strategy without human intervention. Through adaptive control technology, the system can dynamically adjust control parameters based on new operational feedback and environmental information, and continuously improve control effects to ensure the stability and reliability of the system in long-term operation. In addition, incremental learning can also help the system gradually improve its accuracy and response speed in a changing environment and optimize overall control performance. This mechanism enables the system to have the ability to self-optimize in the long term, thereby improving the adaptability and control effect of the vehicle in different driving scenarios.

[0094] The implementation of this application can integrate image sensors, current and voltage sensors, vehicle speed sensors and positioning data to achieve comprehensive perception of vehicle status and environment, greatly improve the adaptability of vehicle status prediction to complex environments, and overcome the limitations of traditional methods that rely on a single information source, thereby facilitating the provision of more precise control. The implementation of this application also adopts a modular design in structure, integrating multiple sensors (such as image sensors, current and voltage sensors, vehicle speed sensors and positioning modules) on a unified platform, thereby not only reducing interference between sensors, but also improving the synchronization and efficiency of data acquisition. Combined with high-performance computing platforms and edge computing technology, it can process large amounts of sensor data in real time, ensuring that data preprocessing and feature extraction processes are fast and accurate.

[0095] The embodiment of the present application also uses a deep belief network to predict vehicle status. The deep belief network can analyze the relationship between vehicle operating status and potential problems in real time through unsupervised learning and layer-by-layer pre-training. This method is more intelligent and adaptive than traditional fuzzy control and PID control, and can more accurately predict abnormal situations such as stalling and overspeeding.

[0096] In actual operation, the DBN model can predict the speed change trend of the vehicle in real time, and dynamically adjust the vehicle's power supply strategy and driving speed based on the prediction results to avoid stalling or overspeeding.

[0097] The electronic equipment automatically adjusts and optimizes the parameters of the DBN model by real-time monitoring and comparing actual data with predicted results, thereby improving the system's adaptability to complex driving environments.

[0098] The embodiment of the present application combines the prediction results of deep learning with real-time data, and the system can dynamically adjust the operating modes of the motor and battery to generate an efficient joint control strategy. This adaptability performs well in dealing with complex driving conditions and significantly improves the vehicle's dynamic response capabilities and energy utilization efficiency.

[0099] The embodiment of the present application may also adopt a fault warning mechanism. Once an abnormal vehicle speed is detected, the system can immediately adjust the control strategy and issue an alarm. The fault warning mechanism can enhance the safety of the vehicle and reduce potential risks.

[0100] The embodiment of the present application can also use Kalman filtering, Min-Max scaling and principal component analysis (PCA) to pre-process various sensor data to improve data accuracy and processing efficiency. This efficient data pre-processing method lays a solid foundation for subsequent multimodal fusion and intelligent analysis.

[0101] In summary, the present invention realizes efficient joint control of the motor and battery of new energy vehicles through innovations such as multimodal information fusion, application of intelligent algorithms, adaptive control strategies and fault warning mechanisms, and significantly improves vehicle performance and safety.

[0102] Based on the same idea as the vehicle state prediction method in the above embodiment, the present application also provides a vehicle state prediction device, which can be used to execute the above vehicle state prediction method. For ease of explanation, the structural schematic diagram of the vehicle state prediction device embodiment only shows the parts related to the embodiment of the present application. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.

[0103] like Figure 4 As shown, the vehicle state prediction device includes a collection module 401, a fusion module 402 and a prediction module 403. In some embodiments, the above modules can be programmable software instructions stored in a memory and can be called and executed by a processor. It is understood that in other embodiments, the above modules can also be program instructions or firmware fixed in the processor.

[0104] A collection module 401 is used to collect a variety of sensor data from the multiple sensors respectively; A fusion module 402 is used to perform data fusion on the multiple sensor data to obtain multi-modal fusion data; The prediction module 403 is used to predict the vehicle state of the vehicle based on the multimodal fusion data and a preset vehicle state prediction model to obtain a predicted vehicle state.

[0105] Figure 5 This is a schematic diagram of an embodiment of an electronic device of the present application.

[0106] The electronic device 100 includes a memory 20, a processor 30, and a computer program 40 stored in the memory 20 and executable on the processor 30. When the processor 30 executes the computer program 40, the steps in the above vehicle prediction method embodiment are implemented, for example Figure 2 Steps 201 to 203 shown, or, as Figure 3 Steps 201 to 205 are shown.

[0107] Exemplarily, the computer program 40 can also be divided into one or more modules / units, which are stored in the memory 20 and executed by the processor 30. The one or more modules / units can be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program 40 in the electronic device 100. For example, it can be divided into Figure 4 The collection module 401, the fusion module 402 and the prediction module 403 are shown.

[0108] Those skilled in the art will appreciate that the schematic diagram is merely an example of the electronic device 100 and does not constitute a limitation on the electronic device 100. The electronic device 100 may include more or fewer components than shown in the diagram, or a combination of certain components, or different components. For example, the electronic device 100 may also include input and output devices, network access devices, buses, etc.

[0109] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, a single-chip microcomputer, or the processor 30 may also be any conventional processor, etc.

[0110] The memory 20 can be used to store the computer program 40 and / or the module / unit. The processor 30 realizes various functions of the electronic device 100 by running or executing the computer program and / or the module / unit stored in the memory 20 and calling the data stored in the memory 20. The memory 20 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data (such as audio data) created according to the use of the electronic device 100, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0111] If the module / unit integrated in the electronic device 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed electronic device and method can be implemented in other ways. For example, the electronic device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation.

[0113] In addition, each functional unit in each embodiment of the present application may be integrated into the same processing unit, or each unit may exist physically separately, or two or more units may be integrated into the same unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0114] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or electronic devices stated in the electronic device claim can also be implemented by the same unit or electronic device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A vehicle state prediction method, characterized in that: The vehicle is equipped with a plurality of sensors, and the method comprises: collecting a plurality of sensor data from the plurality of sensors respectively; Performing data fusion on the multiple sensor data to obtain multimodal fusion data; The vehicle state of the vehicle is predicted based on the multimodal fusion data and a preset vehicle state prediction model to obtain a predicted vehicle state.

2. The vehicle state prediction method according to claim 1, characterized in that: After predicting the vehicle state of the vehicle based on the multimodal fusion data and the preset vehicle state prediction model to obtain the predicted vehicle state, the method further includes: obtaining an actual vehicle state of the vehicle; Model parameters of the vehicle state prediction model are adjusted based on the actual vehicle state and the predicted vehicle state.

3. The vehicle state prediction method according to claim 1, characterized in that: The step of fusing the multiple sensor data to obtain multimodal fusion data includes: Acquire generation time of the multiple sensor data; The multiple sensor data are aligned according to the generation time to obtain the multimodal fusion data.

4. The vehicle state prediction method according to claim 1, characterized in that: Before fusing the multiple sensor data to obtain multimodal fusion data, the method includes: Performing a preprocessing operation on the multiple sensor data to obtain preprocessed sensor data; wherein the preprocessing operation includes at least one of a denoising operation, a principal component analysis, and a normalization operation; The step of fusing the multiple sensor data to obtain multimodal fusion data includes: The preprocessed sensor data is subjected to data fusion to obtain the multimodal fusion data.

5. The vehicle state prediction method according to claim 1, characterized in that: The vehicle state prediction model is obtained by training a deep belief network.

6. The vehicle state prediction method according to any one of claims 1 to 5, characterized in that: The predicted vehicle state is used to indicate the abnormal speed risk situation of the vehicle; After predicting the vehicle state of the vehicle based on the multimodal fusion data and the preset vehicle state prediction model to obtain the predicted vehicle state, the method further includes: determining a power control strategy for the vehicle based on the predicted vehicle state; The vehicle is controlled based on the power control strategy.

7. The vehicle state prediction method according to claim 6, characterized in that: The power control strategy includes a motor control strategy and a battery control strategy, and the power control strategy of the vehicle is determined based on the predicted vehicle state, including: determining a speed abnormality type of the vehicle based on the predicted vehicle state; The motor control strategy and the battery control strategy are determined based on the abnormal vehicle speed type.

8. A vehicle state prediction device, characterized in that: The vehicle is equipped with a variety of sensors, and the device includes: A collection module, used for collecting a plurality of sensor data from the plurality of sensors respectively; A fusion module, used for fusing the data of the multiple sensors to obtain multi-modal fusion data; A prediction module is used to predict the vehicle state of the vehicle based on the multimodal fusion data and a preset vehicle state prediction model to obtain a predicted vehicle state.

9. An electronic device, comprising a processor and a memory, characterized in that: The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the vehicle state prediction method according to any one of claims 1 to claim 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the vehicle state prediction method according to any one of claims 1 to 7.

Images