A vehicle load estimation method and a vehicle control device

Abstract

The present application provides a vehicle load estimation method and a vehicle control device, relating to the technical field of vehicles. The method includes: obtaining a target environmental image at the current moment collected by an image sensor on a target vehicle and a target power signal at the current moment collected by a chassis domain signal sensor on the target vehicle; using a pre-trained multi-modal load estimation network to perform load prediction based on the target environmental image and the target power signal, so as to obtain a first estimated load value of the target vehicle at the current moment; and determining a target load state of the target vehicle at the current moment according to the first estimated load value, where the target load state is used to adjust a drive control strategy of the target vehicle and to perform endurance estimation on the target vehicle. The present application improves the accuracy of load estimation of the target vehicle, making the estimation result of the load of the target vehicle more accurate.

Description

Technical Field

[0001] The present application relates to the technical field of vehicles, and more particularly, to a vehicle load estimation method and a vehicle control device. Background Art

[0002] As a main tool for logistics transportation and cargo distribution, during the operation of commercial vehicles, the fluctuations in load have a direct impact on the power demand of the vehicles. Accurate load estimation can determine the specific power strategy for commercial vehicles, thereby affecting the endurance of commercial vehicles. However, commercial vehicle load estimation faces challenges such as diverse vehicle types, a wide variety of cargo types, and complex loading states.

[0003] Currently, load sensors are not installed in commercial vehicles. Only by using existing sensors such as pressure, strain, and acceleration sensors in commercial vehicles to collect real-time physical response data of the vehicle under the load state, and then analyzing the physical response data through signal processing and algorithm models, can the real-time load of commercial vehicles be inferred.

[0004] However, only determining the real-time load of commercial vehicles through existing sensors has problems such as different update frequencies of various sensors on commercial vehicles, resulting in errors in sensor measurement results and large delays in power transmission. The confidence level of the vehicle mass estimation result is poor, and the estimation strategy is single, unable to adapt to complex and changeable commercial vehicle operation scenarios. If a load sensor is set separately, it will increase the manufacturing cost of the vehicle. Summary of the Invention

[0005] The purpose of the present application is to provide a vehicle load estimation method and a vehicle control device for the deficiencies in the above-mentioned existing technologies, which can make the estimation result of the target vehicle load more accurate while improving the accuracy of the target vehicle load estimation, and reduce the manufacturing cost of the vehicle.

[0006] To achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:

[0007] In a first aspect, an embodiment of the present application provides a vehicle load estimation method, the method comprising:

[0008] Obtaining a target environmental image at the current moment collected by an image sensor on a target vehicle and a target power signal at the current moment collected by a chassis domain signal sensor on the target vehicle;

[0009] According to the target environmental image and the target power signal, using a pre-trained multi-modal load estimation network to perform load prediction, and obtaining a first estimated load value of the target vehicle at the current moment;

[0010] Determine the target load state of the target vehicle at the current moment according to the first estimated load value, where the target load state is used to adjust the drive control strategy of the target vehicle and estimate the endurance of the target vehicle.

[0011] Optionally, the multi-modal load estimation network includes: an image state module, a signal state module, a connection module, and a mapping output module, where the image state module includes: a first feature extraction network and a first deep learning model, and the signal state module includes: a second feature extraction network and a second deep learning model;

[0012] The step of predicting the load by using the pre-trained multi-modal load estimation network according to the target environment image and the target power signal to obtain the first estimated load value of the target vehicle at the current moment includes:

[0013] Use the first feature extraction network and the second feature extraction network to extract features from the target environment image and the target power signal respectively, to obtain target environment features and target power features;

[0014] Use the first deep learning model and the second deep learning model to perform deep fusion on the target environment features and the target power features respectively, to obtain a first fused vehicle feature and a second fused vehicle feature;

[0015] Use the connection module to splice the first fused vehicle feature and the second fused vehicle feature to obtain a target vehicle feature;

[0016] Use the mapping output module to perform mapping output on the target vehicle feature to obtain the first estimated load value.

[0017] Optionally, the target environment image is: an image sequence of at least two environment images, where the at least two environment images include: the environment image at the current moment and the environment images at at least one moment before the current moment, and the target power signal is: a signal sequence of at least two power signals, where the at least two power signals include: the power signal at the current moment and the power signals at the at least one moment;

[0018] The step of using the first feature extraction network and the second feature extraction network to extract features from the target environment image and the target power signal respectively, to obtain target environment features and target power features includes:

[0019] Use the first feature extraction network to extract features from the at least two environment images to obtain at least two environment features as the target environment features;

[0020] Using the second feature extraction network, extract features from the at least two power signals to obtain at least two power features as the target power features.

[0021] Optionally, the first deep learning model includes: a first long short-term memory network, a first cross-attention network, and the second deep learning model includes: a second long short-term memory network and a second cross-attention network;

[0022] The step of predicting the load by using the pre-trained multi-modal load estimation network according to the target environmental image and the target power signal to obtain the first estimated load value of the target vehicle at the current moment includes:

[0023] Using the first long short-term memory network and the second long short-term memory network, perform time series learning on the at least two environmental image features and the at least two power signal features to obtain an environmental feature sequence and a power feature sequence;

[0024] Using the first cross-attention network, perform cross-attention fusion on the environmental feature sequence and the power feature sequence to obtain the first fused vehicle feature;

[0025] Using the second cross-attention network, perform cross-attention fusion on the power feature sequence and the environmental feature sequence to obtain the second fused vehicle feature.

[0026] Optionally, before the step of predicting the load by using the pre-trained multi-modal load estimation network according to the target environmental image and the target power signal to obtain the first estimated load value of the target vehicle at the current moment, the method further includes:

[0027] Obtain sample data, where the sample data includes: sample environmental images, the sample power signals corresponding to the sample environmental images, and the corresponding labeled load values;

[0028] Train a preset initial multi-modal load estimation network according to the sample data to obtain the multi-modal load estimation network.

[0029] Optionally, before the step of determining the target load state of the target vehicle at the current moment according to the first estimated load value, the method further includes:

[0030] Obtain the real-time longitudinal acceleration of the target vehicle at the current moment;

[0031] Determine whether an acceleration event is triggered by the target vehicle at the current moment according to the real-time longitudinal acceleration;

[0032] If the acceleration event is triggered, obtain the peak opening data of the accelerator pedal on the target vehicle within a preset time window after the acceleration event is triggered;

[0033] According to the real-time longitudinal acceleration and the peak opening data, determine a second estimated load value of the target vehicle at the current moment;

[0034] The determining the target load state of the target vehicle at the current moment according to the first estimated load value includes:

[0035] Determine the target load state according to the first estimated load value and the second estimated load value.

[0036] Optionally, the determining the second estimated load value of the target vehicle at the current moment according to the real-time longitudinal acceleration and the peak opening data includes:

[0037] Perform a division operation on the real-time longitudinal acceleration and the peak opening data to obtain a load coefficient at the current moment;

[0038] According to the load coefficient and a pre-established mapping relationship table between the load coefficient and the load value, determine the load value corresponding to the load coefficient as the second estimated load value.

[0039] Optionally, before the determining the target load state of the target vehicle at the current moment according to the first estimated load value, the method further includes:

[0040] Calculate the longitudinal driving force of the target vehicle at the current moment according to the motor torque value of the target vehicle at the current moment;

[0041] Calculate the longitudinal resultant force of the target vehicle at the current moment according to the real-time longitudinal speed, preset air density, preset wind resistance coefficient, and preset road surface gradient of the target vehicle at the current moment;

[0042] Calculate a third estimated load value of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant force;

[0043] The determining the target load state of the target vehicle at the current moment according to the first estimated load value includes:

[0044] Determine the target load state according to the first estimated load value and the third estimated load value.

[0045] Optionally, the calculating the third estimated load value of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant force includes:

[0046] Calculate a target longitudinal force of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force;

[0047] Calculate the third estimated load value by using a preset kinetic energy theorem equation according to the target longitudinal force, the longitudinal speed at the current moment, and the longitudinal speed at the previous moment.

[0048] Optionally, the method further includes:

[0049] Perform weighted averaging on the third estimated load value at the current moment by using a moving average algorithm according to multiple third estimated load values at at least one time step before the current moment to obtain a filtered estimated load value at the current moment.

[0050] In a second aspect, another embodiment of the present application provides a vehicle load estimation device, where the device includes:

[0051] An acquisition module, configured to acquire a target environment image at the current moment collected by an image sensor on the target vehicle and a target power signal at the current moment collected by a chassis domain signal sensor on the target vehicle;

[0052] A prediction module, configured to perform load prediction by using a pre-trained multimodal load estimation network according to the target environment image and the target power signal to obtain a first estimated load value of the target vehicle at the current moment;

[0053] A determination module, configured to determine a target load state of the target vehicle at the current moment according to the first estimated load value, where the target load state is used to adjust a driving control strategy of the target vehicle and perform endurance estimation on the target vehicle.

[0054] In a third aspect, another embodiment of the present application provides a vehicle control device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the vehicle control device runs, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of the vehicle load estimation method according to any one of the first aspects described above.

[0055] In a fourth aspect, another embodiment of the present application provides a storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the vehicle load estimation method according to any one of the first aspects described above.

[0056] In a fifth aspect, another embodiment of the present application provides a vehicle, which at least includes: the vehicle control device described in the third aspect above.

[0057] The beneficial effects of the present application are as follows:

[0058] In the embodiment of the present application, the target environment image at the current moment collected by the image sensor on the target vehicle and the target power signal at the current moment collected by the chassis domain signal sensor on the target vehicle are obtained. According to the target environment image and the target power signal, the pre-trained multi-modal load estimation network is used for load prediction to obtain the first estimated load value of the target vehicle at the current moment. According to the first estimated load value, the target load state of the target vehicle at the current moment is determined, and the target load state is used to adjust the drive control strategy of the target vehicle and estimate the endurance of the target vehicle. Through the target environment image and the target power signal, the present application ensures the richness of the data acquisition of the target vehicle, comprehensively captures more comprehensive vehicle state information, thereby improving the accuracy of load estimation. Moreover, the present application can extract the key features of the target vehicle from multi-modal data, making the estimation result of the load more accurate and reducing the manufacturing cost of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0060] Figure 1 It is a schematic flowchart of a vehicle load estimation method provided by an embodiment of the present application;

[0061] Figure 2 It is a schematic flowchart of determining the first estimated load value in a vehicle load estimation method provided by an embodiment of the present application;

[0062] Figure 3 It is a schematic flowchart of determining the first estimated load value in another vehicle load estimation method provided by an embodiment of the present application;

[0063] Figure 4 It is a schematic structural diagram of a pre-trained multi-modal load estimation network provided by an embodiment of the present application;

[0064] Figure 5 It is a schematic flowchart of determining the multi-modal load estimation network in a vehicle load estimation method provided by an embodiment of the present application;

[0065] Figure 6 It is a schematic flowchart of determining the target load state in a vehicle load estimation method provided by an embodiment of the present application;

[0066] Figure 7Schematic flowchart of determining a second estimated load value in a vehicle load estimation method provided by an embodiment of the present application;

[0067] Figure 8 Schematic flowchart of determining a target load state in another vehicle load estimation method provided by an embodiment of the present application;

[0068] Figure 9 Longitudinal force analysis diagram of a target vehicle provided by an embodiment of the present application;

[0069] Figure 10 Schematic flowchart of determining a third estimated load in a vehicle load estimation method provided by an embodiment of the present application;

[0070] Figure 11 Schematic structural diagram of a vehicle load estimation device provided by an embodiment of the present application;

[0071] Figure 12 Schematic structural diagram of a vehicle control device provided by an embodiment of the present application;

[0072] Figure 13 Schematic structural diagram of a vehicle provided by an embodiment of the present application. Detailed implementation manners

[0073] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the accompanying drawings in the present application are only for the purposes of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in the present application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without logical context relationships may be reversed or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the present application.

[0074] In addition, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and illustrated in the accompanying drawings here may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the protection scope of the present application.

[0075] It should be noted that in the embodiments of the present application, the term "including" will be used to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.

[0076] To clearly describe a vehicle load estimation method provided by the embodiments of the present application, the method provided by the embodiments of the present application will be described below in conjunction with multiple drawings. The method provided by this embodiment can be executed by a vehicle control device integrated on the target vehicle. Figure 1 It is a schematic flowchart of a vehicle load estimation method provided by the embodiments of the present application. As Figure 1 shown, the method includes:

[0077] Step 101, obtain the target environment image at the current moment collected by the image sensor on the target vehicle and the target power signal at the current moment collected by the chassis domain signal sensor on the target vehicle.

[0078] Among them, the target vehicle can be a pure electric vehicle, a hybrid vehicle, a fuel cell vehicle, etc., and the embodiments of the present application do not limit this. The image sensor mounted on the vehicle-mounted camera collects the target environment image of the target vehicle at the current moment, and the target environment image is used to indicate the road surface information during the driving of the target vehicle. The chassis domain signal sensor can include one or more of a temperature sensor, a pressure sensor, a speed sensor, a position sensor, or a flow sensor. The embodiments of the present application do not limit this. The target power signal at the current moment of the target vehicle is determined by the sensor signals collected by the chassis domain signal sensor. The target power signal can be at least one vehicle power system signal such as a motor speed signal, a motor torque signal, an accelerator pedal position signal, or a brake pedal position signal.

[0079] Optionally, determine the road surface information of the target vehicle at the current moment through the target environment image of the target vehicle collected by the image sensor on the target vehicle. Determine the target power signal of the target vehicle at the current moment through multiple sensor signals collected by the chassis domain signal sensor of the target vehicle.

[0080] Step 102, according to the target environment image and the target power signal, use the pre-trained multi-modal load estimation network to perform load prediction to obtain the first estimated load value of the target vehicle at the current moment.

[0081] Among them, the pre-trained multi-modal load estimation network is a model that can process multi-modal data. The load of the vehicle refers to the weight value that the vehicle can bear under normal driving conditions. The first estimated load value is the vehicle estimated load value obtained by performing load prediction through the pre-trained multi-modal load estimation network. The first estimated load value includes: the empty vehicle weight value of the target vehicle and the additional weight value loaded.

[0082] Optionally, before inputting the target environmental image and the target power signal into the pre-trained multi-modal load estimation network, it is also necessary to clean, denoise, and normalize the target environmental image and the target power signal to ensure the data quality of the target environmental image and the target power signal.

[0083] Optionally, the target environmental image and the target power signal are used as the input of the pre-trained multi-modal load estimation network, and the pre-trained multi-modal load estimation network is used for load prediction to obtain the first estimated load value of the target vehicle at the current moment, realizing the end-to-end load prediction from input to output.

[0084] Step 103: Determine the target load state of the target vehicle at the current moment according to the first estimated load value. The target load state is used to adjust the driving control strategy of the target vehicle and estimate the endurance of the target vehicle.

[0085] Among them, the target load value of the target vehicle is mapped through a preset load value and a load state table to obtain the target load state corresponding to the first estimated load value. Among them, the load state can include: unloaded, half-loaded, and fully loaded. The driving control strategies of the target vehicle corresponding to different load states are different, and the endurance corresponding to different load states is different. When the vehicle is in the unloaded state, the endurance mileage is the basic endurance. When the vehicle is in the half-loaded state, the endurance mileage is less than the basic endurance. When the vehicle is in the fully loaded state, the endurance mileage is the smallest.

[0086] Exemplarily, when the target load state of the target vehicle is the unloaded state, the vehicle is lighter, the response of the vehicle motor is faster, the suspension is relatively firm, the acceleration is faster, the control and operation of the vehicle are more flexible, and the braking is relatively small. When the target load state of the target vehicle is the half-loaded state, the response of the vehicle motor is slightly delayed to avoid excessive acceleration caused by the increase in load, the suspension is adjusted to medium hardness, and the response of the braking system is slightly adjusted. When the target load state of the target vehicle is the fully loaded state, the response of the motor is further delayed, the torque output of the motor is adjusted, the suspension is adjusted to be softer, the suspension system requires a greater automatic distance, the vehicle stability control system is more sensitive, and the automatic transmission may shift gears earlier to reduce the motor load.

[0087] In an embodiment of the present application, a target environmental image at the current moment collected by an image sensor on a target vehicle and a target power signal at the current moment collected by a chassis domain signal sensor on the target vehicle are obtained. According to the target environmental image and the target power signal, a pre-trained multi-modal load estimation network is used to perform load prediction, and a first estimated load value of the target vehicle at the current moment is obtained. According to the first estimated load value, a target load state of the target vehicle at the current moment is determined, and the target load state is used to adjust the drive control strategy of the target vehicle and estimate the endurance of the target vehicle. Through the target environmental image and the target power signal, the present application uses a multi-modal load estimation network to realize the prediction of the load value on the basis of fully utilizing the complementarity between multi-modal information, ensuring the richness of the acquisition of target vehicle data, comprehensively capturing more comprehensive vehicle state information, thereby improving the accuracy of load estimation and making the estimation result of the load more accurate. At the same time, the solution of the present application can estimate the vehicle mass in real time and accurately without increasing the number of sensors, only based on existing sensors such as image sensors and chassis domain signal sensors.

[0088] On the basis of the above embodiment, the multi-modal load estimation network includes: an image state module, a signal state module, a connection module, and a mapping output module; wherein, the image state module includes: a first feature extraction network, a first deep learning model, and the signal state module includes: a second feature extraction network, a second deep learning model. The present application also provides a process for determining the first estimated load value in a vehicle load estimation method. Figure 2 It is a schematic flow diagram for determining the first estimated load value in a vehicle load estimation method provided by an embodiment of the present application, as Figure 2 shown, in step 102 above, according to the target environmental image and the target power signal, a pre-trained multi-modal load estimation network is used to perform load prediction, and a first estimated load value of the target vehicle at the current moment is obtained, including:

[0089] Step 201: Use the first feature extraction network and the second feature extraction network to respectively extract features from the target environmental image and the target power signal, and obtain target environmental features and target power features.

[0090] Among them, the first feature extraction network and the second feature extraction network are convolutional neural networks. The convolutional neural network can be a ResNet architecture network. The target environmental features describe the color distribution features, texture features, shape features, etc. of different regions in the image, and the embodiments of the present application do not limit this. The target power features may include: information such as the speed feature, acceleration feature, and force feature of the target vehicle, and the embodiments of the present application do not limit this.

[0091] Step 202: Use the first deep learning model and the second deep learning model to perform deep fusion on the target environmental features and the target power features respectively, to obtain the first fused vehicle features and the second fused vehicle features.

[0092] Optionally, use the first deep learning model to process the target environmental features, and fuse the target power features into the target environmental features to obtain the first fused vehicle features; after using the second deep learning model to process the target power features, fuse the target environmental features into the target power features to obtain the second fused vehicle features.

[0093] Step 203: Use a connection module to splice the first fused vehicle features and the second fused vehicle features to obtain the target vehicle features.

[0094] Optionally, use a connection module to splice the first fused vehicle features and the second fused vehicle features by methods such as superposition or multiplication to obtain the target vehicle features. The embodiments of the present application do not limit the specific splicing method.

[0095] Step 204: Use a mapping output module to perform mapping output on the target vehicle features to obtain the first estimated load value.

[0096] Among them, the mapping output module is a mapping output module based on an activation function. Specifically, it can be a mapping output module based on the Sigmoid activation function.

[0097] Optionally, use a mapping output module to map the target vehicle features based on the Sigmoid activation function to obtain the first estimated load value.

[0098] In the embodiments of the present application, feature extraction is respectively performed on the target environmental image and the target power signal to obtain the target environmental features and the target power features. Deep fusion is respectively performed on the target environmental features and the target power features to obtain the first fused vehicle features and the second fused vehicle features. The first fused vehicle features and the second fused vehicle features are spliced to obtain the target vehicle features, and mapping output is performed on the target vehicle features to obtain the first estimated load value. By simultaneously processing image and signal data, the model can learn the internal relationships of different modality data, make full use of the complementarity between multi-modal information, effectively fuse the features from different sensors, enhance the generalization ability and robustness of the model, and improve the accuracy of model prediction.

[0099] Based on the above embodiments, the target environmental image is: an image sequence of at least two environmental images, where the at least two environmental images include: the environmental image at the current moment and the environmental images at at least one moment before the current moment, and the target power signal is: a signal sequence of at least two power signals, where the at least two power signals include: the power signal at the current moment and the power signals at at least one moment. For this reason, the present application also provides a process for determining the target environmental feature and the target power feature in a vehicle load estimation method. In step 201 above, a first feature extraction network and a second feature extraction network are used to extract features from the target environmental image and the target power signal respectively, to obtain the target environmental feature and the target power feature, including:

[0100] Use the first feature extraction network to extract features from at least two environmental images to obtain at least two environmental features as the target environmental feature.

[0101] Optionally, by using the first feature extraction network to extract features from at least two environmental images, the environmental information of the target vehicle at multiple moments can be obtained, so as to obtain the target environmental feature of the target vehicle at different moments.

[0102] Use the second feature extraction network to extract features from at least two power signals to obtain at least two power features as the target power feature.

[0103] Optionally, by using the second feature extraction network to extract features from at least two power signals, the power information of the target vehicle at multiple moments can be obtained, and the target power feature of the target vehicle at different moments can be obtained.

[0104] In the embodiment of the present application, the first feature extraction network is used to extract features from at least two environmental images to obtain at least two environmental features as the target environmental feature, and the second feature extraction network is used to extract features from at least two power signals to obtain at least two power features as the target power feature, thereby increasing the dimensions of the environmental feature and the power feature and enhancing the robustness of the model.

[0105] Based on the above embodiments, the first deep learning model includes: a first Long-Short-Term Memory (LSTM) network and a first Cross Attention network, and the second deep learning model includes: a second Long-Short-Term Memory network and a second Cross Attention network. The present application also provides a process for determining the first estimated load value in another vehicle load estimation method. Figure 3 It is a schematic diagram of the process for determining the first estimated load value in another vehicle load estimation method provided by the embodiment of the present application, as Figure 3As shown, in the above step 102, according to the target environmental image and the target power signal, a pre-trained multi-modal load estimation network is used to predict the load, and a first estimated load value of the target vehicle at the current moment is obtained, including:

[0106] Step 301: Use the first long short-term memory network and the second long short-term memory network to perform time series learning on at least two environmental image features and at least two power signal features to obtain an environmental feature sequence and a power feature sequence.

[0107] Among them, the first long short-term memory network is used to perform time series learning on at least two environmental image features to obtain an environmental feature sequence. The environmental feature sequence refers to a series of data arranged in a certain order and used to describe environmental-related features. The environmental feature sequence reflects the state changes of the environment at different time points or different spatial positions. The second long short-term memory network is used to perform time series learning on at least two power signal features to obtain a power feature sequence. The power feature sequence refers to a series of data arranged in a specific order and describing the features related to the power system. The power feature sequence reflects the dynamic changes of these systems during operation.

[0108] Optionally, at least two environmental image features are respectively used as the input of the first long short-term network, and the first long short-term network is used to perform time series learning on at least two environmental image features respectively to obtain an environmental feature sequence. At least two power signal features are respectively used as the input of the second long short-term network, and the second long short-term network is used to perform time series learning on at least two power signal features respectively to obtain a power feature sequence.

[0109] Step 302: Use the first cross-attention network to perform cross-attention fusion on the environmental feature sequence and the power feature sequence to obtain a first fused vehicle feature.

[0110] Optionally, the first cross-attention network is used to incorporate the influence of the power feature sequence on the environmental feature sequence into the environmental feature sequence to obtain a first fused vehicle feature.

[0111] Step 303: Use the second cross-attention network to perform cross-attention fusion on the power feature sequence and the environmental feature sequence to obtain a second fused vehicle feature.

[0112] Optionally, the second cross-attention network is used to incorporate the influence of the environmental feature sequence on the power feature sequence into the power feature sequence to obtain a second fused vehicle feature.

[0113] Exemplarily, Figure 4 is a schematic structural diagram of a pre-trained multi-modal load estimation network provided by an embodiment of the present application, as Figure 4As shown in the figure, the multi-modal load estimation network includes: an image module, a signal module, a connection module, and a mapping output module. The image module includes: a first feature extraction network and a first deep learning model. The signal module includes: a second feature extraction network and a second deep learning model. The first deep learning model includes: a first long short-term memory network and a first cross-attention network. The second deep learning model includes: a second long short-term memory network and a second cross-attention network. Among them, the first feature extraction network and the second feature extraction network are respectively used to extract features from the target environment image and the target dynamic signal to obtain the target environment feature and the target dynamic feature. The first long short-term memory network and the second long short-term memory network are used to perform time series learning on at least two environmental image features and at least two dynamic signal features to obtain an environmental feature sequence and a dynamic feature sequence. The first cross-attention network is used to perform cross-attention fusion on the environmental feature sequence and the dynamic feature sequence to obtain a first fusion vehicle feature. The second cross-attention network is used to perform cross-attention fusion on the dynamic feature sequence and the environmental feature sequence to obtain a second fusion vehicle feature. The connection module is used to splice the first fusion vehicle feature and the second fusion vehicle feature to obtain a target vehicle feature. The mapping output module is used to perform mapping output on the target vehicle feature to obtain a first estimated load value.

[0114] In the embodiment of the present application, the environmental feature sequence and the dynamic feature sequence are obtained through the long short-term memory network, and the first fusion vehicle feature and the second fusion vehicle feature are obtained through the cross-attention network. The first fusion vehicle feature and the second fusion vehicle feature obtained in the present application can make full use of the complementary information of the environmental feature sequence and the dynamic feature sequence, thereby enhancing the description of the vehicle feature and obtaining a more accurate fusion feature.

[0115] On the basis of the above embodiment, the present application also provides a process for determining a multi-modal load estimation network in a vehicle load estimation method. Figure 5 As shown in the figure, it is a schematic flow chart of determining a multi-modal load estimation network in a vehicle load estimation method provided by an embodiment of the present application. Figure 5 As shown in the figure, before the above step 102, according to the target environment image and the target dynamic signal, a pre-trained multi-modal load estimation network is used to perform load prediction to obtain a first estimated load value of the target vehicle at the current moment, the method further includes:

[0116] Step 501, obtain sample data.

[0117] Among them, the sample data includes: sample environment images, sample dynamic signals corresponding to the sample environment images, and corresponding labeled load values.

[0118] Optionally, collect the target environment image through an image sensor and the target power signal through the chassis domain signal sensor on the target vehicle, and determine the load value of the target vehicle, and label the load value.

[0119] Step 502: Train a preset initial multi-modal load estimation network according to the sample data to obtain a multi-modal load estimation network.

[0120] Optionally, train a preset initial multi-modal load estimation network according to the sample data to obtain a predicted load value, and adjust the parameters of the preset initial multi-modal load estimation network according to the predicted load value and the labeled load value to obtain a multi-modal load estimation network. Specifically, during the training process of the preset initial multi-modal load estimation network, reinforcement model technologies such as data augmentation and regularization are used to prevent the model from overfitting, and the training process of the model is optimized by dynamically adjusting the learning rate and applying an early stopping strategy.

[0121] In the embodiment of the present application, a preset initial multi-modal load estimation network is trained according to the sample data to obtain a multi-modal load estimation network. The present application uses a sample data set with exact labeled load values to train the model, which can improve the prediction accuracy and generalization ability of the model. And the preset initial multi-modal load estimation network can make full use of the complementarity between these different modal information during the training process. Thus, the relationship between the vehicle operating state and the load can be understood more comprehensively, thereby improving the accuracy of load estimation.

[0122] Based on the above embodiment, the present application also provides a process for determining the target load state in a vehicle load estimation method. Figure 6 It is a schematic flow diagram for determining the target load state in a vehicle load estimation method provided by an embodiment of the present application. As Figure 6 shown, before determining the target load state of the target vehicle at the current moment according to the first estimated load value in step 103 above, the method further includes:

[0123] Step 601: Obtain the real-time longitudinal acceleration of the target vehicle at the current moment.

[0124] Optionally, by setting a wheel speed sensor on the target vehicle, determine the vehicle speed of the target vehicle at the current moment and the previous moment according to the wheel speed sensor, so as to obtain the real-time longitudinal acceleration of the target vehicle at the current moment. Among them, the real-time longitudinal acceleration refers to the change in speed per unit time of the target vehicle in the driving direction.

[0125] Step 602: Determine whether the target vehicle triggers an acceleration event at the current moment according to the real-time longitudinal acceleration.

[0126] Optionally, when the real-time longitudinal acceleration is greater than the preset longitudinal acceleration threshold, it is determined that the target vehicle triggers an acceleration event at the current moment. When the real-time longitudinal acceleration is less than or equal to the preset longitudinal acceleration threshold, it is determined that the target vehicle does not trigger an acceleration event at the current moment. Among them, the preset longitudinal acceleration threshold is set according to the vehicle type and performance of the target vehicle to ensure that acceleration events can be accurately captured during normal driving operations.

[0127] Step 603: If an acceleration event is triggered, obtain the peak opening data of the accelerator pedal on the target vehicle within a preset time window after the acceleration event is triggered.

[0128] Among them, the preset time window can be set to 5 seconds. The peak opening data of the accelerator pedal is the data record indicating the maximum degree to which the accelerator pedal is depressed. The accelerator pedal opening represents the intensity of the driver's power request for the vehicle. The larger the opening, the greater the power the driver expects the vehicle to output.

[0129] Optionally, when the real-time longitudinal acceleration is greater than the preset longitudinal acceleration threshold, it is determined that the target vehicle triggers an acceleration event at the current moment. Then, obtain the accelerator pedal opening data of the target vehicle within a preset time window after the acceleration event is triggered, and record the maximum value of the accelerator pedal opening data within the preset time window as the peak opening data.

[0130] Step 604: Determine the second estimated load value of the target vehicle at the current moment according to the real-time longitudinal acceleration and the peak opening data.

[0131] As in step 103 above, according to the first estimated load value, determine the target load state of the target vehicle at the current moment, including:

[0132] Step 605: Determine the target load state according to the first estimated load value and the second estimated load value.

[0133] Optionally, determine the target load value according to the average value of the first estimated load value and the second estimated load value, and then determine the target load state according to the target load value; or perform a weighted sum operation on the first estimated load value and the second estimated load value to obtain the target load value, and then determine the corresponding target load state according to the target load value.

[0134] In the embodiments of the present application, the second estimated load value is determined by obtaining the real-time longitudinal acceleration of the target vehicle at the current moment, and the target load state is determined according to the first estimated load value and the second estimated load value. The present application can more comprehensively and accurately reflect the actual load state of the vehicle, avoid large errors in a single estimation method under special working conditions, can complement and verify each other, and reduce the errors and uncertainties that may exist in a single strategy.

[0135] Based on the above embodiments, the present application further provides a process for determining a second estimated load value in a vehicle load estimation method. Figure 7 It is a schematic flowchart for determining a second estimated load value in a vehicle load estimation method provided by an embodiment of the present application. As Figure 7 shown, in step 604 above, according to the real-time longitudinal acceleration and peak opening data, determining the second estimated load value of the target vehicle at the current moment includes:

[0136] Step 701: Perform a division operation on the real-time longitudinal acceleration and peak opening data to obtain the load coefficient at the current moment.

[0137] Optionally, use the real-time longitudinal acceleration as the numerator and the peak opening data as the denominator to perform a division operation to obtain the load coefficient at the current moment. Among them, the load coefficient reflects the acceleration ability actually generated by the vehicle under the given accelerator pedal opening data, and is proportional to the vehicle load state.

[0138] Step 702: According to the load coefficient and the pre-established mapping relationship table between the load coefficient and the load value, determine the load value corresponding to the load coefficient as the second estimated load value.

[0139] Among them, the mapping relationship table between the load coefficient and the load value is based on a large amount of experimental data and records the corresponding relationship of the ratio of the real-time longitudinal acceleration and peak opening data under different load states.

[0140] In the embodiment of the present application, a division operation is performed on the real-time longitudinal acceleration and peak opening data to obtain the load coefficient at the current moment, and the load value corresponding to the load coefficient is determined as the second estimated load value. It can quickly obtain the second estimated load value according to the real-time longitudinal acceleration and peak opening data. This strategy is applicable to the working condition where the vehicle starts from a stationary state and generates significant acceleration, ensuring the real-time and accurate requirements of load identification.

[0141] Based on the above embodiments, the present application further provides another process for determining the target load state in a vehicle load estimation method. Figure 8 It is a schematic flowchart for determining the target load state in another vehicle load estimation method provided by an embodiment of the present application. As Figure 8 shown, before determining the target load state of the target vehicle at the current moment according to the first estimated load value in step 103 above, the method further includes:

[0142] Step 801: Calculate the longitudinal driving force of the target vehicle at the current moment according to the motor torque value of the target vehicle at the current moment.

[0143] Optionally, according to the motor output torque value, drive axle speed ratio, current transmission speed ratio, wheel rolling radius, wheel end driving force, and torque transmission efficiency of the target vehicle at the current moment, calculate the longitudinal driving force of the target vehicle at the current moment through formula (1).

[0144] (1)

[0145] Wherein, is the longitudinal driving force of the target vehicle at the current moment, is the motor output torque value of the target vehicle at the current moment, is the drive axle speed ratio, is the current transmission speed ratio, is the wheel rolling radius, is the torque transmission efficiency.

[0146] Step 802: Calculate the longitudinal resultant force of the target vehicle at the current moment according to the real-time longitudinal speed, preset air density, preset wind resistance coefficient, and preset road surface gradient of the target vehicle at the current moment.

[0147] Optionally, during the driving process of the target vehicle, in addition to the driving force, the target vehicle is also affected by various external forces, such as air resistance, rolling resistance, and ramp resistance. External forces such as air resistance, rolling resistance, and ramp resistance constitute the longitudinal resultant force of the target vehicle at the current moment. The longitudinal resultant force at the current moment is related to the real-time longitudinal speed, preset road surface gradient, and vehicle characteristics of the target vehicle at the current moment. Determine the real-time longitudinal speed, real-time longitudinal acceleration, preset road surface gradient, and vehicle characteristic information of the target vehicle through the external sensors on the target vehicle. Thus, determine external forces such as air resistance, rolling resistance, and ramp resistance of the target vehicle according to information such as the real-time longitudinal speed, preset air density, preset wind resistance coefficient, and preset road surface gradient of the target vehicle at the current moment. And determine the longitudinal resultant force of the target vehicle at the current moment according to external forces such as air resistance, rolling resistance, and ramp resistance. Among them, the air resistance is determined by the real-time longitudinal speed, preset air density, preset wind resistance coefficient, and the frontal area of the target vehicle. The rolling resistance is determined by the mass of the target vehicle, gravitational acceleration, preset road surface gradient, and the road surface rolling coefficient at the current moment. The ramp resistance is determined by the mass of the target vehicle, gravitational acceleration, and preset road surface gradient.

[0148] Exemplarily, the longitudinal resultant force in the present application takes air resistance, rolling resistance, and ramp resistance as examples to illustrate the longitudinal resultant force of the target vehicle. Figure 9 is a longitudinal force analysis diagram of a target vehicle provided by an embodiment of the present application. As Figure 9 shown, wherein, the air resistance is , the rolling resistance is , the ramp resistance is , since the directions of the air resistance, rolling resistance, and ramp resistance are the same, the longitudinal resultant force of the target vehicle is determined to be . Among them, is the mass of the vehicle, is the air density, is the wind resistance coefficient, is the frontal area, is the longitudinal speed of the vehicle, is the acceleration due to gravity, is the road surface gradient, is the road surface rolling resistance coefficient.

[0149] Step 803: Calculate the third estimated load value of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant force.

[0150] As in Step 103 above, according to the first estimated load value, determine the target load state of the target vehicle at the current moment, including:

[0151] Step 804: Determine the target load state according to the first estimated load value and the third estimated load value.

[0152] Optionally, determine the target load state according to the average value of the first estimated load value and the third estimated load value; or weight the first estimated load value and the third estimated load value respectively, and determine the target load state according to the weighted sum.

[0153] Optionally, the application determines the target load state according to the average value of the first estimated load value, the second estimated load, and the third estimated load value; or weights the first estimated load value, the second estimated load, and the third estimated load value respectively, and determines the target load state according to the weighted sum.

[0154] In the embodiment of the present application, the third estimated load value is determined by obtaining the real-time longitudinal acceleration of the target vehicle at the current moment. According to the first estimated load value and the third estimated load value, the target load state is determined. The present application does not need to increase the number of sensors, and the target load state can be determined according to the existing number of sensors. Moreover, it can more comprehensively and accurately reflect the actual load state of the vehicle, avoid large errors in a single estimation method under special working conditions, and the first estimated load and the third estimated load can complement and verify each other, reducing the errors and uncertainties that may exist in a single strategy.

[0155] On the basis of the above embodiments, the present application also provides a process for determining the third estimated load in a vehicle load estimation method, Figure 10 which is a schematic diagram of the process for determining the third estimated load in a vehicle load estimation method provided by the embodiment of the present application, as shown in Figure 10As shown, in the above step 803, according to the longitudinal driving force and the longitudinal resultant external force, calculate the third estimated load value of the target vehicle at the current moment, including:

[0156] Step 1001: Calculate the target longitudinal force of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force.

[0157] Optionally, since the directions of the longitudinal driving force and the longitudinal resultant external force are opposite, the difference between the longitudinal driving force and the longitudinal resultant external force is calculated to obtain the target longitudinal force of the target vehicle at the current moment. The target longitudinal force F of the target vehicle at the current moment is calculated by formula (2).

[0158] (2)

[0159] Wherein, is the target longitudinal force of the target vehicle at the current moment, is the longitudinal driving force, is the mass of the vehicle, is the air density, is the wind resistance coefficient, is the frontal area, is the longitudinal speed of the vehicle, is the acceleration due to gravity, is the road surface gradient, is the road surface rolling resistance coefficient.

[0160] Step 1002: Calculate the third estimated load value according to the target longitudinal force, the longitudinal speed at the current moment, and the longitudinal speed at the previous moment by using a preset kinetic energy theorem equation.

[0161] Optionally, the kinetic energy theorem equation can be expressed as formula (3) for example. Substitute the target longitudinal force, the longitudinal speed at the current moment, and the longitudinal speed at the previous moment into the kinetic energy theorem equation to obtain formula (4), thereby obtaining the third estimated load value. Specifically, according to the kinetic energy theorem equation, the load of the target vehicle is expressed as the ratio of the change in kinetic energy to the difference in the squares of the speeds. During the rated time, the displacement of the target longitudinal force is integrated to obtain the change in kinetic energy, and then divided by the difference in the squares of the speeds to obtain the third estimated load value, so as to estimate the load of the target vehicle during the driving of the vehicle, which has high accuracy and practicability.

[0162] (3)

[0163] Wherein, is the target longitudinal force of the target vehicle at the current moment, is at the previous moment from the moment to the current moment is the work done by the target longitudinal force acting on the target vehicle during this period, The current moment respectively The moment and the previous moment The displacement corresponding to the longitudinal speed of the vehicle at the moment, is the speed of the target vehicle at the previous moment, is the speed of the target vehicle at the current moment, is the acceleration of the target vehicle.

[0164] (4)

[0165] The third estimated load value is:

[0166] In the embodiments of the present application, due to problems such as different update frequencies of vehicle sensors, errors in sensor measurement results, and large delays in power transmission, the confidence level of load estimation for the target vehicle is relatively low. By using the kinetic energy theorem equation and calculating the third estimated load value in the form of integration, the influence of instantaneous signal errors and delays on the third estimated load value result is greatly reduced. And by solving the third estimated load value in the form of continuous integration, the influence of instantaneous signal errors and delays on the estimation result of the third estimated load value is greatly reduced, and the load of the target vehicle can be accurately estimated, so as to more accurately judge the dynamic response of the vehicle, make more reasonable control decisions, and improve the power and economic performance of the vehicle.

[0167] Based on the above embodiments, the present application also provides a process for determining the third estimated load in another vehicle load estimation method. On the basis of the above steps 1001-step 1002, this method further includes:

[0168] According to multiple third estimated load values of at least one time step before the current moment, using the moving average algorithm, the third estimated load value at the current moment is weighted and averaged to obtain the filtered estimated load value at the current moment.

[0169] Optionally, according to multiple third estimated load values of at least one time step before the current moment, using the moving average algorithm, the third estimated load value at the current moment is weighted and averaged through formula (5) to obtain the filtered estimated load value at the current moment .

[0170] (5)

[0171] Wherein, is the filtered estimated load value, is the third estimated load value at the current moment, is the third estimated load value at the previous moment, is the filtering coefficient. The filtering coefficient is determined according to the target vehicle, and the embodiments of the present application do not limit this.

[0172] In the embodiments of the present application, the moving average algorithm is used to filter the real-time estimation result. In each time step, the current estimation result is weighted and averaged with the estimation results of several previous time steps to obtain a smoother and more stable estimated value of the vehicle mass. It can effectively reduce the error caused by noise and interference to the estimation result, and improve the reliability and stability of the vehicle mass estimation.

[0173] Based on the same inventive concept, the embodiments of the present application also provide a vehicle load estimation device corresponding to the vehicle load estimation method. Since the principle of solving problems by the device in the embodiments of the present application is similar to the above vehicle load estimation method in the embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0174] Figure 11 is a schematic structural diagram of a vehicle load estimation device provided by the embodiments of the present application. The device includes:

[0175] An acquisition module 1101, configured to acquire the target environment image at the current moment collected by the image sensor on the target vehicle and the target power signal at the current moment collected by the chassis domain signal sensor on the target vehicle;

[0176] A prediction module 1102, configured to perform load prediction using a pre-trained multi-modal load estimation network according to the target environment image and the target power signal, and obtain a first estimated load value of the target vehicle at the current moment;

[0177] A determination module 1103, configured to determine the target load state of the target vehicle at the current moment according to the first estimated load value, and the target load state is used to adjust the drive control strategy of the target vehicle and perform endurance estimation on the target vehicle.

[0178] Optionally, the multi-modal load estimation network includes: an image state module, a signal state module, a connection module, and a mapping output module. Among them, the image state module includes: a first feature extraction network and a first deep learning model, and the signal state module includes: a second feature extraction network and a second deep learning model; the prediction module 1102 is specifically configured to: use the first feature extraction network and the second feature extraction network to respectively perform feature extraction on the target environment image and the target power signal to obtain target environment features and target power features;

[0179] Use the first deep learning model and the second deep learning model to respectively perform deep fusion on the target environment features and the target power features to obtain a first fused vehicle feature and a second fused vehicle feature;

[0180] Adopt a connection module to splice the first fused vehicle feature and the second fused vehicle feature to obtain a target vehicle feature;

[0181] Adopt a mapping output module to map and output the target vehicle feature to obtain a first estimated load value.

[0182] Optionally, the target environmental image is: an image sequence of at least two environmental images, where the at least two environmental images include: the environmental image at the current moment and the environmental images at at least one moment before the current moment, and the target power signal is: a signal sequence of at least two power signals, where the at least two power signals include: the power signal at the current moment and the power signals at at least one moment; The prediction module 1102 is specifically configured to: adopt a first feature extraction network to extract features from at least two environmental images to obtain at least two environmental features as target environmental features.

[0183] And a second feature extraction network to extract features from at least two power signals to obtain at least two power features as target power features.

[0184] Optionally, the first deep learning model includes: a first long short-term memory network and a first cross-attention network, and the second deep learning model includes: a second long short-term memory network and a second cross-attention network; The prediction module 1102 is specifically configured to: adopt the first long short-term memory network and the second long short-term memory network to perform time series learning on at least two environmental image features and at least two power signal features to obtain an environmental feature sequence and a power feature sequence;

[0185] Adopt a first cross-attention network to perform cross-attention fusion on the environmental feature sequence and the power feature sequence to obtain a first fused vehicle feature;

[0186] Adopt a second cross-attention network to perform cross-attention fusion on the power feature sequence and the environmental feature sequence to obtain a second fused vehicle feature.

[0187] Optionally, the prediction module 1102 is further configured to: obtain sample data, where the sample data includes: sample environmental images, sample power signals corresponding to the sample environmental images, and corresponding labeled load values;

[0188] Train a preset initial multi-modal load estimation network according to the sample data to obtain a multi-modal load estimation network.

[0189] Optionally, the determination module 1103 is further configured to: obtain the real-time longitudinal acceleration of the target vehicle at the current moment;

[0190] Determine whether the target vehicle triggers an acceleration event at the current moment according to the real-time longitudinal acceleration;

[0191] If an acceleration event is triggered, obtain the peak opening data of the accelerator pedal on the target vehicle within a preset time window after the acceleration event is triggered;

[0192] According to the real-time longitudinal acceleration and the peak opening data, determine the second estimated load value of the target vehicle at the current moment;

[0193] Optionally, the determining module 1103 is specifically configured to: determine the target load state according to the first estimated load value and the second estimated load value.

[0194] Optionally, the determining module 1103 is specifically configured to: perform a division operation on the real-time longitudinal acceleration and the peak opening data to obtain the load coefficient at the current moment;

[0195] According to the load coefficient and the pre-established mapping relation table between the load coefficient and the load value, determine the load value corresponding to the load coefficient as the second estimated load value.

[0196] Optionally, the determining module 1103 is further configured to: calculate the longitudinal driving force of the target vehicle at the current moment according to the motor torque value of the target vehicle at the current moment;

[0197] Calculate the longitudinal resultant force of the target vehicle at the current moment according to the real-time longitudinal speed, the preset air density, the preset wind resistance coefficient, and the preset road surface gradient of the target vehicle at the current moment;

[0198] Calculate the third estimated load value of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant force;

[0199] Optionally, the determining module 1103 is specifically configured to: determine the target load state according to the first estimated load value and the third estimated load value.

[0200] Optionally, the determining module 1103 is specifically configured to: calculate the target longitudinal force of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant force;

[0201] Calculate the third estimated load value by using the preset kinetic energy theorem equation according to the target longitudinal force, the longitudinal speed at the current moment, and the longitudinal speed at the previous moment.

[0202] Optionally, the determining module 1103 is further configured to: perform a weighted average on the third estimated load value at the current moment by using a moving average algorithm according to multiple third estimated load values at at least one time step before the current moment to obtain the filtered estimated load value at the current moment.

[0203] For the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiments, which will not be elaborated here.

[0204] The embodiments of the present application also provide a vehicle control device. Figure 12 It is a schematic structural diagram of a vehicle control device provided by the embodiments of the present application. As Figure 12 shown, the vehicle control device 1200 includes: a processor 1201, a memory 1202. Optionally, it may further include a bus 1203. The memory 1202 stores machine-readable instructions executable by the processor 1201. When the vehicle control device 1200 runs, the processor 1201 communicates with the memory 1202 through the bus 1203. When the machine-readable instructions are executed by the processor 1201, the steps of the above vehicle load estimation method are executed.

[0205] The embodiments of the present application also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of the above vehicle load estimation method are executed.

[0206] The embodiments of the present application also provide a vehicle. Figure 13 It is a schematic structural diagram of a vehicle provided by the embodiments of the present application. As Figure 13 shown, the vehicle at least includes: the vehicle control device 1200.

[0207] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the method embodiments, which will not be elaborated in the present application. In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or modules can be in electrical, mechanical, or other forms.

[0208] In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, may exist physically alone for each unit, or two or more units may be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, RandomAccess Memory), magnetic disks, or optical discs that can store program codes.

[0209] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application.

Claims

1. A vehicle load estimation method, characterized in that: The method comprises: Acquire a target environment image at the current moment collected by an image sensor on a target vehicle and a target power signal at the current moment collected by a chassis domain signal sensor on the target vehicle; wherein the target environment image is used to indicate road surface information during the driving process of the target vehicle; the target power signal includes: at least one vehicle power system signal of a motor speed signal, a motor torque signal, an accelerator pedal position signal, and a brake pedal position signal; According to the target environment image and the target power signal, a pre-trained multimodal load estimation network is used to perform load prediction to obtain a first estimated load value of the target vehicle at the current moment; Determining a target load state of the target vehicle at the current moment according to the first estimated load value, wherein the target load state is used to adjust a driving control strategy of the target vehicle and estimate the endurance of the target vehicle; The multimodal load estimation network includes: an image state module, a signal state module, a connection module and a mapping output module, wherein the image state module includes: a first feature extraction network and a first deep learning model, and the signal state module includes: a second feature extraction network and a second deep learning model; The method of performing load prediction using a pre-trained multimodal load estimation network according to the target environment image and the target power signal to obtain a first estimated load value of the target vehicle at the current moment includes: Using the first feature extraction network and the second feature extraction network, respectively extracting features from the target environment image and the target power signal to obtain target environment features and target power features; The first deep learning model and the second deep learning model are used to perform deep fusion of the target environment feature and the target power feature, respectively, to obtain a first fused vehicle feature and a second fused vehicle feature; wherein the first deep learning model is used to deeply fuse the target power feature into the target environment feature to obtain the first fused vehicle feature; and the second deep learning model is used to deeply fuse the target environment feature into the target power feature to obtain the second fused vehicle feature; Using the connection module, the first fused vehicle feature and the second fused vehicle feature are spliced ​​to obtain a target vehicle feature; The mapping output module is used to map and output the target vehicle characteristics to obtain the first estimated load value.

2. The method according to claim 1, characterized in that: The target environment image is: an image sequence of at least two environment images, wherein the at least two environment images include: the environment image at the current moment and the environment image at least one moment before the current moment; the target power signal is: a signal sequence of at least two power signals, wherein the at least two power signals include: the power signal at the current moment and the power signal at at least one moment; The first feature extraction network and the second feature extraction network are used to extract features from the target environment image and the target power signal respectively to obtain target environment features and target power features, including: Using the first feature extraction network, extracting features from the at least two environment images to obtain at least two environment features as the target environment features; The second feature extraction network is used to extract features from the at least two power signals to obtain at least two power features as the target power features.

3. The method according to claim 2, characterized in that The first deep learning model includes: a first long short-term memory network and a first cross attention network, and the second deep learning model includes: a second long short-term memory network and a second cross attention network; The first deep learning model and the second deep learning model are used to perform deep fusion on the target environment feature and the target power feature respectively to obtain the first fused vehicle feature and the second fused vehicle feature, including: Using the first long short-term memory network and the second long short-term memory network, time series learning is performed on the at least two environmental image features and the at least two power signal features to obtain an environmental feature sequence and a power feature sequence; Using the first cross-attention network, cross-attention fusion is performed on the environmental feature sequence and the power feature sequence to obtain the first fused vehicle feature; The second cross-attention network is used to perform cross-attention fusion on the power feature sequence and the environmental feature sequence to obtain the second fused vehicle feature.

4. The method according to claim 1, characterized in that: Before the load prediction is performed using a pre-trained multimodal load estimation network according to the target environment image and the target power signal to obtain a first estimated load value of the target vehicle at the current moment, the method further includes: Acquire sample data, the sample data including: a sample environment image, a sample power signal corresponding to the sample environment image, and a corresponding annotated load value; A preset initial multimodal load estimation network is trained according to the sample data to obtain the multimodal load estimation network.

5. The method according to claim 1, characterized in that: Before determining the target load state of the target vehicle at the current moment according to the first estimated load value, the method further includes: Obtaining the real-time longitudinal acceleration of the target vehicle at the current moment; Determining, based on the real-time longitudinal acceleration, whether the target vehicle triggers an acceleration event at the current moment; If the acceleration event is triggered, obtaining peak opening data of the accelerator pedal on the target vehicle within a preset time window after the acceleration event is triggered; Determining a second estimated load value of the target vehicle at the current moment according to the real-time longitudinal acceleration and the peak opening data; Determining the target load state of the target vehicle at the current moment according to the first estimated load value includes: The target load state is determined according to the first estimated load value and the second estimated load value.

6. The method according to claim 5, characterized in that Determining the second estimated load value of the target vehicle at the current moment according to the real-time longitudinal acceleration and the peak opening data includes: Performing a division operation on the real-time longitudinal acceleration and the peak opening data to obtain the load coefficient at the current moment; According to the load coefficient and a pre-established mapping relationship table between the load coefficient and the load value, the load value corresponding to the load coefficient is determined as the second estimated load value.

7. The method according to claim 1, characterized in that Before determining the target load state of the target vehicle at the current moment according to the first estimated load value, the method further includes: Calculating the longitudinal driving force of the target vehicle at the current moment according to the motor torque value of the target vehicle at the current moment; Calculate the longitudinal resultant external force of the target vehicle at the current moment according to the real-time longitudinal speed of the target vehicle at the current moment, the preset air density, the preset drag coefficient, and the preset road slope; Calculating a third estimated load value of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force; Determining the target load state of the target vehicle at the current moment according to the first estimated load value includes: The target load state is determined according to the first estimated load value and the third estimated load value.

8. The method according to claim 7, characterized in that The calculating, according to the longitudinal driving force and the longitudinal resultant external force, a third estimated load value of the target vehicle at the current moment includes: Calculating a target longitudinal force of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force; The third estimated load value is calculated according to the target longitudinal force, the longitudinal speed at the current moment and the longitudinal speed at the previous moment by using a preset kinetic energy theorem equation.

9. A vehicle control device, characterized in that: The vehicle control device includes: a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, and when the vehicle control device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the vehicle load estimation method described in any one of claims 1 to 8.

Images