Vehicle load estimation method and vehicle control equipment

Abstract

The invention provides a vehicle load estimation method and vehicle control equipment, and relates to the technical field of vehicles, and the method comprises the steps: obtaining a target environment image at the current moment collected by an image sensor on a target vehicle and a target power signal at the current moment collected by a chassis domain signal sensor on the target vehicle, according to the target environment image and the target power signal, a pre-trained multi-modal load estimation network is adopted to carry out load prediction, a first estimated load value of the target vehicle at the current moment is obtained, and according to the first estimated load value, a target load state of the target vehicle at the current moment is determined; the target load state is used for adjusting a driving control strategy of the target vehicle and estimating the endurance of the target vehicle. According to the invention, the precision of target vehicle load estimation is improved, and the estimation result of the target vehicle load is more accurate.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a vehicle load estimation method and a vehicle control device. Background Art

[0002] Commercial vehicles are the main tools for logistics transportation and cargo distribution. During the operation of commercial vehicles, the fluctuation of load has a direct impact on the power demand of the vehicle. Accurate load estimation can determine the specific power strategy for commercial vehicles, thereby affecting the endurance of commercial vehicles. However, commercial vehicle load estimation faces challenges such as diverse vehicle types, a wide variety of cargo, and complex loading conditions.

[0003] At present, there are no load sensors installed in commercial vehicles. The only way is to collect the physical response data of the vehicle under load in real time through the existing pressure, strain, acceleration and other sensors in the commercial vehicles, and then analyze the physical response data through signal processing and algorithm models to infer the real-time load of the commercial vehicle.

[0004] However, if the real-time load of commercial vehicles is determined only by existing sensors, the update frequency of each sensor on the commercial vehicle will be different, resulting in errors in sensor measurement results, large delays in power transmission, and other problems. The confidence of the vehicle mass estimation result is poor, and the estimation strategy is single and cannot adapt to the complex and changeable commercial vehicle operation scenarios. If a load sensor is set separately, the manufacturing cost of the vehicle will increase. Summary of the invention

[0005] The purpose of the present application is to provide a vehicle load estimation method and a vehicle control device to address the deficiencies in the above-mentioned prior art, which can improve the accuracy of target vehicle load estimation, make the estimation result of the target vehicle load more accurate, and reduce the manufacturing cost of the vehicle.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows: In a first aspect, an embodiment of the present application provides a vehicle load estimation method, the method comprising: Acquire a target environment image at the current moment collected by an image sensor on the target vehicle and a target power signal at the current moment collected by a chassis domain signal sensor on the target vehicle; According to the target environment image and the target power signal, a pre-trained multimodal load estimation network is used to perform load prediction to obtain a first estimated load value of the target vehicle at the current moment; According to the first estimated load value, a target load state of the target vehicle at the current moment is determined, and the target load state is used to adjust a driving control strategy of the target vehicle and estimate the cruising range of the target vehicle.

[0007] Optionally, the multimodal load estimation network includes: an image state module, a signal state module, a connection module and a mapping output module, wherein the image state module includes: a first feature extraction network and a first deep learning model, and the signal state module includes: a second feature extraction network and a second deep learning model; The method of performing load prediction using a pre-trained multimodal load estimation network according to the target environment image and the target power signal to obtain a first estimated load value of the target vehicle at the current moment includes: Using the first feature extraction network and the second feature extraction network, respectively extracting features from the target environment image and the target power signal to obtain target environment features and target power features; Using the first deep learning model and the second deep learning model, respectively, deeply fuse the target environment feature and the target power feature to obtain a first fused vehicle feature and a second fused vehicle feature; Using the connection module, the first fused vehicle feature and the second fused vehicle feature are spliced ​​to obtain a target vehicle feature; The mapping output module is used to map and output the target vehicle characteristics to obtain the first estimated load value.

[0008] Optionally, the target environment image is: an image sequence of at least two environment images, the at least two environment images include: the environment image at the current moment and the environment image at least one moment before the current moment; the target power signal is: a signal sequence of at least two power signals, the at least two power signals include: the power signal at the current moment and the power signal at at least one moment; The first feature extraction network and the second feature extraction network are used to extract features from the target environment image and the target power signal respectively to obtain target environment features and target power features, including: Using the first feature extraction network, extracting features from the at least two environment images to obtain at least two environment features as the target environment features; The second feature extraction network is used to extract features from the at least two power signals to obtain at least two power features as the target power features.

[0009] Optionally, the first deep learning model includes: a first long short-term memory network and a first cross attention network, and the second deep learning model includes: a second long short-term memory network and a second cross attention network; The method of performing load prediction using a pre-trained multimodal load estimation network according to the target environment image and the target power signal to obtain a first estimated load value of the target vehicle at the current moment includes: Using the first long short-term memory network and the second long short-term memory network, time series learning is performed on the at least two environmental image features and the at least two power signal features to obtain an environmental feature sequence and a power feature sequence; Using the first cross-attention network, cross-attention fusion is performed on the environmental feature sequence and the power feature sequence to obtain the first fused vehicle feature; The second cross-attention network is used to perform cross-attention fusion on the power feature sequence and the environmental feature sequence to obtain the second fused vehicle feature.

[0010] Optionally, before the load prediction is performed using a pre-trained multimodal load estimation network according to the target environment image and the target power signal to obtain a first estimated load value of the target vehicle at the current moment, the method further includes: Acquire sample data, the sample data including: a sample environment image, a sample power signal corresponding to the sample environment image, and a corresponding annotated load value; A preset initial multimodal load estimation network is trained according to the sample data to obtain the multimodal load estimation network.

[0011] Optionally, before determining the target load state of the target vehicle at the current moment according to the first estimated load value, the method further includes: Obtaining the real-time longitudinal acceleration of the target vehicle at the current moment; Determining, based on the real-time longitudinal acceleration, whether the target vehicle triggers an acceleration event at the current moment; If the acceleration event is triggered, obtaining peak opening data of the accelerator pedal on the target vehicle within a preset time window after the acceleration event is triggered; Determining a second estimated load value of the target vehicle at the current moment according to the real-time longitudinal acceleration and the peak opening data; Determining the target load state of the target vehicle at the current moment according to the first estimated load value includes: The target load state is determined according to the first estimated load value and the second estimated load value.

[0012] Optionally, determining a second estimated load value of the target vehicle at the current moment according to the real-time longitudinal acceleration and the peak opening data includes: Performing a division operation on the real-time longitudinal acceleration and the peak opening data to obtain the load coefficient at the current moment; According to the load coefficient and a pre-established mapping relationship table between the load coefficient and the load value, the load value corresponding to the load coefficient is determined as the second estimated load value.

[0013] Optionally, before determining the target load state of the target vehicle at the current moment according to the first estimated load value, the method further includes: Calculating the longitudinal driving force of the target vehicle at the current moment according to the motor torque value of the target vehicle at the current moment; Calculate the longitudinal resultant external force of the target vehicle at the current moment according to the real-time longitudinal speed of the target vehicle at the current moment, the preset air density, the preset drag coefficient, and the preset road slope; Calculating a third estimated load value of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force; Determining the target load state of the target vehicle at the current moment according to the first estimated load value includes: The target load state is determined according to the first estimated load value and the third estimated load value.

[0014] Optionally, calculating a third estimated load value of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force includes: Calculating a target longitudinal force of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force; The third estimated load value is calculated according to the target longitudinal force, the longitudinal speed at the current moment and the longitudinal speed at the previous moment by using a preset kinetic energy theorem equation.

[0015] Optionally, the method further comprises: According to multiple third estimated load values ​​of at least one time step before the current moment, a moving average algorithm is used to perform weighted averaging on the third estimated load value at the current moment to obtain a filtered estimated load value at the current moment.

[0016] In a second aspect, another embodiment of the present application provides a vehicle load estimation device, the device comprising: An acquisition module, used to acquire a target environment image at a current moment acquired by an image sensor on a target vehicle and a target power signal at the current moment acquired by a chassis domain signal sensor on the target vehicle; A prediction module, configured to perform load prediction using a pre-trained multimodal load estimation network according to the target environment image and the target power signal, to obtain a first estimated load value of the target vehicle at the current moment; A determination module is used to determine a target load state of the target vehicle at the current moment according to the first estimated load value, wherein the target load state is used to adjust a driving control strategy of the target vehicle and estimate the endurance of the target vehicle.

[0017] In the third aspect, another embodiment of the present application provides a vehicle control device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, when the vehicle control device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of the vehicle load estimation method as described in any one of the first aspects above.

[0018] In a fourth aspect, another embodiment of the present application provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the vehicle load estimation method as described in any one of the above-mentioned first aspects are executed.

[0019] In a fifth aspect, another embodiment of the present application provides a vehicle, which comprises at least: the vehicle control device described in the third aspect above.

[0020] The beneficial effects of this application are: In an embodiment of the present application, the target environment image at the current moment collected by the image sensor on the target vehicle and the target power signal at the current moment collected by the chassis domain signal sensor on the target vehicle are obtained. Based on the target environment image and the target power signal, a pre-trained multimodal load estimation network is used to perform load prediction to obtain a first estimated load value of the target vehicle at the current moment. Based on the first estimated load value, the target load state of the target vehicle at the current moment is determined, and the target load state is used to adjust the drive control strategy of the target vehicle and to estimate the endurance of the target vehicle. The present application ensures the richness of the target vehicle data acquisition through the target environment image and the target power signal, comprehensively captures more comprehensive vehicle status information, thereby improving the accuracy of load estimation, and the present application can extract the key features of the target vehicle from the multimodal data, so that the load estimation result is more accurate and reduces the manufacturing cost of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A schematic diagram of a vehicle load estimation method provided in an embodiment of the present application; Figure 2 A schematic diagram of a process for determining a first estimated load value in a vehicle load estimation method provided in an embodiment of the present application; Figure 3 A schematic diagram of a process for determining a first estimated load value in another vehicle load estimation method provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of a pre-trained multimodal load estimation network provided in an embodiment of the present application; Figure 5 A schematic diagram of a flow chart of determining a multi-modal load estimation network in a vehicle load estimation method provided in an embodiment of the present application; Figure 6 A schematic diagram of a flow chart of determining a target load state in a vehicle load estimation method provided in an embodiment of the present application; Figure 7 A schematic diagram of a process for determining a second estimated load value in a vehicle load estimation method provided in an embodiment of the present application; Figure 8 A schematic diagram of a flow chart of determining a target load state in another vehicle load estimation method provided in an embodiment of the present application; Fig. 9 A longitudinal force analysis diagram of a target vehicle provided in an embodiment of the present application; Fig.10 A schematic diagram of a process for determining a third estimated load in a vehicle load estimation method provided in an embodiment of the present application; Fig.11 A schematic diagram of the structure of a vehicle load estimation device provided in an embodiment of the present application; Fig.12 A schematic diagram of the structure of a vehicle control device provided in an embodiment of the present application; Fig.13 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0023] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0024] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0025] To clearly describe a vehicle load estimation method provided in an embodiment of the present application, the method provided in an embodiment of the present application is described below in conjunction with multiple drawings. The method provided in this embodiment can be executed by a vehicle control device integrated on a target vehicle. Figure 1 A flow chart of a vehicle load estimation method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method includes: Step 101: Acquire a target environment image at a current moment collected by an image sensor on a target vehicle and a target power signal at a current moment collected by a chassis domain signal sensor on the target vehicle.

[0026] Among them, the target vehicle can be a pure electric vehicle, a hybrid vehicle, a fuel cell vehicle, etc., and the embodiment of the present application does not limit this. The target environment image of the target vehicle at the current moment is collected by the image sensor carried by the vehicle-mounted camera, and the target environment image is used to indicate the road surface information during the driving process of the target vehicle. The chassis domain signal sensor may include one or more of a temperature sensor, a pressure sensor, a speed sensor, a position sensor or a flow sensor. The embodiment of the present application does not limit this. The sensor signal collected by the chassis domain signal sensor determines the target power signal of the target vehicle at the current moment. The target power signal can be at least one vehicle power system signal such as a motor speed signal, a motor torque signal, an accelerator pedal position signal, and a brake pedal position signal.

[0027] Optionally, the target environment image of the target vehicle at the current moment is collected by an image sensor on the target vehicle to determine the road surface information of the target vehicle at the current moment. The target power signal of the target vehicle at the current moment is determined by multiple sensor signals collected by a chassis domain signal sensor of the target vehicle.

[0028] Step 102: According to the target environment image and the target power signal, a pre-trained multimodal load estimation network is used to perform load prediction to obtain a first estimated load value of the target vehicle at the current moment.

[0029] The pre-trained multimodal load estimation network is a model that can process multimodal data. The vehicle load refers to the weight value that the vehicle can bear under normal driving conditions. The first estimated load value is the vehicle estimated load value obtained by load prediction through the pre-trained multimodal load estimation network. The first estimated load value includes: the empty vehicle weight value of the target vehicle and the additional weight value of the load.

[0030] Optionally, before inputting the target environment image and the target power signal into the pre-trained multimodal load estimation network, the target environment image and the target power signal need to be cleaned, denoised and normalized to ensure the data quality of the target environment image and the target power signal.

[0031] Optionally, the target environment image and the target power signal are used as inputs of a pre-trained multimodal load estimation network, and the pre-trained multimodal load estimation network is used to perform load prediction to obtain a first estimated load value of the target vehicle at the current moment, thereby achieving end-to-end load prediction from input to output.

[0032] Step 103: Determine a target load state of the target vehicle at the current moment according to the first estimated load value. The target load state is used to adjust the driving control strategy of the target vehicle and estimate the endurance of the target vehicle.

[0033] Among them, according to the target load value of the target vehicle, the preset load value and the load state table are mapped to obtain the target load state corresponding to the first estimated load value. Among them, the load state may include: no load, half load and full load. Different load states correspond to different drive control strategies of the target vehicle, and different load states correspond to different cruising ranges. When the vehicle is in an unloaded state, the cruising range is the basic cruising range, when the vehicle is in a half-loaded state, the cruising range is less than the basic cruising range, and when the vehicle is in a fully loaded state, the cruising range is the smallest.

[0034] For example, when the target load state of the target vehicle is an unloaded state, the vehicle is lighter, the vehicle motor is adjusted to respond faster, the suspension is stiffer, the acceleration is faster, the vehicle control and handling are more flexible, and the braking is relatively small. When the target load state of the target vehicle is a half-loaded state, the vehicle motor response is slightly delayed to avoid excessive acceleration due to increased load, the suspension is adjusted to medium hardness, and the response of the braking system is slightly adjusted. When the target load state of the target vehicle is a fully loaded state, the motor response is further delayed, the torque output of the motor is adjusted, the suspension is adjusted to be softer, the suspension system requires a larger automatic clutch, the vehicle stability control system is more sensitive, and the automatic transmission may shift up earlier to reduce the motor load.

[0035] In an embodiment of the present application, the target environment image at the current moment collected by the image sensor on the target vehicle and the target power signal at the current moment collected by the chassis domain signal sensor on the target vehicle are obtained. According to the target environment image and the target power signal, a pre-trained multimodal load estimation network is used to predict the load, and the first estimated load value of the target vehicle at the current moment is obtained. According to the first estimated load value, the target load state of the target vehicle at the current moment is determined, and the target load state is used to adjust the drive control strategy of the target vehicle and to estimate the endurance of the target vehicle. This application uses a multimodal load estimation network through the target environment image and the target power signal to fully utilize the complementarity between multimodal information, realize the prediction of the load value, ensure the richness of the target vehicle data acquisition, and fully capture more comprehensive vehicle status information, thereby improving the accuracy of the load estimation, so that the load estimation result is more accurate. At the same time, the scheme of the present application can be based on existing sensors such as image sensors and chassis domain signal sensors without increasing the number of sensors, and can make real-time and accurate estimates of the vehicle mass.

[0036] Based on the above embodiment, the multimodal load estimation network includes: an image state module, a signal state module, a connection module and a mapping output module; wherein the image state module includes: a first feature extraction network and a first deep learning model, and the signal state module includes: a second feature extraction network and a second deep learning model. The present application also provides a process for determining a first estimated load value in a vehicle load estimation method, Figure 2 A schematic diagram of a process for determining a first estimated load value in a vehicle load estimation method provided in an embodiment of the present application, such as Figure 2 As shown, in the above step 102, according to the target environment image and the target power signal, a pre-trained multimodal load estimation network is used to perform load prediction to obtain a first estimated load value of the target vehicle at the current moment, including: Step 201: Use a first feature extraction network and a second feature extraction network to extract features from a target environment image and a target power signal, respectively, to obtain target environment features and target power features.

[0037] Among them, the first feature extraction network and the second feature extraction network are convolutional neural networks. The convolutional neural network can be a ResNet architecture network. The target environment characteristics describe the color distribution characteristics, texture characteristics, shape characteristics, etc. of different regions in the image, and the embodiments of the present application are not limited to this. The target dynamic characteristics may include: speed characteristics, acceleration characteristics, force characteristics and other information of the target vehicle, and the embodiments of the present application are not limited to this.

[0038] Step 202: Use the first deep learning model and the second deep learning model to perform deep fusion on the target environment characteristics and the target power characteristics respectively to obtain the first fused vehicle characteristics and the second fused vehicle characteristics.

[0039] Optionally, the target environment characteristics are processed using a first deep learning model, and the target power characteristics are fused into the target environment characteristics to obtain a first fused vehicle characteristic; after the target power characteristics are processed using a second deep learning model, the target environment characteristics are fused into the target power characteristics to obtain a second fused vehicle characteristic.

[0040] Step 203: Use a connection module to splice the first fused vehicle feature and the second fused vehicle feature to obtain a target vehicle feature.

[0041] Optionally, a connection module is used to superimpose or multiply the first fused vehicle feature and the second fused vehicle feature to obtain the target vehicle feature. The embodiment of the present application does not limit the specific splicing method.

[0042] Step 204: Use a mapping output module to map and output the target vehicle characteristics to obtain a first estimated load value.

[0043] The mapping output module is a mapping output module based on an activation function, and specifically, can be a mapping output module based on a Sigmoid activation function.

[0044] Optionally, a mapping output module is used to map the target vehicle characteristics based on a Sigmoid activation function to obtain a first estimated load value.

[0045] In an embodiment of the present application, feature extraction is performed on the target environment image and the target power signal, respectively, to obtain target environment features and target power features. The target environment features and target power features are deeply fused, respectively, to obtain a first fused vehicle feature and a second fused vehicle feature, the first fused vehicle feature and the second fused vehicle feature are spliced ​​to obtain a target vehicle feature, the target vehicle feature is mapped and output, and a first estimated load value is obtained. By processing image and signal data simultaneously, the present application enables the model to learn the intrinsic connection between different modal data, fully utilize the complementarity between multimodal information, effectively fuse features from different sensors, enhance the generalization ability and robustness of the model, and improve the accuracy of model prediction.

[0046] On the basis of the above embodiment, the target environment image is: an image sequence of at least two environment images, the at least two environment images include: an environment image at the current moment and an environment image at least one moment before the current moment, and the target power signal is: a signal sequence of at least two power signals, the at least two power signals include: a power signal at the current moment and a power signal at at least one moment. To this end, the present application also provides a process for determining target environment characteristics and target power characteristics in a vehicle load estimation method. In the above step 201, a first feature extraction network and a second feature extraction network are used to extract features from the target environment image and the target power signal, respectively, to obtain target environment characteristics and target power characteristics, including: A first feature extraction network is used to extract features from at least two environment images to obtain at least two environment features as target environment features.

[0047] Optionally, by using a first feature extraction network to perform feature extraction on at least two environmental images, environmental information of the target vehicle at multiple times can be obtained, thereby obtaining target environmental features of the target vehicle at different times.

[0048] A second feature extraction network is used to extract features from at least two power signals to obtain at least two power features as target power features.

[0049] Optionally, a second feature extraction network is used to perform feature extraction on at least two power signals, so as to obtain power information of the target vehicle at multiple times and obtain target power characteristics of the target vehicle at different times.

[0050] In an embodiment of the present application, a first feature extraction network is used to perform feature extraction on at least two environmental images to obtain at least two environmental features as target environmental features, and a second feature extraction network is used to perform feature extraction on at least two power signals to obtain at least two power features as target power features, thereby increasing the dimensions of environmental features and power features and enhancing the robustness of the model.

[0051] Based on the above embodiment, the first deep learning model includes: a first long short-term memory (LSTM) network and a first cross attention network, and the second deep learning model includes: a second long short-term memory network and a second cross attention network. The present application also provides another process for determining a first estimated load value in a vehicle load estimation method. Figure 3 A schematic diagram of a process for determining a first estimated load value in another vehicle load estimation method provided in an embodiment of the present application, such as Figure 3 As shown, in the above step 102, according to the target environment image and the target power signal, a pre-trained multimodal load estimation network is used to perform load prediction to obtain a first estimated load value of the target vehicle at the current moment, including: Step 301: Use a first long short-term memory network and a second long short-term memory network to perform time series learning on at least two environmental image features and at least two dynamic signal features to obtain an environmental feature sequence and a dynamic feature sequence.

[0052] Among them, the first long short-term memory network is used to perform time series learning on at least two environmental image features to obtain an environmental feature sequence. The environmental feature sequence refers to a series of data arranged in a certain order to describe environmental related features. The environmental feature sequence reflects the state changes of the environment at different time points or different spatial positions. The second long short-term memory network is used to perform time series learning on at least two power signal features to obtain a power feature sequence. The power feature sequence refers to a data series arranged in a specific order that describes the relevant features of the power system. The power feature sequence reflects the dynamic changes of these systems during operation.

[0053] Optionally, at least two environmental image features are used as inputs of a first long-term and short-term network, and the first long-term and short-term network is used to perform time series learning on at least two environmental image features to obtain an environmental feature sequence. At least two power signal features are used as inputs of a second long-term and short-term network, and the second long-term and short-term network is used to perform time series learning on at least two power signal features to obtain a power feature sequence.

[0054] Step 302: Use a first cross-attention network to perform cross-attention fusion on the environmental feature sequence and the dynamic feature sequence to obtain a first fused vehicle feature.

[0055] Optionally, a first cross-attention network is used to integrate the influence of the power feature sequence on the environment feature sequence into the environment feature sequence to obtain a first fused vehicle feature.

[0056] Step 303: Use a second cross-attention network to perform cross-attention fusion on the power feature sequence and the environmental feature sequence to obtain a second fused vehicle feature.

[0057] Optionally, a second cross-attention network is used to integrate the influence of the environmental feature sequence on the power feature sequence into the power feature sequence to obtain a second fused vehicle feature.

[0058] For example, Figure 4 A schematic diagram of the structure of a pre-trained multi-modal load estimation network provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the multimodal load estimation network includes: an image state module, a signal state module, a connection module and a mapping output module. The image state module includes: a first feature extraction network and a first deep learning model, and the signal state module includes: a second feature extraction network and a second deep learning model. The first deep learning model includes: a first long short-term memory network and a first cross attention network, and the second deep learning model includes: a second long short-term memory network and a second cross attention network. Among them, the first feature extraction network and the second feature extraction network are used to extract features of the target environment image and the target power signal respectively to obtain the target environment feature and the target power feature. The first long short-term memory network and the second long short-term memory network are used to perform time series learning on at least two environment image features and at least two power signal features to obtain an environment feature sequence and a power feature sequence. The first cross attention network is used to perform cross attention fusion on the environment feature sequence and the power feature sequence to obtain a first fused vehicle feature; the second cross attention network is used to perform cross attention fusion on the power feature sequence and the environment feature sequence to obtain a second fused vehicle feature. The connection module is used to splice the first fused vehicle feature and the second fused vehicle feature to obtain the target vehicle feature; the mapping output module is used to map and output the target vehicle feature to obtain the first estimated load value.

[0059] In the embodiment of the present application, the environmental feature sequence and the power feature sequence are obtained through the long short-term memory network, and the first fused vehicle feature and the second fused vehicle feature are obtained through the cross attention network. The first fused vehicle feature and the second fused vehicle feature obtained in the present application can make full use of the complementary information of the environmental feature sequence and the power feature sequence, thereby enhancing the description of the vehicle features, and the obtained fused features are more accurate.

[0060] Based on the above embodiments, the present application also provides a process for determining a multi-modal load estimation network in a vehicle load estimation method. Figure 5 A schematic diagram of a flow chart of determining a multi-modal load estimation network in a vehicle load estimation method provided in an embodiment of the present application, such as Figure 5 As shown, before the above step 102 uses a pre-trained multimodal load estimation network to perform load prediction according to the target environment image and the target power signal to obtain a first estimated load value of the target vehicle at the current moment, the method further includes: Step 501: Obtain sample data.

[0061] The sample data includes: a sample environment image, a sample power signal corresponding to the sample environment image, and a corresponding annotated load value.

[0062] Optionally, the target environment image is collected by an image sensor and the target power signal is collected by a chassis domain signal sensor on the target vehicle, and the load value of the target vehicle is determined and marked.

[0063] Step 502: Train a preset initial multimodal load estimation network according to the sample data to obtain a multimodal load estimation network.

[0064] Optionally, a preset initial multimodal load estimation network is trained according to sample data to obtain predicted load values, and parameters of the preset initial multimodal load estimation network are adjusted according to the predicted load values ​​and the labeled load values ​​to obtain a multimodal load estimation network. Specifically, during the training of the preset initial multimodal load estimation network, data enhancement, regularization and other model enhancement techniques are used to prevent model overfitting, and the model training process is optimized by dynamically adjusting the learning rate and applying an early stopping strategy.

[0065] In an embodiment of the present application, a preset initial multimodal load estimation network is trained according to sample data to obtain a multimodal load estimation network. The present application uses a sample data set containing exact labeled load values ​​to train the model, which can improve the prediction accuracy and generalization ability of the model. In addition, the preset initial multimodal load estimation network can make full use of the complementarity between these different modal information during the training process. This allows for a more comprehensive understanding of the relationship between the vehicle operating state and the load, thereby improving the accuracy of load estimation.

[0066] Based on the above embodiments, the present application also provides a process for determining a target load state in a vehicle load estimation method. Figure 6 A schematic diagram of a process for determining a target load state in a vehicle load estimation method provided in an embodiment of the present application, such as Figure 6As shown, before the above step 103 determines the target load state of the target vehicle at the current moment according to the first estimated load value, the method further includes: Step 601: Obtain the real-time longitudinal acceleration of the target vehicle at the current moment.

[0067] Optionally, a wheel speed sensor is provided on the target vehicle, and the speed of the target vehicle at the current moment and the previous moment is determined according to the wheel speed sensor, so as to obtain the real-time longitudinal acceleration of the target vehicle at the current moment. The real-time longitudinal acceleration refers to the change in speed of the target vehicle in the driving direction per unit time.

[0068] Step 602: Determine whether the target vehicle triggers an acceleration event at the current moment according to the real-time longitudinal acceleration.

[0069] Optionally, when the real-time longitudinal acceleration is greater than a preset longitudinal acceleration threshold, it is determined that the target vehicle triggers an acceleration event at the current moment. When the real-time longitudinal acceleration is less than or equal to the preset longitudinal acceleration threshold, it is determined that the target vehicle does not trigger an acceleration event at the current moment. The preset longitudinal acceleration threshold is set according to the vehicle type and performance of the target vehicle to ensure that the acceleration event can be accurately captured in normal driving operations.

[0070] Step 603: If an acceleration event is triggered, obtain peak opening data of the accelerator pedal on the target vehicle within a preset time window after the acceleration event is triggered.

[0071] The preset time window can be set to 5 seconds, and the peak opening data of the accelerator pedal is used to indicate the data record of the maximum degree to which the accelerator pedal is depressed. The accelerator pedal opening represents the intensity of the driver's request for vehicle power. The larger the opening, the greater the power the driver expects the vehicle to output.

[0072] Optionally, when the real-time longitudinal acceleration is greater than a preset longitudinal acceleration threshold, it is determined that the target vehicle triggers an acceleration event at the current moment, and the accelerator pedal opening data of the accelerator pedal on the target vehicle within a preset time window after the acceleration event is triggered is obtained, and the maximum value of the accelerator pedal opening data within the preset time window is recorded as the peak opening data.

[0073] Step 604: Determine a second estimated load value of the target vehicle at the current moment according to the real-time longitudinal acceleration and the peak opening data.

[0074] As in step 103 above, determining the target load state of the target vehicle at the current moment according to the first estimated load value includes: Step 605: Determine a target load state according to the first estimated load value and the second estimated load value.

[0075] Optionally, the target load value is determined based on the average of the first estimated load value and the second estimated load value, and then the target load state is determined based on the target load value; or a weighted sum operation is performed on the first estimated load value and the second estimated load value to obtain the target load value, and then the corresponding target load state is determined based on the target load value.

[0076] In the embodiment of the present application, the second estimated load value is determined by obtaining the real-time longitudinal acceleration of the target vehicle at the current moment, and the target load state is determined according to the first estimated load value and the second estimated load value. The present application can more comprehensively and accurately reflect the actual load state of the vehicle, avoid large errors in a single estimation method under special working conditions, and can complement and verify each other to reduce the errors and uncertainties that may exist in a single strategy.

[0077] Based on the above embodiments, the present application further provides a process for determining a second estimated load value in a vehicle load estimation method. Figure 7 A schematic diagram of a process for determining a second estimated load value in a vehicle load estimation method provided in an embodiment of the present application, such as Figure 7 As shown, in the above step 604, the second estimated load value of the target vehicle at the current moment is determined according to the real-time longitudinal acceleration and the peak opening data, including: Step 701: Perform division operation on the real-time longitudinal acceleration and the peak opening data to obtain the load factor at the current moment.

[0078] Optionally, the real-time longitudinal acceleration is used as the numerator and the peak opening data is used as the denominator for division to obtain the load factor at the current moment. The load factor reflects the actual acceleration capability of the vehicle under the given accelerator pedal opening data, and is proportional to the vehicle load state.

[0079] Step 702: According to the load coefficient and a pre-established mapping relationship table between the load coefficient and the load value, determine that the load value corresponding to the load coefficient is a second estimated load value.

[0080] Among them, the mapping relationship table between load coefficient and load value is based on a large amount of experimental data, and records the corresponding relationship between the real-time longitudinal acceleration and the peak opening data ratio under different load conditions.

[0081] In the embodiment of the present application, the real-time longitudinal acceleration and the peak opening data are divided to obtain the load coefficient at the current moment, and the load value corresponding to the load coefficient is determined as the second estimated load value. The second estimated load value can be quickly obtained based on the real-time longitudinal acceleration and the peak opening data. This strategy is suitable for the working condition where the vehicle starts from a stationary state and generates significant acceleration, ensuring the real-time and accuracy requirements of load identification.

[0082] Based on the above embodiment, the present application also provides another process for determining a target load state in a vehicle load estimation method. Figure 8 A schematic diagram of a flow chart of determining a target load state in another vehicle load estimation method provided in an embodiment of the present application, such as Figure 8 As shown, before the above step 103 determines the target load state of the target vehicle at the current moment according to the first estimated load value, the method further includes: Step 801: Calculate the longitudinal driving force of the target vehicle at the current moment according to the motor torque value of the target vehicle at the current moment.

[0083] Optionally, the longitudinal driving force of the target vehicle at the current moment is calculated by formula (1) according to the motor output torque value of the target vehicle at the current moment, the drive axle wheel speed ratio, the current gearbox speed ratio, the wheel rolling radius, the wheel end driving force of the target vehicle and the torque transmission efficiency.

[0084] (1) in, is the longitudinal driving force of the target vehicle at the current moment, is the motor output torque value of the target vehicle at the current moment, is the drive axle speed ratio, is the current gearbox ratio, is the wheel rolling radius, The torque transmission efficiency.

[0085] Step 802: Calculate the longitudinal resultant external force of the target vehicle at the current moment according to the real-time longitudinal speed of the target vehicle at the current moment, the preset air density, the preset drag coefficient, and the preset road slope.

[0086] Optionally, in addition to the driving force, the target vehicle is also subjected to a variety of external forces during its travel, such as air resistance, rolling resistance, and ramp resistance. External forces such as air resistance, rolling resistance, and ramp resistance constitute the longitudinal resultant external force of the target vehicle at the current moment, and the longitudinal resultant external force at the current moment is related to the real-time longitudinal speed of the target vehicle at the current moment, the preset road surface slope, and the vehicle characteristics. The real-time longitudinal speed, real-time longitudinal acceleration, preset road surface slope, and vehicle characteristics information of the target vehicle at the current moment are determined by an external sensor on the target vehicle. Thus, according to the real-time longitudinal speed of the target vehicle at the current moment, the preset air density, the preset wind resistance coefficient, and the preset road surface slope, the external forces such as air resistance, rolling resistance, and ramp resistance of the target vehicle are determined. And the longitudinal resultant external force of the target vehicle at the current moment is determined according to the external forces such as air resistance, rolling resistance, and ramp resistance. Among them, the air resistance is determined by the real-time longitudinal speed, the preset air density, the preset wind resistance coefficient, and the windward area of ​​the target vehicle. The rolling resistance is determined by the mass of the target vehicle, the acceleration of gravity, the preset road surface slope, and the road surface rolling coefficient at the current moment. The hill resistance is determined by the mass of the target vehicle, the acceleration due to gravity, and the preset road slope.

[0087] For example, the longitudinal resultant external force in this application is explained by taking air resistance, rolling resistance and ramp resistance as examples to illustrate the longitudinal resultant external force of the target vehicle. Fig. 9 A longitudinal force analysis diagram of a target vehicle provided in an embodiment of the present application. Fig. 9 As shown, the air resistance is , the rolling resistance is , the ramp resistance is , since the directions of air resistance, rolling resistance and ramp resistance are the same, the longitudinal force of the target vehicle is determined to be .in, is the mass of the vehicle, is the air density, is the drag coefficient, is the windward area, is the longitudinal velocity of the vehicle, is the acceleration due to gravity, is the road slope, is the road rolling resistance coefficient.

[0088] Step 803: Calculate a third estimated load value of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force.

[0089] As in step 103 above, determining the target load state of the target vehicle at the current moment according to the first estimated load value includes: Step 804: Determine a target load state according to the first estimated load value and the third estimated load value.

[0090] Optionally, the target load state is determined based on an average value of the first estimated load value and the third estimated load value; or the first estimated load value and the third estimated load value are weighted respectively, and the target load state is determined based on the weighted sum.

[0091] Optionally, the application determines the target load state based on the average of the first estimated load value, the second estimated load and the third estimated load value; or weights the first estimated load value, the second estimated load and the third estimated load value respectively, and determines the target load state based on the weighted sum.

[0092] In the embodiment of the present application, the third estimated load value is determined by obtaining the real-time longitudinal acceleration of the target vehicle at the current moment, and the target load state is determined according to the first estimated load value and the third estimated load value. The present application does not need to increase the number of sensors, and the target load state can be determined according to the existing number of sensors, and can more comprehensively and accurately reflect the actual load state of the vehicle, avoiding large errors in a single estimation method under special working conditions. The first estimated load and the third estimated load can complement and verify each other, reducing the errors and uncertainties that may exist in a single strategy.

[0093] Based on the above embodiments, the present application further provides a process for determining a third estimated load in a vehicle load estimation method. Fig.10 A schematic diagram of a process for determining a third estimated load in a vehicle load estimation method provided in an embodiment of the present application, such as Fig.10 As shown, in the above step 803, the third estimated load value of the target vehicle at the current moment is calculated according to the longitudinal driving force and the longitudinal resultant external force, including: Step 1001: Calculate the target longitudinal force of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force.

[0094] Optionally, since the longitudinal driving force and the longitudinal resultant external force are in opposite directions, the longitudinal driving force and the longitudinal resultant external force are subtracted to calculate the target longitudinal force of the target vehicle at the current moment. The target longitudinal force F of the target vehicle at the current moment is calculated by formula (2).

[0095] (2) in, is the target longitudinal force of the target vehicle at the current moment, is the longitudinal driving force, is the mass of the vehicle, is the air density, is the drag coefficient, is the windward area, is the longitudinal velocity of the vehicle, is the acceleration due to gravity, is the road slope, is the road rolling resistance coefficient.

[0096] Step 1002: Calculate a third estimated load value according to the target longitudinal force, the longitudinal velocity at the current moment, and the longitudinal velocity at the previous moment using a preset kinetic energy theorem equation.

[0097] Optionally, the kinetic energy theorem equation can be expressed as formula (3), for example, and the target longitudinal force, the longitudinal velocity at the current moment, and the longitudinal velocity at the previous moment are substituted into the kinetic energy theorem equation to obtain formula (4), thereby obtaining the third estimated load value. Specifically, according to the kinetic energy theorem equation, the load of the target vehicle is expressed as the ratio of the kinetic energy change to the square difference of the velocity. Within the rated time, the displacement of the target longitudinal force is integrated to obtain the kinetic energy change, and then divided by the square difference of the velocity to obtain the third estimated load value, thereby estimating the load of the target vehicle during the vehicle driving process, with high accuracy and practicality.

[0098] (3) in, is the target longitudinal force of the target vehicle at the current moment, For the previous moment Time to current time The work done by the target longitudinal force acting on the target vehicle during time, Current time Time and the previous time The longitudinal velocity of the vehicle at the time instant corresponds to the displacement, is the speed of the target vehicle at the previous moment, is the speed of the target vehicle at the current moment, is the acceleration of the target vehicle.

[0099] (4) The third estimated load value for:

[0100] In the embodiment of the present application, due to the different update frequencies of the vehicle sensors, there are errors in the sensor measurement results, and there are large delays in power transmission, resulting in a low confidence level in the load estimation of the target vehicle. The third estimated load value is calculated based on the integral form through the kinetic energy theorem equation, which greatly reduces the impact of the instantaneous signal error and the delayed third estimated load value result. And by solving the third estimated load value in the form of continuous integration, the impact of the instantaneous signal error and delay on the estimation result of the third estimated load value is greatly reduced, and the load of the target vehicle can be accurately estimated, thereby more accurately judging the dynamic response of the vehicle, making more reasonable control decisions, and improving the power and economic performance of the vehicle.

[0101] Based on the above embodiment, the present application also provides another process for determining a third estimated load in a vehicle load estimation method. Based on the above steps 1001 and 1002, the method further includes: According to a plurality of third estimated load values ​​of at least one time step before the current moment, a moving average algorithm is used to perform weighted averaging on the third estimated load value at the current moment to obtain a filtered estimated load value at the current moment.

[0102] Optionally, based on multiple third estimated load values ​​of at least one time step before the current moment, a moving average algorithm is used to perform weighted averaging on the third estimated load value at the current moment through formula (5) to obtain the filtered estimated load value at the current moment: .

[0103] (5) in, is the estimated load value after filtering, is the third estimated load value at the current moment, is the third estimated load value at the previous moment, The filter coefficient is determined according to the target vehicle, and the embodiment of the present application does not limit this.

[0104] In the embodiment of the present application, a moving average algorithm is used to filter the real-time estimation results, and in each time step, the current estimation result is weighted averaged with the estimation results of several previous time steps to obtain a smoother and more stable vehicle mass estimation value. This can effectively reduce the error caused by noise and interference to the estimation results, and improve the reliability and stability of vehicle mass estimation.

[0105] Based on the same inventive concept, a vehicle load estimation device corresponding to the vehicle load estimation method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned vehicle load estimation method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0106] Fig.11 A schematic diagram of the structure of a vehicle load estimation device provided in an embodiment of the present application, the device includes: An acquisition module 1101 is used to acquire a target environment image at a current moment acquired by an image sensor on a target vehicle and a target power signal at a current moment acquired by a chassis domain signal sensor on a target vehicle; Prediction module 1102, used to perform load prediction using a pre-trained multi-modal load estimation network according to the target environment image and the target power signal, and obtain a first estimated load value of the target vehicle at the current moment; The determination module 1103 is used to determine the target load state of the target vehicle at the current moment according to the first estimated load value, and the target load state is used to adjust the driving control strategy of the target vehicle and estimate the endurance of the target vehicle.

[0107] Optionally, the multimodal load estimation network includes: an image state module, a signal state module, a connection module and a mapping output module, wherein the image state module includes: a first feature extraction network and a first deep learning model, and the signal state module includes: a second feature extraction network and a second deep learning model; the prediction module 1102 is specifically used to: use the first feature extraction network and the second feature extraction network to extract features of the target environment image and the target power signal respectively, so as to obtain target environment features and target power features; Using a first deep learning model and a second deep learning model, respectively, deeply fuse the target environment feature and the target power feature to obtain a first fused vehicle feature and a second fused vehicle feature; Using a connection module, the first fused vehicle feature and the second fused vehicle feature are spliced ​​to obtain a target vehicle feature; A mapping output module is used to map and output the target vehicle characteristics to obtain a first estimated load value.

[0108] Optionally, the target environmental image is: an image sequence of at least two environmental images, the at least two environmental images include: an environmental image at a current moment and an environmental image at at least one moment before the current moment; the target power signal is: a signal sequence of at least two power signals, the at least two power signals include: a power signal at a current moment and a power signal at at least one moment; the prediction module 1102 is specifically used to: use a first feature extraction network to extract features from at least two environmental images, and obtain at least two environmental features as target environmental features.

[0109] and a second feature extraction network, performing feature extraction on at least two power signals to obtain at least two power features as target power features.

[0110] Optionally, the first deep learning model includes: a first long short-term memory network and a first cross attention network, and the second deep learning model includes: a second long short-term memory network and a second cross attention network; the prediction module 1102 is specifically used to: use the first long short-term memory network and the second long short-term memory network to perform time series learning on at least two environmental image features and at least two dynamic signal features to obtain an environmental feature sequence and a dynamic feature sequence; The first cross-attention network is used to perform cross-attention fusion on the environmental feature sequence and the dynamic feature sequence to obtain the first fused vehicle feature; The second cross-attention network is used to perform cross-attention fusion on the power feature sequence and the environmental feature sequence to obtain the second fused vehicle feature.

[0111] Optionally, the prediction module 1102 is further used to: obtain sample data, the sample data including: a sample environment image, a sample power signal corresponding to the sample environment image, and a corresponding annotated load value; According to the sample data, a preset initial multimodal load estimation network is trained to obtain a multimodal load estimation network.

[0112] Optionally, the determination module 1103 is further used to: obtain the real-time longitudinal acceleration of the target vehicle at the current moment; According to the real-time longitudinal acceleration, determine whether the target vehicle triggers an acceleration event at the current moment; If an acceleration event is triggered, the peak opening data of the accelerator pedal on the target vehicle within a preset time window after the acceleration event is triggered is obtained; Determine a second estimated load value of the target vehicle at the current moment according to the real-time longitudinal acceleration and the peak opening data; Optionally, the determination module 1103 is specifically used to determine a target load state according to the first estimated load value and the second estimated load value.

[0113] Optionally, the determination module 1103 is specifically used to: perform a division operation on the real-time longitudinal acceleration and the peak opening data to obtain the load coefficient at the current moment; According to the load coefficient and a pre-established mapping relationship table between the load coefficient and the load value, the load value corresponding to the load coefficient is determined as the second estimated load value.

[0114] Optionally, the determination module 1103 is further used to: calculate the longitudinal driving force of the target vehicle at the current moment according to the motor torque value of the target vehicle at the current moment; Calculate the longitudinal resultant external force of the target vehicle at the current moment according to the real-time longitudinal speed of the target vehicle at the current moment, the preset air density, the preset drag coefficient, and the preset road slope; Calculating a third estimated load value of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force; Optionally, the determination module 1103 is specifically used to determine a target load state according to the first estimated load value and the third estimated load value.

[0115] Optionally, the determination module 1103 is specifically used to: calculate the target longitudinal force of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force; The third estimated load value is calculated according to the target longitudinal force, the longitudinal velocity at the current moment and the longitudinal velocity at the previous moment using a preset kinetic energy theorem equation.

[0116] Optionally, the determination module 1103 is further used to: perform weighted averaging of the third estimated load value at the current moment using a moving average algorithm based on multiple third estimated load values ​​of at least one time step before the current moment to obtain a filtered estimated load value at the current moment.

[0117] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0118] The embodiment of the present application also provides a vehicle control device, Fig.12 A schematic diagram of the structure of a vehicle control device provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the vehicle control device 1200 includes: a processor 1201, a memory 1202, and optionally, a bus 1203. The memory 1202 stores machine-readable instructions executable by the processor 1201. When the vehicle control device 1200 is running, the processor 1201 communicates with the memory 1202 via the bus 1203. When the machine-readable instructions are executed by the processor 1201, the steps of the above-mentioned vehicle load estimation method are performed.

[0119] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned vehicle load estimation method are executed.

[0120] The embodiment of the present application also provides a vehicle, Fig.13 A schematic diagram of a vehicle structure provided in an embodiment of the present application is shown in FIG. Fig.13 As shown, the vehicle at least includes: a vehicle control device 1200.

[0121] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0122] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the technical solution that contributes to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program code.

[0123] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A vehicle load estimation method, characterized in that: The method comprises: Acquire a target environment image at the current moment collected by an image sensor on the target vehicle and a target power signal at the current moment collected by a chassis domain signal sensor on the target vehicle; According to the target environment image and the target power signal, a pre-trained multimodal load estimation network is used to perform load prediction to obtain a first estimated load value of the target vehicle at the current moment; According to the first estimated load value, a target load state of the target vehicle at the current moment is determined, and the target load state is used to adjust a driving control strategy of the target vehicle and estimate the cruising range of the target vehicle.

2. The method according to claim 1, characterized in that The multimodal load estimation network includes: an image state module, a signal state module, a connection module and a mapping output module, wherein the image state module includes: a first feature extraction network and a first deep learning model, and the signal state module includes: a second feature extraction network and a second deep learning model; The method of performing load prediction using a pre-trained multimodal load estimation network according to the target environment image and the target power signal to obtain a first estimated load value of the target vehicle at the current moment includes: Using the first feature extraction network and the second feature extraction network, respectively extracting features from the target environment image and the target power signal to obtain target environment features and target power features; Using the first deep learning model and the second deep learning model, respectively, deeply fuse the target environment feature and the target power feature to obtain a first fused vehicle feature and a second fused vehicle feature; Using the connection module, the first fused vehicle feature and the second fused vehicle feature are spliced ​​to obtain a target vehicle feature; The mapping output module is used to map and output the target vehicle characteristics to obtain the first estimated load value.

3. The method according to claim 2, characterized in that The target environment image is: an image sequence of at least two environment images, wherein the at least two environment images include: the environment image at the current moment and the environment image at least one moment before the current moment; the target power signal is: a signal sequence of at least two power signals, wherein the at least two power signals include: the power signal at the current moment and the power signal at at least one moment; The first feature extraction network and the second feature extraction network are used to extract features from the target environment image and the target power signal respectively to obtain target environment features and target power features, including: Using the first feature extraction network, extracting features from the at least two environment images to obtain at least two environment features as the target environment features; The second feature extraction network is used to extract features from the at least two power signals to obtain at least two power features as the target power features.

4. The method according to claim 3, characterized in that: The first deep learning model includes: a first long short-term memory network and a first cross attention network, and the second deep learning model includes: a second long short-term memory network and a second cross attention network; The method of performing load prediction using a pre-trained multimodal load estimation network according to the target environment image and the target power signal to obtain a first estimated load value of the target vehicle at the current moment includes: Using the first long short-term memory network and the second long short-term memory network, time series learning is performed on the at least two environmental image features and the at least two power signal features to obtain an environmental feature sequence and a power feature sequence; Using the first cross-attention network, cross-attention fusion is performed on the environmental feature sequence and the power feature sequence to obtain the first fused vehicle feature; The second cross-attention network is used to perform cross-attention fusion on the power feature sequence and the environmental feature sequence to obtain the second fused vehicle feature.

5. The method according to claim 1, characterized in that Before the load prediction is performed using a pre-trained multimodal load estimation network according to the target environment image and the target power signal to obtain a first estimated load value of the target vehicle at the current moment, the method further includes: Acquire sample data, the sample data including: a sample environment image, a sample power signal corresponding to the sample environment image, and a corresponding annotated load value; A preset initial multimodal load estimation network is trained according to the sample data to obtain the multimodal load estimation network.

6. The method according to claim 1, characterized in that Before determining the target load state of the target vehicle at the current moment according to the first estimated load value, the method further includes: Obtaining the real-time longitudinal acceleration of the target vehicle at the current moment; Determining, based on the real-time longitudinal acceleration, whether the target vehicle triggers an acceleration event at the current moment; If the acceleration event is triggered, obtaining peak opening data of the accelerator pedal on the target vehicle within a preset time window after the acceleration event is triggered; Determining a second estimated load value of the target vehicle at the current moment according to the real-time longitudinal acceleration and the peak opening data; Determining the target load state of the target vehicle at the current moment according to the first estimated load value includes: The target load state is determined according to the first estimated load value and the second estimated load value.

7. The method according to claim 6, characterized in that Determining the second estimated load value of the target vehicle at the current moment according to the real-time longitudinal acceleration and the peak opening data includes: Performing a division operation on the real-time longitudinal acceleration and the peak opening data to obtain the load coefficient at the current moment; According to the load coefficient and a pre-established mapping relationship table between the load coefficient and the load value, the load value corresponding to the load coefficient is determined as the second estimated load value.

8. The method according to claim 1, characterized in that Before determining the target load state of the target vehicle at the current moment according to the first estimated load value, the method further includes: Calculating the longitudinal driving force of the target vehicle at the current moment according to the motor torque value of the target vehicle at the current moment; Calculate the longitudinal resultant external force of the target vehicle at the current moment according to the real-time longitudinal speed of the target vehicle at the current moment, the preset air density, the preset drag coefficient, and the preset road slope; Calculating a third estimated load value of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force; Determining the target load state of the target vehicle at the current moment according to the first estimated load value includes: The target load state is determined according to the first estimated load value and the third estimated load value.

9. The method according to claim 8, characterized in that The calculating, according to the longitudinal driving force and the longitudinal resultant external force, a third estimated load value of the target vehicle at the current moment includes: Calculating a target longitudinal force of the target vehicle at the current moment according to the longitudinal driving force and the longitudinal resultant external force; The third estimated load value is calculated according to the target longitudinal force, the longitudinal speed at the current moment and the longitudinal speed at the previous moment by using a preset kinetic energy theorem equation.

10. A vehicle control device, characterized in that: The vehicle control device includes: a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, and when the vehicle control device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the vehicle load estimation method described in any one of claims 1 to 9.

Images