A Deep Learning-Based Method and Apparatus for Determining Vehicle Load

Abstract

This invention provides a method and apparatus for determining vehicle load based on deep learning. The method includes: acquiring the acceleration of a vehicle with a load to be determined; extracting temporal feature data of the acceleration based on a pre-established long short-term memory network model; and determining the load of the vehicle based on the temporal feature data and a pre-established fully connected neural network model. This invention relies solely on the vehicle's speed, utilizing the powerful feature extraction and learning capabilities of artificial neural networks to estimate the vehicle load, avoiding overfitting caused by excessive information dimensions and improving model portability.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation detection technology, specifically to the application of artificial intelligence technology and intelligent connected vehicle technology in the transportation field, and particularly to a method and device for determining vehicle load based on deep learning. Background Technology

[0002] Commercial vehicles include both freight and passenger vehicles. Currently, load monitoring is primarily focused on freight vehicles, and vehicle load monitoring systems are the main means of controlling vehicle overloading in my country. However, the load issue of passenger vehicles has not received sufficient attention, and my country has experienced several major accidents caused by overloaded luggage compartments in buses. Therefore, it is necessary to comprehensively monitor the load capacity of commercial vehicles through load measurement technology to effectively alleviate infrastructure damage and traffic safety problems caused by overloading. Summary of the Invention

[0003] To address the problems in existing technologies, the present invention provides a vehicle load determination method and apparatus based on deep learning. This method estimates the vehicle load by relying solely on the vehicle's speed and utilizing the powerful feature extraction and learning capabilities of artificial neural networks. This avoids the model overfitting problem caused by too many information dimensions and improves the model's portability.

[0004] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0005] In a first aspect, the present invention provides a vehicle load determination method based on deep learning, comprising:

[0006] Obtain the acceleration of the vehicle with the unknown load.

[0007] The temporal feature data of the acceleration are extracted based on a pre-established long short-term memory network model;

[0008] The load of the vehicle to be determined is determined based on the time-series feature data and the pre-established fully connected neural network model.

[0009] In one embodiment, the acceleration is a three-dimensional acceleration, and further includes:

[0010] The three-dimensional jerk of the vehicle to be determined is generated based on the three-dimensional acceleration.

[0011] In one embodiment, establishing the Long Short-Term Memory network model includes the following steps:

[0012] Establish the initial model of the Long Short-Term Memory network model;

[0013] The input layer of the initial model is used to learn the forward and reverse time series features of the three-dimensional acceleration and the three-dimensional jerk.

[0014] The spatiotemporal feature data of the three-dimensional acceleration and the three-dimensional jerk are learned using the hidden layer of the initial model;

[0015] The Long Short-Term Memory (LSTM) network model is established based on the learned input layer and hidden layer.

[0016] In one embodiment, the step of extracting the temporal feature data of the acceleration based on a pre-established long short-term memory network model includes:

[0017] Extract the state features of each node in the hidden layer;

[0018] The state features are flattened using a matrix vectorization method to obtain the time-series feature data.

[0019] In one embodiment, establishing the fully connected neural network model includes the following steps:

[0020] Establish the initial model of the fully connected neural network model;

[0021] The initial model is trained using the time-series feature data and a preset error threshold to establish the connection neural network model.

[0022] In one embodiment, determining the load of the vehicle to be determined based on the time-series feature data and a pre-established fully connected neural network model includes:

[0023] The point estimation calculation is performed on the vehicle to be determined based on the fully connected neural network model to determine the load of the vehicle.

[0024] Secondly, the present invention provides a vehicle load determination device based on deep learning, comprising:

[0025] An acceleration acquisition module is used to acquire the acceleration of the vehicle whose load is to be determined.

[0026] The temporal feature extraction module is used to extract the temporal feature data of the acceleration based on a pre-established long short-term memory network model;

[0027] The load determination module is used to determine the load of the vehicle to be determined based on the time-series feature data and a pre-established fully connected neural network model.

[0028] In one embodiment, the acceleration is three-dimensional acceleration, and the deep learning-based vehicle load determination device further includes:

[0029] The jerkness determination module is used to generate the three-dimensional jerkness of the vehicle to be determined based on the three-dimensional acceleration.

[0030] In one embodiment, the deep learning-based vehicle load determination device further includes: a memory model building module, used to build the long short-term memory network model, the memory model building module comprising:

[0031] The initial model building unit is used to build the initial model of the Long Short-Term Memory Network model.

[0032] The time-series data learning unit is used to learn the forward and reverse time-series features of the three-dimensional acceleration and the three-dimensional jerk using the input layer of the initial model;

[0033] The spatiotemporal data learning unit is used to learn the spatiotemporal feature data of the three-dimensional acceleration and the three-dimensional jerk using the hidden layer of the initial model;

[0034] The memory model building unit is used to build the long short-term memory network model based on the learned input layer and hidden layer.

[0035] In one embodiment, the temporal feature extraction module includes:

[0036] A node feature extraction unit is used to extract the state features of each node in the hidden layer;

[0037] The state feature flattening unit is used to flatten the state features using a matrix vectorization method to obtain the time series feature data.

[0038] In one embodiment, the deep learning-based vehicle load determination device further includes: a fully connected model building module, used to build the fully connected neural network model, the fully connected model building module comprising:

[0039] A fully connected initial module establishment unit is used to establish the initial model of the fully connected neural network model;

[0040] The fully connected model building unit is used to train the initial model with the time-series feature data and a preset error threshold to build the connected neural network model.

[0041] In one embodiment, the load determination module includes:

[0042] The load determination unit is used to perform point estimation calculations on the vehicle to be determined based on the fully connected neural network model, so as to determine the load of the vehicle to be determined.

[0043] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a deep learning-based method for determining vehicle load.

[0044] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a deep learning-based method for determining vehicle load.

[0045] As described above, the vehicle load determination method and apparatus based on deep learning provided in this embodiment of the invention first acquires the acceleration of the vehicle to be loaded; then, it extracts the temporal feature data of the acceleration based on a pre-established long short-term memory network model; finally, it determines the load of the vehicle to be loaded based on the temporal feature data and a pre-established fully connected neural network model. This invention relies solely on the vehicle's speed and acceleration data, utilizing the powerful feature extraction and learning capabilities of artificial neural networks to estimate the vehicle load, avoiding the model overfitting problem caused by too many information dimensions and improving the model's portability. Simultaneously, by considering the time-series characteristics of the vehicle's driving data, the model ensures measurement accuracy. Furthermore, the method proposed in this application can be integrated through a network platform, enabling online cross-testing between multiple types of vehicle data, further improving the model's generalization ability while achieving large-scale real-time monitoring of multiple vehicles. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of a first structure of a deep learning-based vehicle load determination system according to an embodiment of this application;

[0048] Figure 2 This is a schematic diagram of a second structure of a vehicle load determination system based on deep learning, according to an embodiment of this application.

[0049] Figure 3 This is a flowchart illustrating a deep learning-based vehicle load determination method in an embodiment of the present invention. Figure 1 ;

[0050] Figure 4 This is a flowchart illustrating a deep learning-based vehicle load determination method in an embodiment of the present invention. Figure 2 ;

[0051] Figure 5 This is a flowchart illustrating a deep learning-based vehicle load determination method in an embodiment of the present invention. Figure 3 ;

[0052] Figure 6 This is a flowchart illustrating step 500 in an embodiment of the present invention;

[0053] Figure 7 This is a flowchart illustrating step 200 in an embodiment of the present invention;

[0054] Figure 8 This is a flowchart illustrating a deep learning-based vehicle load determination method in an embodiment of the present invention. Figure 4 ;

[0055] Figure 9 This is a flowchart illustrating step 600 in an embodiment of the present invention;

[0056] Figure 10 This is a flowchart illustrating step 300 in an embodiment of the present invention;

[0057] Figure 11 This is a flowchart illustrating the deep learning-based vehicle load determination method in a specific application example of the present invention.

[0058] Figure 12 This is a schematic diagram of the overall framework of the deep learning-based vehicle load determination method in an embodiment of the present invention.

[0059] Figure 13 This is a structural frame of the deep learning-based vehicle load determination device in an embodiment of the present invention. Figure 1 ;

[0060] Figure 14 This is a structural frame of the deep learning-based vehicle load determination device in an embodiment of the present invention. Figure 2 ;

[0061] Figure 15 This is a structural frame of the deep learning-based vehicle load determination device in an embodiment of the present invention. Figure 3 ;

[0062] Figure 16 This is a schematic diagram of the memory model establishment module 50 in a specific application example of the present invention;

[0063] Figure 17 This is a schematic diagram of the temporal feature extraction module 20 in a specific application example of the present invention;

[0064] Figure 18 This is a structural frame of the deep learning-based vehicle load determination device in an embodiment of the present invention. Figure 4 ;

[0065] Figure 19 This is a schematic diagram of the fully connected model establishment module 60 in a specific application example of the present invention;

[0066] Figure 20 This is a schematic diagram of the load determination module 30 in a specific application example of the present invention;

[0067] Figure 21 This is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] In one embodiment, this application also provides a vehicle load determination system based on deep learning, see [link to relevant documentation]. Figure 1 The system can be a server A1, which can communicate with multiple preset acceleration sensing devices B1 on vehicles. Server A1 can also communicate with multiple databases, or as... Figure 2 As shown, these databases can also be set up directly on server A1. Accelerometer B1 is used to measure the acceleration data of the vehicle to be loaded in real time. After receiving the acceleration data, server A1 extracts the temporal feature data of the acceleration using a long short-term memory network model; then, based on the temporal feature data and a fully connected neural network model, it determines the load of the vehicle to be loaded. This estimated load data is then displayed to the user through client C1.

[0070] It is understandable that client C1 can include smartphones, tablets, set-top boxes, laptops, desktops, personal digital assistants (PDAs), in-vehicle devices, and smart wearable devices. Among these, smart wearable devices can include smart glasses, smartwatches, and smart bracelets.

[0071] In practical applications, the temporal feature data of acceleration is extracted based on the Long Short-Term Memory network model; then, the load of the vehicle to be determined is determined based on the temporal feature data and the fully connected neural network model; the results display can be executed on server A1 as described above, i.e., as... Figure 1 or Figure 2The architecture shown can also be implemented entirely within the client C1 device. The choice can be made based on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations on this. If all operations are performed on the client device, the client device may further include a processor for operations such as extracting temporal feature data of acceleration based on a long short-term memory network model, and determining the load of the vehicle to be determined based on the temporal feature data and a fully connected neural network model.

[0072] The aforementioned client C1 device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission. The server may include a server that extracts temporal feature data of acceleration based on a Long Short-Term Memory (LSTM) network model, determines the load of the vehicle to be determined based on the temporal feature data and a fully connected neural network model, and displays the results. Other implementation scenarios may also include an intermediate platform server, such as a third-party server platform server that has a communication link with the prediction server that extracts temporal feature data of acceleration based on the LSM network model, determines the load of the vehicle to be determined based on the temporal feature data and a fully connected neural network model, and displays the results. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0073] The server and client devices can communicate using any suitable network protocol, including those not yet developed as of the date of this application. Network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Of course, network protocols may also include, for example, RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer) protocols used on top of the aforementioned protocols.

[0074] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0075] An embodiment of the present invention provides a specific implementation of a deep learning-based vehicle load determination method, see below. Figure 3 The method specifically includes the following:

[0076] Step 100: Obtain the acceleration of the vehicle with the load to be determined.

[0077] In recent years, there has been considerable research on vehicle-mounted weighing technologies. For trucks, vehicle-mounted weighing methods measure the load by measuring the deformation of load-bearing structural components under different load conditions. For example, measuring the load using leaf springs requires a pressure sensor to be installed at the connection between the leaf spring and the axle, resulting in installation difficulties, high costs, complex debugging procedures, and difficult maintenance. Furthermore, it necessitates changes to the vehicle structure, significantly impacting driving stability. The measurement is also easily affected by environmental interference, affecting accuracy. However, this method is unsuitable for commercial vehicles. Measurements using ultrasonic and resistive sensing of vehicle component changes are also susceptible to the influence of air temperature and humidity. Measurements using tire pressure changes are similarly affected by the sensitivity of tire deformation to ambient temperature, increasing measurement disturbances.

[0078] To collect acceleration data for a vehicle with a determined load, an acceleration sensing device needs to be installed on the vehicle to measure its acceleration during operation. Specifically, using the vehicle's onboard acceleration sensing device, multiple related sensors are fused to obtain the vehicle's acceleration in real time, quickly, and accurately, and this data is used to accurately estimate the vehicle's load weight. A typical acceleration sensing device is the piezoelectric accelerometer, also known as a piezoelectric accelerometer. It belongs to the inertial sensor category. It utilizes the piezoelectric effect of certain materials, such as quartz crystals. When the accelerometer is subjected to vibration, the force exerted by the mass block on the piezoelectric element changes accordingly. When the frequency of the measured vibration is much lower than the natural frequency of the accelerometer, the change in force is proportional to the measured acceleration. Piezoelectric accelerometers operate based on the piezoelectric effect of piezoelectric crystals. When certain crystals are deformed by force in a specific direction, polarization occurs within them, and opposite charges are generated on their two surfaces. When the external force is removed, they return to their uncharged state. This phenomenon is called the "piezoelectric effect," and crystals exhibiting the piezoelectric effect are called piezoelectric crystals. Commonly used piezoelectric crystals include quartz and piezoelectric ceramics.

[0079] Step 200: Extract the temporal feature data of the acceleration based on the pre-established long short-term memory network model.

[0080] Long Short-Term Memory (LSTM) networks are a type of recurrent neural network specifically designed to address the long-term dependency problem inherent in general RNNs (Recurrent Neural Networks). All RNNs have a chain-like structure of repeating neural network modules. In a standard RNN, this repeating structural module has only one very simple structure.

[0081] Specifically, an appropriate Long Short-Term Memory (LSTM) neural network structure is selected to extract the temporal features of truck acceleration and jerk. Based on vehicle dynamics theory, vehicle weight information is related to its acceleration and jerk, and these characteristics have time-series features. Therefore, an LSTM network is needed to extract these temporal features. The LSTM network consists of LSTM network layers and LSTM network hidden layers.

[0082] Step 300: Determine the load of the vehicle to be determined based on the time-series feature data and the pre-established fully connected neural network model.

[0083] A fully connected neural network model is a type of multilayer perceptron (MLP). The principle of a perceptron is to find the most reasonable and robust hyperplane between categories. The most representative perceptron algorithm is the Support Vector Machine (SVM). Neural networks draw inspiration from both perceptrons and bionics. Typically, when an animal's nervous system receives a signal, it sends signals to its various neurons. Each neuron, upon receiving the input, makes its own judgment, activates, and generates an output signal, which is then aggregated to achieve the identification and classification of the information source. Training a fully connected neural network consists of two processes: forward propagation and backpropagation. Forward propagation calculates the loss function value after the data travels from the input to the output. Backpropagation is an optimization process that uses gradient descent to reduce the loss function value generated during forward propagation, thereby optimizing and updating the parameters.

[0084] As described above, the deep learning-based vehicle load determination method provided in this embodiment of the invention first obtains the acceleration of the vehicle to be loaded; then, it extracts temporal feature data of the acceleration based on a pre-established long short-term memory network model; finally, it determines the load of the vehicle to be loaded based on the temporal feature data and a pre-established fully connected neural network model. The deep learning-based vehicle load determination method provided in this embodiment of the invention can accurately, conveniently, and efficiently estimate the vehicle's load information.

[0085] In one embodiment, the acceleration is a three-dimensional acceleration, see [link to relevant documentation]. Figure 4 Deep learning-based methods for determining vehicle load also include:

[0086] Step 400: Generate the three-dimensional jerk of the vehicle to be determined based on the three-dimensional acceleration.

[0087] Three-dimensional acceleration and three-dimensional jerkness refer to the acceleration and jerkness of a vehicle in the longitudinal, lateral, and vertical directions, i.e., the x, y, and z directions. Jerkness, also called force rate of change, is the rate of change of acceleration, i.e., the third derivative of the position vector with respect to time. In other words, jerkness is the first difference of acceleration.

[0088] In one embodiment, see Figure 5Deep learning-based methods for determining vehicle load also include:

[0089] Step 500: Establish the Long Short-Term Memory network model.

[0090] Further, see Figure 6 Step 500 includes:

[0091] Step 501: Establish the initial model of the Long Short-Term Memory network model;

[0092] Step 502: Use the input layer of the initial model to learn the forward and reverse time series features of the three-dimensional acceleration and the three-dimensional jerk.

[0093] Specifically, the input layer of the initial model is used to learn the forward and reverse time series features of vehicle acceleration and jerkiness in the time dimension.

[0094] Step 503: Use the hidden layer of the initial model to learn the spatiotemporal feature data of the three-dimensional acceleration and the three-dimensional jerk.

[0095] A bidirectional long short-term memory network hidden layer is established to further learn the temporal and spatial coupling relationship between three-dimensional acceleration and three-dimensional jerk in various dimensions, and fully extract the spatiotemporal features between vehicle acceleration and jerk.

[0096] Step 504: Establish the Long Short-Term Memory network model based on the learned input layer and hidden layer.

[0097] In one embodiment, see Figure 7 Step 200 includes:

[0098] Step 201: Extract the state features of each node in the hidden layer;

[0099] Step 202: Flatten the state features using a matrix vectorization method to obtain the time series feature data.

[0100] In steps 201 and 202, the state features of each bidirectional long short-term memory node in the hidden layer of each bidirectional long short-term memory network are extracted. A state feature matrix is ​​constructed with each hidden layer as the row index and the hidden layer nodes as the column index. The matrix is ​​rearranged so that each row is connected to form a new row vector, thereby flattening the state features of all nodes to obtain the spatiotemporal features of vehicle acceleration and jerk.

[0101] In one embodiment, see Figure 8 Deep learning-based methods for determining vehicle load also include:

[0102] Step 600: Establish the fully connected neural network model. Further, see... Figure 9 Step 600 includes:

[0103] Step 601: Establish the initial model of the fully connected neural network model;

[0104] Step 602: Train the initial model with the time-series feature data and a preset error threshold to establish the connection neural network model.

[0105] The spatiotemporal features of vehicle acceleration and jerk extracted in step 202 are used as input to the fully connected neural network. Simultaneously, hyperparameters of the model, such as the objective function, optimizer type, learning rate, and dataset splitting ratio, are set. Next, the parameters of the fully connected neural network and the time-series spatiotemporal feature extraction network are updated by backpropagating the error obtained from the objective function, thus completing the overall network parameter training.

[0106] In one embodiment, see Figure 10 Step 300 specifically includes:

[0107] Step 301: Perform point estimation calculation on the vehicle to be determined based on the fully connected neural network model to determine the load of the vehicle to be determined.

[0108] Specifically, the vehicle load point estimation results are marked. The marked vehicle load point estimation values ​​are then divided into fixed weight intervals. The weight interval with the highest coverage point estimation value is determined to ultimately determine the vehicle's load value. The final load value is calculated using the following formula:

[0109]

[0110] Among them, w a ,w b These represent the boundary weight values ​​of the weight interval with the most estimated coverage points.

[0111] To further illustrate this solution, this invention provides a specific application example of a deep learning-based vehicle load determination method, using passenger vehicles as an example. This specific application example includes the following (see details). Figure 11 .

[0112] In one embodiment, see Figure 12 A framework for vehicle load estimation was established by combining data processing and the aforementioned deep learning modules. This framework includes preprocessing of acceleration and jerk data; an LSTM input layer for learning temporal features; an LSTM hidden layer for learning spatiotemporal features; a flattening layer that flattens state node features using vectorization; and a fully connected layer that estimates weight using these features. The final output is the vehicle load.

[0113] Non-intrusive vehicle weighing technology, which indirectly measures and estimates vehicle load through the vehicle's dynamic characteristics, has been explored by researchers in recent years. For example, some scholars, based on dynamic equations, have obtained parameters such as traction force, braking force, tire slip ratio, acceleration, angular velocity, and speed, and used the least squares method and Kalman filtering to conduct weight estimation experiments on SUV test vehicles. Similarly, by utilizing the kinematic characteristics of passenger cars, torque and vehicle motion parameters have been used to estimate vehicle weight and gradient using recursive least squares and Kalman filtering methods, respectively. Furthermore, by using CAN bus data, combined with vehicle driving parameters such as gear position and engine torque, and road gradient coefficients, a vehicle load analytical model based on the characteristics of passenger cars and road operation has been constructed, achieving high measurement accuracy. However, all of these studies require obtaining vehicle dynamic parameters such as traction force or engine torque, and are limited to passenger cars, not applicable to commercial vehicles.

[0114] Indirect measurement and estimation of load based on the dynamic characteristics of large vehicles such as commercial vehicles is an emerging vehicle-mounted weight estimation technology. Research on this technology is limited, and it also relies on truck dynamic parameters such as traction force and engine torque, as well as road environment information such as road slope, to estimate the vehicle's load. However, different types of trucks have significantly different characteristic values ​​for various parameters due to structural differences. Too many parameters often lead to poor model generalization ability, making it impossible to directly transfer to other types of vehicles. Furthermore, this technology is still in its early stages; limited by experimental conditions, few studies have built models specifically for the motion characteristics of commercial vehicles. In addition, commercial vehicle driving data has time-series characteristics, and the aforementioned models and methods lack consideration for these time-series features, limiting further improvements in measurement accuracy.

[0115] S1: Acquire three-dimensional acceleration and three-dimensional jerk data during vehicle movement.

[0116] S2: Obtain a point estimate of the vehicle's load based on the vehicle's three-dimensional acceleration and three-dimensional jerk data, as well as a pre-established long short-term memory network.

[0117] Specifically, based on Newton's second law of motion, which states that the magnitude of an object's acceleration is directly proportional to the force acting on it and inversely proportional to its mass, the vehicle's mass can be indirectly calculated using its acceleration value.

[0118] Specifically, firstly, a vehicle acceleration feature extraction model based on a long short-term memory network is used to extract the time-series features of the vehicle's three-dimensional acceleration and three-dimensional jerk. Then, using these features as input, a pre-constructed fully connected neural network is used to estimate the vehicle's load at points. The method proposed in step 200 can accurately calculate the time-point estimate of the vehicle's load.

[0119] S3: Based on the point estimation results of the vehicle load, further estimate the travel distance to finally determine the vehicle's load weight.

[0120] Specifically, firstly, the estimated load points of the vehicle during the travel period are marked; then, the estimated load points are divided into fixed weight intervals; finally, the interval with the most covered estimated load points is determined to obtain the accurate estimated value of the vehicle. Using the method proposed in step 300, the load weight value of the vehicle for each segment of the journey can be calculated more accurately.

[0121] As described above, the deep learning-based vehicle load determination method provided in this embodiment of the invention first obtains the acceleration of the vehicle to be loaded; then, it extracts the temporal feature data of the acceleration based on a pre-established long short-term memory network model; finally, it determines the load of the vehicle to be loaded based on the temporal feature data and a pre-established fully connected neural network model. This invention relies solely on the vehicle's speed and acceleration data, utilizing the powerful feature extraction and learning capabilities of artificial neural networks to estimate the vehicle load, avoiding the model overfitting problem caused by too many information dimensions and improving the model's portability. Simultaneously, by considering the time-series characteristics of vehicle driving data, the model ensures measurement accuracy. Furthermore, the method proposed in this application can be integrated through a networked platform, enabling online cross-testing between multiple types of vehicle data, further improving the model's generalization ability while achieving large-scale real-time monitoring of multiple vehicles.

[0122] Based on the same inventive concept, this application also provides a vehicle load determination device based on deep learning, which can be used to implement the method described in the above embodiments, as shown in the following embodiments. Since the principle of the vehicle load determination device based on deep learning is similar to that of the vehicle load determination method based on deep learning, the implementation of the vehicle load determination device based on deep learning can refer to the implementation of the vehicle load determination method based on deep learning, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0123] Embodiments of the present invention provide a specific implementation of a deep learning-based vehicle load determination device capable of implementing a deep learning-based vehicle load determination method. See [link to specific implementation details]. Figure 13 The deep learning-based vehicle load determination device specifically includes the following components:

[0124] Acceleration acquisition module 10 is used to acquire the acceleration of the vehicle to be loaded.

[0125] The temporal feature extraction module 20 is used to extract the temporal feature data of the acceleration based on a pre-established long short-term memory network model;

[0126] The load determination module 30 is used to determine the load of the vehicle to be determined based on the time-series feature data and a pre-established fully connected neural network model.

[0127] In one embodiment, the acceleration is a three-dimensional acceleration, see [link to relevant documentation]. Figure 14 The deep learning-based vehicle load determination device also includes:

[0128] The jerkness determination module 40 is used to generate the three-dimensional jerkness of the load-bearing vehicle to be determined based on the three-dimensional acceleration.

[0129] In one embodiment, see Figure 15 The deep learning-based vehicle load determination device also includes: a memory model building module 50, used to build the long short-term memory network model, see [link to relevant documentation]. Figure 16 The memory model building module 50 includes:

[0130] The initial model establishment unit 501 is used to establish the initial model of the Long Short-Term Memory Network model;

[0131] The time-series data learning unit 502 is used to learn the forward and reverse time-series features of the three-dimensional acceleration and the three-dimensional jerk using the input layer of the initial model;

[0132] The spatiotemporal data learning unit 503 is used to learn the spatiotemporal feature data of the three-dimensional acceleration and the three-dimensional jerk using the hidden layer of the initial model;

[0133] The memory model building unit 504 is used to build the long short-term memory network model based on the learned input layer and hidden layer.

[0134] In one embodiment, see Figure 17 The time-series feature extraction module 20 includes:

[0135] The node feature extraction unit 201 is used to extract the state features of each node in the hidden layer;

[0136] The state feature flattening unit 202 is used to flatten the state features using a matrix vectorization method to obtain the time series feature data.

[0137] In one embodiment, see Figure 18 The deep learning-based vehicle load determination device also includes: a fully connected model building module 60, used to build the fully connected neural network model, see [link to relevant documentation]. Figure 19The fully connected model building module 60 includes:

[0138] The fully connected initial module establishment unit 601 is used to establish the initial model of the fully connected neural network model;

[0139] The fully connected model building unit 602 is used to train the initial model with the time-series feature data and a preset error threshold to build the connected neural network model.

[0140] In one embodiment, see Figure 20 The load determination module 30 includes:

[0141] The load determination unit 301 is used to perform point estimation calculations on the vehicle to be determined based on the fully connected neural network model, so as to determine the load of the vehicle to be determined.

[0142] As described above, the deep learning-based vehicle load determination device provided in this embodiment of the invention first acquires the acceleration of the vehicle to be loaded; then, it extracts the temporal feature data of the acceleration based on a pre-established long short-term memory network model; finally, it determines the load of the vehicle to be loaded based on the temporal feature data and a pre-established fully connected neural network model. This invention relies solely on the vehicle's speed and acceleration data, utilizing the powerful feature extraction and learning capabilities of artificial neural networks to estimate the vehicle load, avoiding the model overfitting problem caused by too many information dimensions and improving the model's portability. Simultaneously, by considering the time-series characteristics of the vehicle's driving data, the model ensures measurement accuracy. Furthermore, the method proposed in this application can be integrated through a network platform, enabling online cross-testing between multiple types of vehicle data, further improving the model's generalization ability while achieving large-scale real-time monitoring of multiple vehicles.

[0143] The embodiments of this application also provide a specific implementation of an electronic device capable of implementing all steps in the deep learning-based vehicle load determination method described in the above embodiments. See [link to relevant documentation]. Figure 21 The electronic devices specifically include the following:

[0144] Processor 1201, memory 1202, communications interface 1203, and bus 1204;

[0145] The processor 1201, memory 1202, and communication interface 1203 communicate with each other via bus 1204; the communication interface 1203 is used to realize information transmission between server-side devices, sensors, and client-side devices and other related devices.

[0146] The processor 1201 is used to call the computer program in the memory 1202. When the processor executes the computer program, it implements all the steps in the deep learning-based vehicle load determination method in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0147] Step 100: Obtain the acceleration of the vehicle with the load to be determined;

[0148] Step 200: Extract the temporal feature data of the acceleration based on the pre-established long short-term memory network model;

[0149] Step 300: Determine the load of the vehicle to be determined based on the time-series feature data and the pre-established fully connected neural network model.

[0150] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the deep learning-based vehicle load determination method in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the deep learning-based vehicle load determination method in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0151] Step 100: Obtain the acceleration of the vehicle with the load to be determined;

[0152] Step 200: Extract the temporal feature data of the acceleration based on the pre-established long short-term memory network model;

[0153] Step 300: Determine the load of the vehicle to be determined based on the time-series feature data and the pre-established fully connected neural network model.

[0154] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, hardware + program embodiments are relatively simple in description because they are fundamentally similar to method embodiments; relevant parts can be referred to the descriptions in the method embodiments.

[0155] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0156] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0158] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0159] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0160] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for determining vehicle load based on deep learning, characterized in that, include: Obtain the acceleration of the vehicle with the unknown load. The temporal feature data of the acceleration are extracted based on a pre-established long short-term memory network model; The load of the vehicle to be determined is determined based on the time-series feature data and the pre-established fully connected neural network model. The acceleration is a three-dimensional acceleration, and the method for determining the vehicle load also includes: The three-dimensional jerk of the vehicle to be determined is generated based on the three-dimensional acceleration. Establishing the Long Short-Term Memory network model includes the following steps: Establish the initial model of the Long Short-Term Memory network model; The input layer of the initial model is used to learn the forward and reverse time series features of the three-dimensional acceleration and the three-dimensional jerk. The spatiotemporal feature data of the three-dimensional acceleration and the three-dimensional jerk are learned using the hidden layer of the initial model; The Long Short-Term Memory network model is established based on the learned input layer and hidden layer. The step of extracting the temporal feature data of the acceleration based on a pre-established long short-term memory network model includes: Extract the state features of each node in the hidden layer; The state features are flattened using a matrix vectorization method to obtain the time-series feature data, specifically: Extract the state features of each bidirectional long short-term memory node in the hidden layer of each bidirectional long short-term memory network. Construct a state feature matrix with each hidden layer as the row index and the hidden layer nodes as the column index. Rearrange the state feature matrix so that each row is connected to form a new row vector, thereby flattening the state features of all nodes to obtain the spatiotemporal features of vehicle acceleration and jerk.

2. The vehicle load determination method as described in claim 1, characterized in that, Establishing the fully connected neural network model includes the following steps: Establish the initial model of the fully connected neural network model; The initial model is trained using the time-series feature data and a preset error threshold to establish the connection neural network model.

3. The vehicle load determination method as described in claim 1, characterized in that, Determining the load of the vehicle to be determined based on the time-series feature data and a pre-established fully connected neural network model includes: The point estimation calculation is performed on the vehicle to be determined based on the fully connected neural network model to determine the load of the vehicle.

4. A vehicle load determination device based on deep learning, characterized in that, include: An acceleration acquisition module is used to acquire the acceleration of the vehicle with the load to be determined. The temporal feature extraction module is used to extract the temporal feature data of the acceleration based on a pre-established long short-term memory network model; The load determination module is used to determine the load of the vehicle to be determined based on the time-series feature data and a pre-established fully connected neural network model. The acceleration is a three-dimensional acceleration, and the vehicle load determination device further includes: The jerkness determination module is used to generate the three-dimensional jerkness of the vehicle to be determined based on the three-dimensional acceleration. The memory model building module is used to build the Long Short-Term Memory (LSTM) network model, and the memory model building module includes: The initial model building unit is used to build the initial model of the Long Short-Term Memory Network model. The time-series data learning unit is used to learn the forward and reverse time-series features of the three-dimensional acceleration and the three-dimensional jerk using the input layer of the initial model; The spatiotemporal data learning unit is used to learn the spatiotemporal feature data of the three-dimensional acceleration and the three-dimensional jerk using the hidden layer of the initial model; The memory model building unit is used to build the long short-term memory network model based on the learned input layer and hidden layer. The temporal feature extraction module includes: A node feature extraction unit is used to extract the state features of each node in the hidden layer; The state feature flattening unit is used to flatten the state features using a matrix vectorization method to obtain the time-series feature data, specifically: Extract the state features of each bidirectional long short-term memory node in the hidden layer of each bidirectional long short-term memory network. Construct a state feature matrix with each hidden layer as the row index and the hidden layer nodes as the column index. Rearrange the state feature matrix so that each row is connected to form a new row vector, thereby flattening the state features of all nodes to obtain the spatiotemporal features of vehicle acceleration and jerk.

5. The vehicle load determination device as described in claim 4, characterized in that, Also includes: A fully connected model building module is used to build the fully connected neural network model. The fully connected model building module includes: A fully connected initial module establishment unit is used to establish the initial model of the fully connected neural network model; The fully connected model building unit is used to train the initial model with the time-series feature data and a preset error threshold to build the connected neural network model.

6. The vehicle load determination device as described in claim 4, characterized in that, The load determination module includes: The load determination unit is used to perform point estimation calculations on the vehicle to be determined based on the fully connected neural network model, so as to determine the load of the vehicle to be determined.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the deep learning-based vehicle load determination method according to any one of claims 1 to 3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the deep learning-based vehicle load determination method according to any one of claims 1 to 3.

Images