Embedded neural network reasoning method, device and equipment for vehicle-mounted vibration energy

Abstract

The invention provides an embedded neural network reasoning method, device and equipment for vehicle-mounted vibration energy, and relates to the technical field of artificial intelligence, and the reasoning method comprises the steps: in the active energy collection process of vehicle-mounted vibration, obtaining energy collection data, energy storage data and energy consumption data of a vehicle, obtaining multi-early-exit exit structure data and each exit reasoning result of the embedded neural network; based on a preset Markov decision process model, integrating the energy acquisition data, the energy storage data and the energy consumption data to obtain a current energy state; and according to a reward rule of the Markov decision process model, based on a preset deep reinforcement learning agent, making a decision on the current energy state, the multi-early-exit exit structure data and each exit reasoning result to obtain a decision result. According to the method, the neural network early exit is dynamically selected, the reasoning precision is preferentially guaranteed when energy is sufficient, the energy consumption is emphasized to be reduced when the energy is insufficient, and intelligent balance of the precision and the energy consumption is realized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to an embedded neural network reasoning method, apparatus, and device for vehicle vibration energy. Background Technology

[0002] In automotive embedded systems, neural network inference technology is widely used for critical tasks such as environmental perception and state monitoring. However, its stable operation is highly dependent on a continuous energy supply. Currently, automotive environments often utilize vibration energy harvesting technology to obtain renewable energy to alleviate the range pressure of traditional battery power. However, the generation of vibration energy is affected by factors such as road conditions and vehicle driving status, exhibiting significant randomness. This leads to drastic fluctuations in energy supply, which can easily result in insufficient energy reserves interrupting inference tasks, or overcharging leading to resource waste.

[0003] Meanwhile, embedded neural network inference faces an inherent contradiction between accuracy and energy consumption: while deep networks can provide higher inference accuracy, they are accompanied by a significant increase in energy consumption; shallow networks have lower energy consumption, but struggle to meet the demands of high-precision inference. Existing technologies generally employ fixed inference paths, making it difficult to dynamically adjust inference strategies based on real-time energy conditions. This frequently leads to problems with efficiency or low inference accuracy. For example, when energy is sufficient, the lack of a high-precision inference mode results in performance redundancy; when energy is insufficient, continuous high-energy-consumption inference causes task interruptions. More critically, the operating parameters (such as vibration response speed) of current vehicle-mounted vibration energy acquisition devices are mostly fixed settings, making it difficult to adapt to changes in energy supply and demand: when energy is scarce, acquisition efficiency is low, and it is difficult to quickly replenish energy storage; when energy is abundant, high-power acquisition is maintained, resulting in ineffective energy consumption, which further exacerbates the energy supply and demand contradiction in vehicle scenarios. Summary of the Invention

[0004] The problem solved by this invention is one or more of the aforementioned related technical problems.

[0005] To address the above problems, this invention provides an embedded neural network inference method, apparatus, and device for vehicle vibration energy.

[0006] In a first aspect, the present invention provides an embedded neural network inference method for vehicle vibration energy, comprising: During the active energy harvesting process of vehicle vibration, energy harvesting data, energy storage data, energy consumption data of the vehicle, as well as multi-early exit structure data of the embedded neural network and inference results of each exit are acquired. Based on a pre-defined Markov decision process model, the energy acquisition data, energy storage data, and energy consumption data are integrated to obtain the current energy state. According to the reward rule of the Markov decision process model, a deep reinforcement learning agent makes a decision on the current energy state, the multiple early exit structure data, and the inference results of each exit based on a preset deep reinforcement learning agent, and obtains a decision result. The decision result includes the early exit selection of the embedded neural network and the speed parameters of the vehicle's on-board vibration energy acquisition device.

[0007] Optionally, the energy acquisition data includes real-time acquisition data and historical cumulative acquisition data, the energy storage data includes real-time storage data, and the energy consumption data includes real-time consumption data and historical cumulative consumption data; the integration of the energy acquisition data, energy storage data, and energy consumption data based on a preset Markov decision process model to obtain the current energy state includes: Get the data of the last decision action; The real-time collected data, the historical cumulative collected data, the real-time stored data, the real-time consumption data, the historical cumulative consumption data, and the data of the previous decision action are integrated and processed in a preset dimension order to obtain the current energy state.

[0008] Optionally, the current energy state includes: ; in, The current energy state, For the real-time stored data, The historical cumulative data, For the real-time collected data, The historical cumulative consumption data, The real-time consumption data, This refers to the data from the previous decision-making action.

[0009] Optionally, the deep reinforcement learning agent includes a self-attention mechanism; the deep reinforcement learning agent based on a preset decision makes decisions based on the current energy state, the multiple early exit structure data, and the inference results of each exit, obtaining decision results including: The current energy state is processed according to the self-attention mechanism to obtain the state features of each dimension of the current energy state, including: decomposing each dimension of the current energy state to obtain the corresponding query matrix, key matrix and value matrix; obtaining the attention weight matrix of each dimension of the data according to each query matrix and the corresponding key matrix; and fusing each attention weight matrix with the corresponding value matrix to obtain the state features of each dimension of the data. The decision result is obtained based on the state characteristics of each dimension of data, the multi-early exit structure data, and the exit reasoning results.

[0010] Optionally, the preset deep reinforcement learning agent training process includes: A training environment is constructed to simulate the vehicle vibration energy acquisition scenario and the inference scenario of the embedded neural network, generating simulated data. The simulated data includes simulated energy acquisition data, simulated energy storage data, simulated energy consumption data, and the inference results of each exit of the simulated neural network and the structure data of multiple early exits corresponding to the embedded neural network. Based on the Markov decision process model, the corresponding state space, action space, and reward function are defined in the training environment. Obtain historical data; The network parameters of the original deep reinforcement learning agent are initialized. Based on the DDQN algorithm, the original deep reinforcement learning agent is iteratively trained according to the simulated data and the historical data until the decision performance of the original deep reinforcement learning agent meets the preset convergence condition, thus obtaining the deep reinforcement learning agent.

[0011] Optionally, the reward rules include: Energy state data is obtained based on the energy acquisition data and the energy storage data; The energy state data is compared with a preset threshold to obtain a comparison result, and the corresponding reward function is determined based on the comparison result.

[0012] Optionally, the preset threshold includes a first threshold and a second threshold; determining the corresponding reward function based on the comparison result includes: When the energy state data is less than the first threshold, the reward function is the inference result of the corresponding exit of the embedded neural network; When the energy state data is greater than the second threshold, the reward function is obtained based on the inference result of the corresponding exit of the embedded neural network and the corresponding overcharge energy penalty term; When the energy state data is greater than or equal to the first threshold and less than or equal to the second threshold, the reward function is obtained based on the inference result and penalty coefficient of the corresponding exit of the embedded neural network.

[0013] Secondly, the present invention provides an embedded neural network inference device for vehicle vibration energy, comprising: The acquisition unit is used to acquire vehicle energy harvesting data, energy storage data, energy consumption data, and multi-early exit structure data and inference results of each exit of the embedded neural network during the active energy harvesting process of vehicle vibration. The integration unit is used to integrate the energy acquisition data, energy storage data and energy consumption data based on a preset Markov decision process model to obtain the current energy state; The processing unit is configured to make decisions based on the reward rules of the Markov decision process model, using a preset deep reinforcement learning agent, on the current energy state, the multiple early exit structure data, and the inference results of each exit, to obtain decision results. The decision results include the early exit selection of the embedded neural network and the speed parameters of the vehicle's onboard vibration energy acquisition device.

[0014] Thirdly, the present invention provides an embedded neural network inference device for vehicle vibration energy, including a memory and a processor; the memory is used to store a computer program; the processor is used to implement the embedded neural network inference method for vehicle vibration energy as described in the first aspect when the computer program is executed.

[0015] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the embedded neural network inference method for vehicle vibration energy as described in the first aspect.

[0016] The beneficial effects of the embedded neural network inference method, device, and equipment for vehicle vibration energy of the present invention are: By acquiring energy acquisition, storage, and consumption data in real time during the active energy harvesting process of vehicle vibration, along with the multi-early exit structure and inference accuracy data of each exit of the embedded neural network, and integrating this data with a pre-defined Markov decision process model to form the current energy state, a deep reinforcement learning agent makes decisions based on this state and the neural network data, outputting the early exit selection and the speed parameters of the vehicle vibration energy harvesting device. This approach offers significant technical advantages: it can dynamically select the neural network's early exit exit, prioritizing inference accuracy when energy is sufficient and reducing energy consumption when energy is scarce, achieving an intelligent balance between accuracy and energy consumption; it can also actively adapt to the dynamic changes in vehicle vibration energy by adjusting the speed parameters of the harvesting device, improving vibration energy utilization efficiency and reducing energy waste and overcharging risks; simultaneously, leveraging the Markov decision process model's precise characterization of the energy state and the real-time decision-making capabilities of the reinforcement learning agent, it can flexibly respond to the complex fluctuations in energy supply in the vehicle environment and the accuracy requirements of different inference tasks. While ensuring real-time inference, it comprehensively enhances the energy utilization efficiency, environmental adaptability, and operational robustness of the vehicle-mounted embedded system, providing a complete solution for efficient neural network inference based on vibration energy in vehicle scenarios. Attached Figure Description

[0017] Figure 1This is a flowchart illustrating an embedded neural network inference method for vehicle vibration energy according to an embodiment of the present invention. Figure 2 This is an example diagram of the deep reinforcement learning agent architecture according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the Dual Deep Q Network (DDQN) training algorithm according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the relevant hyperparameter settings in an embodiment of the present invention; Figure 5 This is a schematic diagram of the vehicle-mounted vibration energy neural network inference system according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the toolchain design according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the vibration energy harvester and vibration energy principle according to an embodiment of the present invention; Figure 8 This is a schematic diagram of a benchmark neural network used for testing in an embodiment of the present invention; Figure 9 This is a schematic diagram of the energy harvesting environment used in an embodiment of the present invention. Figure 10 The results are verification results for different inference schemes in the embodiments of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0019] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0020] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to"; the term "based on" means "at least partially based on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; and the term "optionally" means "optional embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first," "second," etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0021] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0022] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0023] like Figure 1 As shown in the figure, an embedded neural network inference method for vehicle vibration energy provided by an embodiment of the present invention includes: Step S100: During the active energy harvesting process of vehicle vibration, acquire the vehicle's energy harvesting data, energy storage data, energy consumption data, as well as the multi-early exit structure data of the embedded neural network and the inference results of each exit.

[0024] Specifically, during the active energy harvesting of vehicle-mounted vibrations, two types of key data are acquired simultaneously: Energy-related data: including real-time and historical energy acquisition data from the vehicle vibration energy acquisition module (reflecting dynamic energy input), energy storage data from the energy storage unit (reflecting current energy reserves), and energy consumption data generated by the embedded neural network (reflecting energy output). Neural network related data includes structural data of multiple early exits of embedded neural networks (such as the number of exits, the corresponding network layer depth, etc., reflecting the hardware foundation of different inference paths) and inference results of each exit (such as inference accuracy, time consumption, etc., reflecting the performance characteristics of different paths).

[0025] Active energy harvesting differs from traditional passive energy harvesting in that it focuses on actively monitoring and responding to changes in environmental energy. By using intelligent control algorithms, active energy harvesting systems can activate energy harvesting devices at appropriate times to maximize energy extraction efficiency. For example, in vibration energy harvesting scenarios, the system can optimize the device's operating frequency to adapt to different vibration modes, thereby achieving more efficient energy harvesting.

[0026] Embedded neural networks typically consist of connection layers, each containing multiple nodes or neurons, with each node performing a simple computational task. Information is transferred between layers, and the nonlinear expressive power of the model is increased through transformations of activation functions. After training, neural networks can approximate complex functions and excel at tasks such as classification, regression, and image recognition.

[0027] For early-exit mechanisms in embedded neural networks: Early-exit mechanisms allow the inference process to terminate prematurely at different intermediate layers of the neural network. Typically, neural networks require complete inference through all layers; early-exit strategies output results at a specific intermediate layer, which helps reduce computational demands and power consumption, making them suitable for energy-constrained embedded devices. For each exit, there is a metric called average accuracy, which measures the overall inference performance of the samples exiting from that exit. The later the exit (the longer the inference path), the higher the average accuracy.

[0028] For classification tasks, network inference Accuracy per sample as follows: ; Among them, the network has a total of One export (p early exits). From the first The number of samples for each exit. From the first The average accuracy of each exit.

[0029] By constructing an information connection between energy supply and neural network inference through multi-dimensional data acquisition, the acquired data provides initial support for subsequent dynamic adjustment of inference strategies based on energy state. That is, synchronous acquisition of energy and neural network data can avoid decision-making biases caused by missing information (such as focusing only on energy state while ignoring inference accuracy requirements, or considering only inference performance while ignoring energy constraints), ensuring that subsequent decisions take into account both energy supply and demand and inference performance. Real-time data acquisition during active energy harvesting can promptly capture random fluctuations in vibration energy in the vehicle environment (such as sudden changes in the amount of data collected due to road bumps) and the dynamic needs of neural network inference (such as the different accuracy requirements of different tasks), laying the foundation for real-time adjustment of inference paths and acquisition strategies. At the same time, clearly distinguishing the categories and connotations of energy data and neural network data allows the subsequent Markov decision process model to directly integrate energy state based on structured data, reducing data preprocessing complexity and improving the overall response efficiency of the process.

[0030] Step S200: Based on a preset Markov decision process model, the energy acquisition data, energy storage data, and energy consumption data are integrated to obtain the current energy state.

[0031] Specifically, using a pre-defined Markov Decision Process (MDP) model, the energy acquisition data (including real-time and cumulative acquisition amounts), energy storage data (such as current remaining energy storage), and energy consumption data (including real-time and cumulative consumption) obtained in step S100 are integrated and processed to ultimately form a current energy state that reflects the system's energy dynamics. In particular, the MDP model, through pre-defined state definition rules, associates scattered energy data into structured state vectors (such as those containing historical and real-time energy changes, energy storage levels, etc.), transforming the originally isolated energy information into a panoramic description of the system's energy state that can be understood by a deep reinforcement learning agent.

[0032] By integrating energy data through the MDP model, fragmented information on energy collection, storage, and consumption is transformed into a structured current energy state. This effectively solves the problem of insufficient decision-making basis caused by the fragmentation of raw energy data. On the one hand, the integrated state can comprehensively depict the real-time dynamics and historical trends of energy supply and demand, providing complete energy environment information for subsequent agent decisions and avoiding short-sighted decisions due to data bias. On the other hand, the structured state vector meets the input requirements of the MDP model and can be directly used as the decision-making basis for subsequent agents, reducing redundant processing of data format conversion, improving decision response speed, and ensuring that the characterization of energy state forms a coherent technical chain with subsequent reward rules and decision logic, laying the foundation for accurately balancing energy utilization and inference performance.

[0033] Step S300: Based on the reward rule of the Markov decision process model, a deep reinforcement learning agent makes a decision on the current energy state, the multi-early exit structure data, and the inference results of each exit based on a preset deep reinforcement learning agent, and obtains a decision result. The decision result includes the early exit selection of the embedded neural network and the speed parameters of the vehicle's on-board vibration energy acquisition device.

[0034] Specifically, relying on the reward rules preset by the Markov decision process model, the deep reinforcement learning agent is used as the decision-making body. The current energy state obtained in step S200 and the embedded neural network multi-early exit structure data (such as the number of exits and the corresponding network layer depth) and the inference results of each exit (such as inference accuracy and time consumption) obtained in step S100 are input. Through the agent's collaborative analysis of the dynamics of energy supply and demand and the requirements of inference performance, two types of decision results are finally output: one is the selection of early exits of the embedded neural network (determining the appropriate inference path under the current energy conditions), and the other is the speed parameters of the vehicle's on-board vibration energy acquisition device (adjusting the energy acquisition efficiency to match the current inference energy consumption requirements).

[0035] By combining energy state and neural network data in decision-making with a deep reinforcement learning agent, precise synergistic optimization of energy utilization and inference performance is achieved. On the one hand, the decision logic based on reward rules guides the agent to prioritize early exit points where energy consumption matches current energy storage and inference accuracy meets task requirements. This avoids task interruption due to high-energy-consumption inference when energy is insufficient, or performance waste due to low-precision inference when energy is sufficient. On the other hand, the speed parameters of the synchronous decision-making acquisition device can dynamically adapt to changes in inference energy consumption. When energy is scarce, the acquisition speed is increased to supplement energy storage, and when energy is abundant, the acquisition speed is reduced to reduce ineffective consumption. At the same time, the linkage output of the two types of decision results ensures real-time matching between energy acquisition and inference processes, effectively improving the operational stability and resource utilization efficiency of the vehicle-mounted embedded system under random vibration energy fluctuation scenarios.

[0036] This embodiment acquires energy acquisition, storage, and consumption data in real time during the active energy harvesting process of vehicle vibration, along with the multi-early exit structure and inference accuracy data of each exit point of the embedded neural network. It integrates these data using a pre-defined Markov decision process model to form the current energy state, and then uses a deep reinforcement learning agent to make decisions based on this state and the neural network data, outputting the early exit point selection and the speed parameters of the vehicle vibration energy harvesting device. This approach offers significant technical advantages: it can dynamically select the neural network's early exit point, prioritizing inference accuracy when energy is sufficient and reducing energy consumption when energy is scarce, achieving an intelligent balance between accuracy and energy consumption; it can also actively adapt to the dynamic changes in vehicle vibration energy by adjusting the speed parameters of the harvesting device, improving vibration energy utilization efficiency and reducing energy waste and overcharging risks; simultaneously, leveraging the Markov decision process model's precise characterization of the energy state and the real-time decision-making capabilities of the reinforcement learning agent, it can flexibly respond to the complex fluctuations in energy supply in the vehicle environment and the accuracy requirements of different inference tasks. While ensuring real-time inference, it comprehensively enhances the energy utilization efficiency, environmental adaptability, and operational robustness of the vehicle-mounted embedded system, providing a complete solution for efficient neural network inference based on vibration energy in vehicle scenarios.

[0037] Optionally, the energy acquisition data includes real-time acquisition data and historical cumulative acquisition data, the energy storage data includes real-time storage data, and the energy consumption data includes real-time consumption data and historical cumulative consumption data; the integration of the energy acquisition data, energy storage data, and energy consumption data based on a preset Markov decision process model to obtain the current energy state includes: Get the data of the last decision action; The real-time collected data, the historical cumulative collected data, the real-time stored data, the real-time consumption data, the historical cumulative consumption data, and the data of the previous decision action are integrated and processed in a preset dimension order to obtain the current energy state.

[0038] Optionally, the current energy state includes: ; in, The current energy state, For the real-time stored data, The historical cumulative data, For the real-time collected data, The historical cumulative consumption data, The real-time consumption data, This refers to the data from the previous decision-making action.

[0039] Optionally, the reward rules include: Energy state data is obtained based on the energy acquisition data and the energy storage data; The energy state data is compared with a preset threshold to obtain a comparison result, and the corresponding reward function is determined based on the comparison result.

[0040] Optionally, the preset threshold includes a first threshold and a second threshold; determining the corresponding reward function based on the comparison result includes: When the energy state data is less than the first threshold, the reward function is the inference result of the corresponding exit of the embedded neural network; When the energy state data is greater than the second threshold, the reward function is obtained based on the inference result of the corresponding exit of the embedded neural network and the corresponding overcharge energy penalty term; When the energy state data is greater than or equal to the first threshold and less than or equal to the second threshold, the reward function is obtained based on the inference result and penalty coefficient of the corresponding exit of the embedded neural network.

[0041] In some embodiments, to address the challenge of balancing the randomness of vibration energy supply with the accuracy of neural network inference and energy consumption in automotive embedded scenarios, a three-level collaborative system of data perception, state integration, and intelligent decision-making is constructed. The core relies on a Markov Decision Process (MDP) model and a deep reinforcement learning agent to achieve dynamic optimization. The specific structure and logic are as follows: Firstly, through the vehicle-mounted sensing and monitoring module, two types of core data are acquired in real time during the active vibration energy acquisition process: Energy flow data: This includes the real-time output power and historical cumulative collection volume of vehicle-mounted vibration energy acquisition equipment (energy acquisition data), the real-time remaining power of energy storage units such as supercapacitors (energy storage data), and the real-time energy consumption and historical cumulative energy consumption during the operation of embedded neural networks (energy consumption data), comprehensively depicting the dynamics of the entire energy acquisition-storage-consumption chain.

[0042] Neural network data includes the structure data of multiple early exits of embedded neural networks (such as the number of exits, the network layer depth corresponding to each exit, the neuron connection topology, etc.), as well as the inference results of each early exit for the input sample, such as inference accuracy and computation time, to clarify the performance and energy consumption characteristics of different inference paths.

[0043] The data acquisition process employs a millisecond-level synchronization mechanism to ensure the temporal consistency between energy state and inference performance data, providing an accurate input basis for subsequent decision-making.

[0044] Within a defined time period of duration T, let the total number of samples that the early-retreat neural network Net (embedded neural network) with p exit points needs to process be N. The goal is to determine the exit point and energy harvesting level for each sample based on the energy state of the vehicle vibration energy neural network inference system, thereby maximizing the total inference accuracy over N runs. In this decision problem, the current energy state of the system is determined by both the previous energy state and the current decision. Based on this, the problem can be modeled as a standard Markov decision process. And design an agent learning strategy π: S→A to maximize reward R: ; in, For mathematical expectation, The first action performed by the agent (deep reinforcement learning agent) Sub-sample reasoning It is a discount factor that describes the importance of future rewards. The four key elements, namely The definition is as follows: state : Represents the set of states of the system, observed by the agent, where It is the first The current energy state before subsample inference.

[0045] action : Represents the set of actions determined by the agent, where It is aimed at the first The actions that each sample needs to take are as follows: ; in, This refers to the exit point of the neural network (the early exit point selection for embedded neural networks), and there are a total of [number missing]. Possible values ,in This indicates an action that keeps the system idle; the idle interval is the earliest delay before exiting inference, ranging from 1 to... This indicates the action the agent decides to take. One possible exit point. These are the speed parameters used to adjust the energy harvesting controller in the active energy harvesting process. Possible values .

[0046] probability : Indicates from state arrive Transition probability ), in a defined action Down The probability is expressed as In a defined Markov decision process, there is a transition probability between any two valid states and one valid action.

[0047] award : A necessary element to guide intelligent agents to achieve problem objectives. It is in the The reward obtained after each inference sample is designed according to the following rules: ; In some embodiments, It is calculated based on four scenarios: : in the After the initial energy harvesting, the system energy level (energy state data) is within the normal operating range of the storage medium. The first threshold, (The second threshold). The reward is the reward for performing embedded neural network inference operations. And calculate the average accuracy of the embedded neural network inference (inference accuracy data for each output): ; in, It is the first The average precision of each sample is obtained based on the exit point. The value of is in between.

[0048] That is, when performing neural network inference (i.e., when there is an inference task), Equal to the average accuracy of embedded neural network inference; When the system is in an idle state (no inference tasks are being executed), In simple terms, it determines the reward value by quantifying the accuracy of the neural network's inference process; the higher the inference accuracy, the higher the reward. The larger the value, the more motivated the system is to perform high-precision neural network inference when the energy state is appropriate.

[0049] After energy harvesting, if the energy level falls below the minimum threshold (first threshold) required for the system to remain idle, the reward becomes a penalty. The calculation is as follows: ; in, The calculation is based on the formula in the first case above. It is a scaling factor used to stabilize the learning process of an agent, and its specific value depends on the specific training data.

[0050] Following energy harvesting, the energy level exceeded the storage medium's maximum permissible capacity (second threshold). This indicates overcharging occurred after the latest decision. Therefore, in Based on this, penalty items are given . The excess energy portion is calculated as follows: ; One sample was not completed: after a certain period of time. After that, the number of inference samples is less than This means the system failed to complete the overall reasoning. The given penalty term... for: ; in, This refers to the actual number of samples completed. It is a scaling factor used to stabilize the learning process of an agent, and its specific value depends on the specific training data.

[0051] The main purpose of the MDP model is to link and integrate data and to characterize the dynamic dependencies of energy. This refers to the connection between energy harvesting and storage: real-time data collection With real-time data storage Correlation reflects the supplementary role of harvested energy in energy storage (e.g., The larger, The more significant the improvement, the more it should be avoided to exceed [a certain limit]. (Preset threshold) leads to overcharging); The relationship between energy consumption and storage: real-time consumption data With real-time data storage Correlation reflects the effect of inference energy consumption on energy storage consumption (e.g.) The larger, The more significant the decline, the more important it is to avoid falling below [a certain level]. (This causes the system to be idle). Correlation between cumulative data and real-time data: This involves linking cumulatively collected data... Cumulative consumption Separately with real-time acquisition Real-time consumption Correlation, reflecting long-term energy supply and demand trends (such as...) This indicates that the energy supply is sufficient, and high-precision early exit ports can be prioritized. Correlation between historical actions and energy data: the previous action It is correlated with three types of energy data to reflect the impact of historical decisions on the current energy state (such as the previous selection of a high acquisition level). Then the current (Higher probability)

[0052] By integrating real-time and historical energy data, as well as data from the previous decision-making action, the current energy state can comprehensively and dynamically present the energy status of the vehicle-mounted vibration energy neural network inference system, providing sufficient and accurate basis for decision-making by the deep reinforcement learning agent. Based on the energy state and preset thresholds, the differentiated reward rules can flexibly adapt to the decision-making needs under different energy scenarios. When energy is insufficient, the value of the inference results is guaranteed; when energy is overcharged, invalid energy collection is suppressed; and when energy is moderate, accuracy and energy consumption are balanced. The three work together to enable the vehicle-mounted embedded system to efficiently balance energy utilization and inference performance in an environment with randomly changing vibration energy, while improving the accuracy and adaptability of decision-making, ensuring the stable and efficient operation of the system.

[0053] Optionally, the deep reinforcement learning agent includes a self-attention mechanism; the deep reinforcement learning agent based on a preset decision makes decisions based on the current energy state, the multiple early exit structure data, and the inference results of each exit, obtaining decision results including: The current energy state is processed according to the self-attention mechanism to obtain the state features of each dimension of the current energy state, including: decomposing each dimension of the current energy state to obtain the corresponding query matrix, key matrix and value matrix; obtaining the attention weight matrix of each dimension of the data according to each query matrix and the corresponding key matrix; and fusing each attention weight matrix with the corresponding value matrix to obtain the state features of each dimension of the data. The decision result is obtained based on the state characteristics of each dimension of data, the multi-early exit structure data, and the exit reasoning results.

[0054] Specifically, firstly, regarding the current energy state vector (e.g. The six dimensions of data (previous action, real-time stored energy, historical cumulative collected energy, real-time collected energy, historical cumulative consumed energy, and real-time consumed energy) are mapped through the linear transformation layer of the agent's neural network to generate a query matrix Q, a key matrix K, and a value matrix V for each dimension.

[0055] Where Q represents the query demand for the current dimension of data, K represents the matching information that the dimension of data can provide, and V represents the core feature value of the dimension of data. All three dimensions are uniformly represented as (number of dimensions × number of neurons), such as... Figure 2 The diagram shown is an example of a deep reinforcement learning agent architecture. Figure 2This paper demonstrates a deep reinforcement learning agent architecture for an onboard vibration energy neural network inference system in active energy harvesting. The system harvests energy from vibration sources, which is stored in a supercapacitor to power an early-exit neural network on a processor. The deep reinforcement learning agent employs a self-attention mechanism to capture the interdependencies within runtime states s∈S. After processing by the self-attention module, the results are combined with an environmental pre-configuration representing early environmental features. (Including data on the early exit structure of embedded neural networks, etc.), which is then processed to ultimately make decisions that affect the energy harvesting process v and the neural network inference x.

[0056] After acquiring the state features of data from various dimensions, the deep reinforcement learning agent combines the multi-early exit structure data (such as the network layer depth and neuron connectivity corresponding to each early exit) and the inference results of each exit (such as the inference accuracy and time consumption of different exits) to make a comprehensive decision: the agent processes the state features, multi-early exit structure data, and inference results of each exit through its internal decision network (such as a structure containing fully connected layers), evaluates the value of different actions (such as choosing which early exit, adjusting the speed of the vibration energy acquisition device, etc.), and finally selects the optimal decision result that can balance energy utilization and inference accuracy under the current energy state, that is, determines the early exit selection of the embedded neural network and the speed parameters of the vehicle-mounted vibration energy acquisition device, etc.

[0057] In this embodiment Figure 2 The linear layer of the agent contains 16 neurons, so the matrix dimension is (6×16).

[0058] Based on the constructed Q and K, multiplication is performed using matrix transpose. Calculate the correlation similarity between data in each dimension, and then divide by the square root of the number of neurons (e.g., The scaled values ​​are then applied to avoid excessively large values ​​that could lead to vanishing or exploding gradients. The scaled result is then transformed into a probability distribution-based attention weight matrix using the Softmax function. Each element in this matrix represents the correlation importance between two corresponding dimensions of data. For example, higher weights for the dimensions of real-time energy storage and real-time energy consumption indicate a stronger synergistic influence between the two dimensions on the decision-making process.

[0059] The attention weight matrix is ​​multiplied by the corresponding value matrix V, which weights and fuses the core features (V) of each dimension of data according to the importance of the relationships between dimensions. The final output is the attention-optimized state feature of each dimension. These features retain the original information of each dimension and incorporate the interaction relationships between dimensions, forming a high-dimensional state representation with greater decision-making value.

[0060] In other embodiments, the architecture of the deep reinforcement learning agent consists of fully connected layers. Encoded states, including energy storage, collection, and consumption, are fed as input into the deep reinforcement learning agent (i.e., Figure 2 The runtime state, i.e., the current energy state. After input, a self-attention module (self-attention mechanism) is introduced to capture the relationships between different elements in the state. This allows the deep reinforcement learning agent to identify various internal dependencies in the state, such as the relationship between energy state and energy collected, energy state and energy consumed, energy collected and energy consumed, and the relationship between the latest energy and accumulated energy. The self-attention mechanism for handling runtime states is as follows: ; ; ; Where Q is the query matrix, K is the key matrix, and V is the value matrix. To input The linear transformation yields the weight matrix of the query matrix Q. To input The linear transformation yields the weight matrix of the bond matrix K. To input The linear transformation is the weight matrix of the value matrix V.

[0061] The agent's input state has a six-dimensional dimension, and the agent's neural network architecture consists of three linear layers, each with sixteen neurons, used to capture a high-dimensional representation of each state attribute. Once derived... , and The agent then calculates the attention weight matrix and the final attention output: ; ; in, The state features of the corresponding dimension data (the final attention output, i.e., the weighted attention information under each state attribute (each dimension data)). For the attention weight matrix of the corresponding dimension data (the attention weight matrix of each state attribute relative to other state attributes), the matrix product is... quilt Scaling is typically done using the square root of the number of neurons as a scaling factor to prevent maxima that could lead to vanishing or exploding gradients. These values ​​are then transformed into a probability distribution using the SoftMax function, and... Combined to generate output. This self-attention module can be applied to any neural network-based agent to generate relationships between each state component.

[0062] The self-attention mechanism effectively addresses the decision-making bias caused by the traditional deep reinforcement learning agent's equal treatment of energy state dimension information. On one hand, by decomposing and generating Q, K, and V matrices, it achieves refined representation of energy state data across dimensions, preventing the obscuring of single-dimensional information. On the other hand, the attention weight matrix dynamically captures the correlation strength between dimensions, such as the dependence between energy storage and consumption, and between historical cumulative data and real-time data, enabling the agent to prioritize the dimension features more critical to decision-making (such as the correlation between real-time energy storage and real-time energy consumption when energy is insufficient). Simultaneously, the state features generated by weighted fusion can be directly used as input to the agent's subsequent fully connected layers, reducing interference from invalid information, improving decision-making efficiency and accuracy, and ultimately ensuring that, in scenarios with random fluctuations in vehicle vibration energy, the agent's decisions on early exit point selection and speed adjustment of acquisition equipment better align with the coordinated needs of energy supply and demand and inference accuracy.

[0063] Optionally, the preset deep reinforcement learning agent training process includes: A training environment is constructed to simulate the vehicle vibration energy acquisition scenario and the inference scenario of the embedded neural network, generating simulated data. The simulated data includes simulated energy acquisition data, simulated energy storage data, simulated energy consumption data, and the inference results of each exit of the simulated neural network and the structure data of multiple early exits corresponding to the embedded neural network. Based on the Markov decision process model, the corresponding state space, action space, and reward function are defined in the training environment. Obtain historical data; The network parameters of the original deep reinforcement learning agent are initialized. Based on the DDQN algorithm, the original deep reinforcement learning agent is iteratively trained according to the simulated data and the historical data until the decision performance of the original deep reinforcement learning agent meets the preset convergence condition, thus obtaining the deep reinforcement learning agent.

[0064] In some embodiments, such as Figure 3 The diagram shows a schematic of the Dual Deep Q-Network (DDQN) training algorithm. Agent training based on DDQN revolves around pre-configuration sampling, environment initialization, action selection, experience storage and updating, and network parameter iteration. Each step is closely linked to ensure training stability and decision accuracy. The specific process is as follows: Training Core Foundation: DDQN Dual-Network Architecture Setup. In the initial training phase, two Q-networks with identical structures but different update frequencies are initialized: the main Q-network (…). ) and target Q network ( ): The main Q-network is responsible for calculating the Q-value of the current state-action pair in real time (evaluating the value of the action) and dynamically updating the parameters based on the training data through gradient descent. The target Q-network is responsible for calculating the target Q-value (as a reference benchmark for training). Its parameters are not updated synchronously with the main Q-network, but are copied from the main Q-network at a preset frequency (such as once every C training rounds). This avoids the problem of excessive fluctuation in Q-value estimation in traditional DQN and improves training stability and Q-value estimation accuracy.

[0065] Training process: the entire process from pre-configuration to network iteration.

[0066] Pre-configuration sampling and environment initialization (corresponding) Figure 3 (Lines 3-5): First, randomly select a set of pre-configurations from the preset pre-configuration set in a uniform distribution.

[0067] The pre-configuration includes two key types of information: (1) the type of neural network architecture (such as a three-outlet early termination neural network for datasets such as GSC and CIFAR-10); and (2) the system energy parameters (including the maximum capacity of the energy storage medium and the initial energy level).

[0068] Subsequently, based on the extracted pre-configuration, the corresponding parameters are initialized in the training environment (such as an on-board vibration energy acquisition simulator or physical platform), including loading the early termination neural network of the specified architecture, setting the energy capacity and initial charge of the supercapacitor, to ensure that the energy characteristics and network inference characteristics of the training environment are consistent with those of the actual application scenario.

[0069] During training, for each inference sample's current state (runtime energy state + pre-configured vector), an ε-greedy strategy with linear ε decay is used to select actions: when the randomly generated probability is less than ε, the agent randomly selects an action (exploring the unknown action space to avoid getting trapped in local optima); when the probability is greater than or equal to ε, the agent selects the action with the largest Q value based on the Q value output by the main Q network (utilizing known optimal strategies to improve training efficiency). The initial value of ε is set to 0.95, and it decays linearly by 0.01 with each training round, achieving a balance between "more exploration in the early stages and more utilization in the later stages."

[0070] When an agent observes the current runtime state (such as a vector composed of energy storage, collection, and consumption data), After that, the state vector is concatenated with the extracted pre-configured vector and input into the self-attention module. The self-attention mechanism captures the dependencies between various dimensions of the energy state (such as real-time energy acquisition and real-time energy consumption), the state and the pre-configuration, and outputs a feature vector that integrates the associated information. Then, the feature vector is sent to a fully connected layer with 24 neurons, and the main Q network calculates and outputs the Q value corresponding to each possible action (early exit selection x, acquisition device speed v), providing a basis for action selection.

[0071] Experience storage and batch update (corresponding) Figure 3 Lines 8-18 of the algorithm): Experience storage: After each action is executed, the current state is stored. The selected action Rewards received Next state The single-step experience, composed of samples, is stored in the experience replay pool D, breaking the temporal correlation between samples and improving data utilization efficiency.

[0072] Batch Update: Once the storage capacity of the experience replay pool D reaches a preset threshold, a mini-batch of samples of size Z (e.g., Z=64) is randomly drawn from the pool each time. Calculate the current state using the main Q network Calculate the Q value of each action and select the action with the highest Q value. ; Calculate the next state using the target Q-network The target Q value of the next action a ,in,( The discount factor is set to 0.90 to balance current and future rewards. The Q value output by the main Q network ( ) and target Q value The mean squared error is used as the loss function, and the parameters of the main Q network are updated through the Adam optimizer (learning rate 0.001). ; When the training epochs satisfy ε mod C = 0 (C is the target network update frequency, e.g., C = 100), the current parameters of the main Q-network are... Copy to the target Q network and update. .

[0073] The training parameters are fixed. All key hyperparameters during training (such as the empirical replay pool capacity, mini-batch size, target network update frequency, etc.) can be configured as follows: Figure 4 Configure the settings as shown, for example, set the experience replay pool D capacity to 2000, mini-batch size Z=64, and target network update frequency C=100 to ensure the stability and reproducibility of the training process.

[0074] In some embodiments, such as Figure 5 The diagram shows a vehicle-mounted vibration energy neural network inference system, which includes an energy harvester, a processor, and a storage medium for collecting real-world training data (historical data) for a deep reinforcement learning agent. A supporting toolchain is also introduced, such as... Figure 6 A schematic diagram of the toolchain design, which uses an onboard simulator to provide a deep reinforcement learning agent with sufficient and diverse energy harvesting training data (simulated data).

[0075] from Figure 6 As can be seen, the toolchain is divided into four core parts: road data module, Unity simulation core module, energy parameter configuration module, and trajectory output and application module. Each module is connected in sequence through a data transmission link: the road data module provides the terrain foundation for simulation, the Unity simulation core module realizes the linkage between virtual scene and energy collection, the energy parameter configuration module ensures the consistency between simulation and reality, and the trajectory output and application module transforms the simulation results into training data for intelligent agents, forming a closed loop of data-simulation-application.

[0076] Road Data Module: Generation and Import of Basic Terrain Data (corresponding) Figure 6 (Left-side road height data section) This module is the input source of the toolchain, responsible for providing basic data for virtual terrain generation. The specific process involves two implementation paths: Random Path Generation: The algorithm uses a preset sampling algorithm to automatically generate a road height data matrix (unit: cm) that conforms to statistical patterns, based on preset parameters for road surface roughness (e.g., high bumps, medium flatness, low undulation) and road surface type (e.g., gravel road, cement road, dirt road). For example, when generating road data for high-energy acquisition scenarios, the algorithm increases the proportion of sampling points with height differences greater than 5cm to simulate the characteristics of a real bumpy road surface; when generating data for low-energy acquisition scenarios, it controls the height difference to within 1cm to simulate a flat road surface.

[0077] Existing data import path: Supports importing height recording data of real roads (such as historical road surface data collected by vehicle-mounted LiDAR and inertial measurement units). The data format needs to be converted to CSV format compatible with Unity simulator to ensure that there is a height value corresponding to every 10cm sampling interval, so as to achieve a 1:1 restoration of the real road surface scene.

[0078] The road height data generated by both paths are transmitted to the Unity simulation core module through a data interface as the underlying basis for terrain generation.

[0079] Unity Simulation Core Module: Virtual Terrain Construction and Energy Harvesting Simulation (corresponding to) Figure 6(Core components of the Unity simulator) This module is the core execution unit of the toolchain, relying on the Unity 3D engine to achieve the linkage of terrain generation, virtual vehicle operation, and vibration energy acquisition and calculation, specifically in three steps: Virtual Terrain Construction: After receiving road height data, the Terrain component of the Unity engine automatically converts the data into a 3D terrain model. Using the X-axis (road length) and Z-axis (road width) of the road height data as planar coordinates and the Y-axis as the height coordinate, it generates virtual terrain that matches the scale of the real road surface (such as a virtual road that is 100m long and 5m wide). It also matches the corresponding texture material according to the road surface type (such as loading gravel texture on gravel roads and smooth texture on cement roads) to enhance the realism of the scene.

[0080] Virtual vehicle parameter configuration and operation control: Deploy a 1:1 scale 3D model of a small car (consistent with the vehicle used in the physical experimental platform) in the virtual terrain, configure the vehicle suspension system parameters (such as shock absorber stiffness and spring damping coefficient) and active energy harvesting control logic: According to the active control requirements of deep reinforcement learning, the driving speed of the virtual vehicle (such as 20km / h, 30km / h, 40km / h) can be adjusted through scripts, thereby changing the intensity of vibration interaction between the vehicle and the road surface, simulating vibration scenarios under different active harvesting strategies.

[0081] Vibration energy acquisition and calculation: A preset vibration energy acquisition model is used, and a voltage is generated between magnets through electromagnetic induction by vibrating coils, such as... Figure 7 The diagram shows a vibration energy harvester and the principle of vibration energy. A vibration-to-electrical-energy conversion calculation script is bound to the shock absorber location of the virtual vehicle. This script contains an energy harvesting formula consistent with the physical acquisition platform (i.e.,...). ,in It is the vibration speed of the coil. and These are the load and coil resistance, respectively. These are parameters related to the magnetic flux of the coil and the magnet. ,in, It is the average magnetic flux density. , , These are the number of turns, effective length, and fill factor of the coil, respectively. (This vibration energy harvester is mounted on a virtual vehicle.)

[0082] As the virtual vehicle travels on the terrain, the script calculates the vibration amplitude of the vehicle's suspension system in real time and converts it into coil vibration velocity. (Unit: m / s), combined with the preset coil resistance (10Ω), load resistance (20Ω), magnetic flux parameter K (0.5 Vs / m), real-time output of voltage and power data of virtual vibration energy acquisition device, forming a continuous energy acquisition power trajectory (unit: mW, sampling frequency: 10Hz).

[0083] Energy Consumption: To execute the early exit neural network, a 16MHz TI MSP430FR5994 processor with 256KB FRAM and 8KB on-chip SRAM was used. The minimum operating voltage was set to 1.8V. Energy Trace was used to analyze the runtime energy consumption of the neural network at different exit points. Figure 8 The diagram shows a schematic of the benchmark neural network used for testing; it lists the tested benchmarks, including four pre-trained, three-exit neural networks for four different datasets (GSC, FSDD, CIFAR-10, SVHN), covering two task types: image classification and audio processing.

[0084] Energy Storage: A 220mF supercapacitor serves as the energy storage medium for the system, allowing for charging and discharging capabilities. The energy level within the supercapacitor can be obtained by measuring its voltage.

[0085] Intermediate connection section: The collected AC input first passes through an AC-DC module, and then is input to the TPS63020, which is a buck-boost converter. The converted DC input will either power the processor or be stored in a supercapacitor.

[0086] In some specific embodiments, the embedded neural network inference method for vehicle vibration energy is verified, specifically: Currently, there is still a lack of dedicated methods for active energy harvesting + neural network inference. Therefore, we compare the existing methods for embedded neural network inference of vehicle vibration energy, including REAP, GEM-RL and TRAIN: REAP (

[15] ), GEM-RL ([8]) and TRAIN ([2]) are all aimed at passive energy harvesting scenarios, and only achieve adaptive neural network inference through different optimization strategies (such as REAP selecting an effective exit point based on linear optimization, and GEM-RL using the TD3 algorithm to design an energy consumption decay type reward function).

[0087] To ensure fairness in the comparison, all methods were deployed on the unified vehicle-mounted vibration energy acquisition + embedded neural network inference system built in this application, and tested under the same conditions of energy acquisition hardware (vibration acquisition device, supercapacitor), neural network architecture (three-exit early termination model), and inference task (image / audio classification). Four different datasets (GSC, FSDD, CIFAR-10, SVHN) were used as the data basis.

[0088] Furthermore, considering the adjustability of deep reinforcement learning (DRL) algorithms, this embodiment also derives two comparative variants for the proposed method: one is a proxy version using Q-learning (QL, tabular reinforcement learning, without self-attention mechanism) and DQN (deep Q-network) to verify the advantages of the self-attention + dual deep Q-network (DDQN) architecture; the other is a method designed to "maximize energy harvesting" (MaxEH), which aims to maximize harvested energy (rather than maximizing inference accuracy, which is the core objective of this application), to highlight the impact of goal orientation on system performance. It should be specifically noted that all the above comparative methods (including derived variants) adopt the active energy harvesting mechanism defined in this application, ensuring that the superiority of the proposed method is only reflected through differences in optimization strategy, algorithm architecture, and goal orientation.

[0089] All of the above design series methods employ active energy harvesting processes. The following two evaluation metrics were used in the subsequent results: System idle time ( (in seconds): Records the idle time period of the system within a given period, during which the processor does not execute early exit from the neural network, but only performs energy harvesting. This metric represents the continuity of neural network inference and indicates the real-time execution capability of the system.

[0090] Accuracy ( , \%): Following the goal of the problem, this metric represents the accuracy of N inference samples.

[0091] The verification process involved collecting energy harvesting training data by driving a small car on three real-world roads, which is represented as follows: Simultaneously, we conducted simulated vehicle tests on five simulated terrains using our designed toolchain, denoted as follows: .like Figure 9 The energy harvesting environments for example tests are shown (Real worldroad examples, Simulated road examples); these different roads have different levels of energy harvesting, representing different external environments. Note that there are two relationships between the energy harvested and the energy consumed in the problem, where: (1) In In these scenarios, the energy harvested for each decision cannot be directly used for neural network execution. In these cases, the initial battery charge is set to 100%. These six environments are divided into three groups, further categorized by their energy harvesting levels as high-energy harvesting environments (…). Medium energy harvesting environment Low-energy harvesting environment (2) We also simulated In this scenario, the energy collected for each decision can be directly used for neural network execution, with the initial battery charge set to 0%.

[0092] like Figure 10 The results show the verification results of different inference schemes. Figure 9 As shown, our method (Ours) demonstrates results compared to other comparative methods across 32 settings (4 tasks × 8 energy harvesting environments). Our method exhibits the highest accuracy results in 31 of the 32 settings and the lowest system idle time across all settings (particularly with no system idle time in 29 of the 32 settings). Compared to passive energy harvesting methods (REAP, GEM-RL, and TRAIN), our method achieves an average improvement of 5.14% inference accuracy and a reduction of 99.42% in system idle time.

[0093] An embedded neural network inference device for vehicle vibration energy provided in this embodiment of the invention includes: The acquisition unit is used to acquire vehicle energy harvesting data, energy storage data, energy consumption data, and multi-early exit structure data and inference results of each exit of the embedded neural network during the active energy harvesting process of vehicle vibration. The integration unit is used to integrate the energy acquisition data, energy storage data and energy consumption data based on a preset Markov decision process model to obtain the current energy state; The processing unit is configured to make decisions based on the reward rules of the Markov decision process model and a preset deep reinforcement learning agent regarding the current energy state, the multi-early exit structure data, and the inference results of each exit, to obtain decision results. The decision results include the early exit selection of the embedded neural network and the speed parameters of the vehicle's onboard vibration energy acquisition device.

[0094] An embodiment of the present invention provides an embedded neural network inference device for vehicle vibration energy, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to implement the embedded neural network inference method for vehicle vibration energy as described above when the computer program is executed.

[0095] This invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the embedded neural network inference method for vehicle vibration energy as described above.

[0096] The following describes an embedded neural network inference device for vehicle vibration energy that can serve as a server or client of the present invention, which is an example of a hardware device applicable to various aspects of the present invention. The embedded neural network inference device for vehicle vibration energy is intended to represent various forms of digital stator computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The embedded neural network inference device for vehicle vibration energy can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0097] The embedded neural network inference device for vehicle vibration energy includes a computing unit that can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) or a computer program loaded from a storage unit into random access memory (RAM). The RAM can also store various programs and data required for device operation. The computing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0098] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. In this application, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention according to actual needs. Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units can be implemented in hardware or as software functional units.

[0099] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. An embedded neural network inference method for vehicle-mounted vibration energy, characterized in that, include: During the active energy harvesting process of vehicle vibration, energy harvesting data, energy storage data, energy consumption data of the vehicle, as well as multi-early exit structure data of the embedded neural network and inference results of each exit are acquired. Based on a pre-defined Markov decision process model, the energy acquisition data, energy storage data, and energy consumption data are integrated to obtain the current energy state. According to the reward rule of the Markov decision process model, a deep reinforcement learning agent makes a decision on the current energy state, the multiple early exit structure data, and the inference results of each exit based on a preset deep reinforcement learning agent, and obtains a decision result. The decision result includes the early exit selection of the embedded neural network and the speed parameters of the vehicle's on-board vibration energy acquisition device.

2. The embedded neural network inference method for vehicle vibration energy according to claim 1, characterized in that, The energy acquisition data includes real-time acquisition data and historical cumulative acquisition data; the energy storage data includes real-time storage data; and the energy consumption data includes real-time consumption data and historical cumulative consumption data. The current energy state is obtained by integrating the energy acquisition data, energy storage data, and energy consumption data based on a preset Markov decision process model, including: Get the data of the last decision action; The real-time collected data, the historical cumulative collected data, the real-time stored data, the real-time consumption data, the historical cumulative consumption data, and the data of the previous decision action are integrated and processed in a preset dimension order to obtain the current energy state.

3. The embedded neural network inference method for vehicle vibration energy according to claim 2, characterized in that, The current energy state includes: ； in, The current energy state, For the real-time stored data, The historical cumulative data, For the real-time collected data, The historical cumulative consumption data, The real-time consumption data, This refers to the data from the previous decision-making action.

4. The embedded neural network inference method for vehicle vibration energy according to claim 1, characterized in that, The deep reinforcement learning agent includes a self-attention mechanism; the deep reinforcement learning agent, based on a preset mechanism, makes decisions on the current energy state, the multiple early exit structure data, and the inference results of each exit, to obtain decision results, including: The current energy state is processed according to the self-attention mechanism to obtain the state features of each dimension of the current energy state, including: decomposing each dimension of the current energy state to obtain the corresponding query matrix, key matrix and value matrix; obtaining the attention weight matrix of each dimension of the data according to each query matrix and the corresponding key matrix; and fusing each attention weight matrix with the corresponding value matrix to obtain the state features of each dimension of the data. The decision result is obtained based on the state characteristics of each dimension of data, the multi-early exit structure data, and the exit reasoning results.

5. The embedded neural network inference method for vehicle vibration energy according to claim 1, characterized in that, The pre-defined deep reinforcement learning agent training process includes: A training environment is constructed to simulate the vehicle vibration energy acquisition scenario and the inference scenario of the embedded neural network, generating simulated data. The simulated data includes simulated energy acquisition data, simulated energy storage data, simulated energy consumption data, and the inference results of each exit of the simulated neural network and the structure data of multiple early exits corresponding to the embedded neural network. Based on the Markov decision process model, the corresponding state space, action space, and reward function are defined in the training environment. Obtain historical data; The network parameters of the original deep reinforcement learning agent are initialized. Based on the DDQN algorithm, the original deep reinforcement learning agent is iteratively trained according to the simulated data and the historical data until the decision performance of the original deep reinforcement learning agent meets the preset convergence condition, thus obtaining the deep reinforcement learning agent.

6. The embedded neural network inference method for vehicle vibration energy according to claim 1, characterized in that, The reward rules include: Energy state data is obtained based on the energy acquisition data and the energy storage data; The energy state data is compared with a preset threshold to obtain a comparison result, and the corresponding reward function is determined based on the comparison result.

7. The embedded neural network inference method for vehicle vibration energy according to claim 6, characterized in that, The preset threshold includes a first threshold and a second threshold; determining the corresponding reward function based on the comparison result includes: When the energy state data is less than the first threshold, the reward function is the inference result of the corresponding exit of the embedded neural network; When the energy state data is greater than the second threshold, the reward function is obtained based on the inference result of the corresponding exit of the embedded neural network and the corresponding overcharge energy penalty term; When the energy state data is greater than or equal to the first threshold and less than or equal to the second threshold, the reward function is obtained based on the inference result and penalty coefficient of the corresponding exit of the embedded neural network.

8. An embedded neural network inference device for vehicle-mounted vibration energy, characterized in that, include: The acquisition unit is used to acquire vehicle energy harvesting data, energy storage data, energy consumption data, and multi-early exit structure data and inference results of each exit of the embedded neural network during the active energy harvesting process of vehicle vibration. The integration unit is used to integrate the energy acquisition data, energy storage data and energy consumption data based on a preset Markov decision process model to obtain the current energy state; The processing unit is configured to make decisions based on the reward rules of the Markov decision process model, using a preset deep reinforcement learning agent, on the current energy state, the multiple early exit structure data, and the inference results of each exit, to obtain decision results. The decision results include the early exit selection of the embedded neural network and the speed parameters of the vehicle's onboard vibration energy acquisition device.

9. An embedded neural network inference device for vehicle-mounted vibration energy, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the embedded neural network inference method for vehicle vibration energy as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the embedded neural network inference method for vehicle vibration energy as described in any one of claims 1 to 7.

Images