Resource allocation method of vehicle-mounted network, vehicle-mounted network server, system and medium

Abstract

The invention discloses a resource allocation method of a vehicle-mounted network, a vehicle-mounted network server, a system and a medium. The method comprises the following steps: receiving request content of a target vehicle and request delay of the target vehicle, and obtaining environment data corresponding to a vehicle-mounted network, basic data of each base station in the vehicle-mounted network and mutual data between each base station and the target vehicle according to the request content; inputting the request content, the request delay, the environment data, the basic data and the mutual data into an active reasoning model, so that the active reasoning model combines priori knowledge to perform task unloading, and allocates a proper target base station for the request vehicle to process an unloading task; wherein the active reasoning model is constructed based on a reinforcement learning algorithm of a reward-free function and a free energy principle. In this way, the accuracy and rationality of resource allocation can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of resource allocation, and in particular to a resource allocation method, a vehicle network server, a system, and a medium for a vehicle network. Background Art

[0002] Intelligent transportation system is an important part of modern urban management and transportation, and its development benefits from the progress of wireless communication technology. Mobile edge computing (MEC) plays a key role in this context, placing computing and storage resources at the edge of the network to achieve efficient processing of vehicle requests, reduce dependence on the cloud, thereby reducing latency and improving bandwidth utilization.

[0003] The accuracy and rationality of resource allocation in current vehicle networks needs to be improved. Summary of the invention

[0004] In order to solve the above problems, the vehicle network resource allocation method, vehicle network server, system, and medium provided in the present application can improve the accuracy and rationality of resource allocation.

[0005] In order to solve the above technical problems, a technical solution adopted in the present application is: to provide a resource allocation method for an in-vehicle network, the method comprising: receiving the request content of a target vehicle and the request delay of the target vehicle, and obtaining the environmental data corresponding to the in-vehicle network, the basic data of each base station in the in-vehicle network, and the mutual data between each base station and the target vehicle according to the request content; inputting the request content, request delay, environmental data, basic data and mutual data into an active reasoning model, so that the active reasoning model combines prior knowledge to perform task offloading, and allocates a suitable target base station to the target vehicle to handle the offloading task; wherein the active reasoning model is constructed based on a reinforcement learning algorithm without a reward function and the free energy principle.

[0006] Among them, the request content, request delay, environmental data, basic data and mutual data are input into the active reasoning model, so that the active reasoning model combines prior knowledge to perform task offloading, and allocates a suitable target base station to the target vehicle to handle the unloading task, including: inputting the request content, request delay, environmental data, basic data and mutual data into the active reasoning model to obtain several base station strategies to be determined; combining prior knowledge to select a base station strategy for several base station strategies to be determined, and allocating a suitable target base station to the target vehicle to handle the unloading task.

[0007] Among them, the request content, request delay, environmental data, basic data and mutual data are input into the active reasoning model to obtain several base station strategies to be determined, including: inputting the request content, request delay, environmental data, basic data and mutual data into the active reasoning model as the state space, and pruning the state space in combination with prior knowledge; using the pruned state space for reasoning to obtain several base station strategies to be determined.

[0008] Among them, the pruned state space is used for reasoning to obtain several base station strategies to be determined, including: using the pruned state space for reasoning to obtain the KL divergence between the future observation distribution and the actual observation distribution, which is represented as the perception part; and using the KL divergence to judge the mutual information between the current strategy and the observation state, which is represented as the active part; wherein the perception part is obtained based on the likelihood and strategy between the state space of the external world model and the internal belief model, the parameters of the active reasoning model and the conditional observation probability; the active part is obtained based on the likelihood and mutual information between the state space of the internal belief model, the future observations of the model, the parameters of the active reasoning model and the strategy; according to the perception part and the active part, several base station strategies to be determined are obtained.

[0009] Among them, according to the sensing part and the active part, a number of base station strategies to be determined are obtained, including: according to the sensing part and the active part, the expected future free energy and a number of base station strategies to be determined are obtained.

[0010] The method further includes: obtaining a quantized value using the prediction error obtained by the perception part and the information entropy of the active part; the quantized value is used to characterize the intelligence level of the active inference model.

[0011] Among them, the basic data includes: the computing power of each base station, the remaining cache space of each base station, and the reputation of each base station; the mutual data includes the signal-to-noise ratio between the target vehicle and each base station.

[0012] The reputation of the base station is adjusted in the following ways: in response to any base station successfully completing a request within the expected delay, its reputation is enhanced; in response to any base station failing to successfully complete a request within the expected delay, its reputation is reduced.

[0013] In order to solve the above technical problems, a technical solution adopted in the present application is: to provide an in-vehicle network server, which includes a processor and a memory and a communication module coupled to the processor; wherein the memory is used to store computer programs, and the processor is used to execute computer programs to implement the method provided by the above technical solution.

[0014] In order to solve the above technical problems, a technical solution adopted in the present application is: to provide an in-vehicle network system, which includes an in-vehicle network server and several base stations. The in-vehicle network server is such as the in-vehicle network server provided by the above technical solution.

[0015] To solve the above technical problems, a technical solution adopted in the present application is: to provide a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a processor, it is used to implement the method provided by the above technical solution.

[0016] The beneficial effects of the embodiments of the present application are as follows: Different from the prior art, the vehicle network resource allocation method, vehicle network server, system, and medium provided by the present application utilize a reinforcement learning algorithm based on no reward function and the free energy principle to construct an active reasoning model that combines prior knowledge to offload tasks from the target vehicle's request content, and allocates a suitable target base station to the target vehicle to handle the offloading task. In this process, the active reasoning model does not need to adopt a reward function, thereby reducing the flexibility and generalization limitations caused by over-reliance on the reward function. The active reasoning model combines prior knowledge, which helps the active reasoning model identify and focus on important perception features and environmental change features, thereby reducing uncertainty and improving the accuracy of resource allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0018] Figure 1 It is a flowchart of an embodiment of a resource allocation method for an in-vehicle network provided by the present application;

[0019] Figure 2 It is a flowchart of another embodiment of the resource allocation method of the vehicle network provided by the present application;

[0020] Figure 3 This application provides Figure 2 A schematic diagram of a flow chart of an embodiment of step 22;

[0021] Figure 4 This application provides Figure 3 A flow chart of an embodiment of step 222 in FIG.

[0022] Figure 5 This application provides Figure 4 A schematic diagram of a flow chart of an embodiment of step 42;

[0023] Figure 6 It is a schematic diagram of the structure of the active inference model provided by this application;

[0024] Figure 7 It is a structural diagram of an embodiment of a vehicle-mounted network server provided by the present application;

[0025] Figure 8 It is a structural diagram of an embodiment of the vehicle network system provided by the present application;

[0026] Fig. 9 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0028] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0029] Intelligent transportation system is an important part of modern urban management and transportation, and its development benefits from the progress of wireless communication technology. Mobile edge computing (MEC) plays a key role in this context, placing computing and storage resources at the edge of the network to achieve efficient processing of vehicle requests, reduce dependence on the cloud, thereby reducing latency and improving bandwidth utilization.

[0030] The accuracy and rationality of resource allocation in current vehicle networks needs to be improved.

[0031] Based on this, the present application proposes to use a reinforcement learning algorithm based on no reward function and the free energy principle to construct an active reasoning model to combine prior knowledge to offload the request content of the target vehicle, and assign a suitable target base station to the target vehicle to handle the offloading task. In this process, the active reasoning model does not need to adopt a reward function, thereby reducing the flexibility and generalization limitations caused by over-reliance on the reward function. The active reasoning model combined with prior knowledge helps the active reasoning model to identify and focus on important perception features and environmental change features, thereby reducing uncertainty and improving the accuracy of resource allocation. Please refer to any of the following embodiments for details.

[0032] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of a method for allocating resources of an in-vehicle network provided by the present application. The method comprises:

[0033] Step 11: Receive the request content and request delay of the target vehicle, and obtain the environmental data corresponding to the vehicle network, the basic data of each base station in the vehicle network, and the mutual data between each base station and the target vehicle according to the request content.

[0034] In some embodiments, the request content of the target vehicle may include a request for data upload and / or data download. For example, if the target vehicle needs to upload the data collected at this time, the data to be uploaded can be used as the request upload content. For example, if the target vehicle needs to download data, the data to be downloaded can be used as the request download content.

[0035] In some embodiments, the request delay of the target vehicle refers to the delay time. The delay time may refer to the transmission time of data from the target vehicle to the base station. This transmission time includes not only the time for the data packet to propagate on the physical transmission medium, but also the time for the data packet to be processed on the network device and the time waiting for processing in the queue. The delay is usually expressed in milliseconds (ms), although in some applications, a more precise measurement may also be in microseconds (μs).

[0036] In some embodiments, the environmental data corresponding to the vehicle network may be real-time, and may include the number of base stations, the number of servers, a number of vehicles, and basic data of each base station, each server, and each vehicle in the vehicle network.

[0037] In some embodiments, basic data includes: computing power of each base station, remaining cache space of each base station, reputation of each base station; mutual data includes signal-to-noise ratio between the target vehicle and each base station. The channel between the vehicle and the base station follows a time-varying correlated quasi-static and frequency-flat fading model. It is assumed that data is transmitted in segments with fixed time intervals. In each time period, the channel state remains stable, while there is a certain correlation between the channel states in different time periods.

[0038] The computing power of each base station is related to the hardware circuit of the base station. If a processor with strong computing power is used, the computing power of the base station is strong. Strong computing power is reflected in high computing efficiency.

[0039] Each base station has a corresponding storage function, which can be used to cache corresponding data, such as receiving the request content of the target vehicle and caching the data generated by processing the request content.

[0040] In some embodiments, the reputation of a base station is used to characterize the success rate of the base station in executing the requested task, wherein the reputation is represented by a numerical value, and the numerical value of the reputation is proportional to the success rate of the base station in executing the requested task.

[0041] In some embodiments, the reputation of a base station is adjusted by:

[0042] In response to any base station successfully completing the request within the expected delay, its reputation is enhanced.

[0043] In response to any base station failing to successfully complete the request within the expected delay, its reputation is reduced.

[0044] In some embodiments, each base station has a reputation, denoted as G. When a base station successfully completes a request within the expected delay, its reputation can be enhanced. This enhancement is quantified as G′=G+G c . G c The specific value of depends on the current reputation level of the base station. It is designed to be inversely proportional to the current reputation level, ensuring that increases in reputation are more significant for base stations with lower reputations. This means that base stations with lower reputations require larger increases (e.g. G c = K + 1) to achieve significant improvements, while base stations with higher reputation experience smaller range increases (e.g. G c =K).

[0045] If the selected base station fails to complete the request, its reputation needs to be reduced accordingly, which can be expressed as G′=GG c Specifically, if the base station lacks sufficient cache or bandwidth to receive and process requests, the reputation degradation is more serious (e.g., G c= K + 1). On the contrary, if the base station has enough resources to process the request but still fails, the reputation will be reduced less (e.g. G c =K).

[0046] Step 12: Input the request content, request delay, environmental data, basic data and mutual data into the active reasoning model, so that the active reasoning model combines prior knowledge to perform task offloading and allocates a suitable target base station to the target vehicle to handle the offloading task.

[0047] Among them, the active reasoning model is built based on the reinforcement learning algorithm without reward function and the free energy principle.

[0048] Task offloading in edge computing refers to the process of completing part or all of the computing work on edge devices for tasks or parts of tasks that originally need to be processed in the cloud, thereby reducing the burden on the cloud and reducing response time and transmission costs. In task offloading, edge devices can take on more computing and storage tasks to meet user needs. Task offloading can be performed automatically between the cloud and edge devices according to some rules, thereby improving the efficiency and reliability of the system. In this application, it is reflected in the allocation of a suitable target base station to the target vehicle to handle the offloaded task.

[0049] In some embodiments, the active inference model can infer several base station strategies based on the request content, request delay, environmental data, basic data and mutual data. Then, a target base station strategy is selected from several base station strategies in combination with prior knowledge. The base station in the target base station strategy can execute the request content of the target vehicle, thereby completing the resource allocation of the vehicle network. In some embodiments, the active inference model can be trained using relevant training samples to learn the corresponding base station selection strategy. The active inference model after training can infer several base station strategies based on the request content, request delay, environmental data, basic data and mutual data.

[0050] Prior knowledge can be knowledge about the attributes, laws and relationships of things, or it can be knowledge acquired through personal experience, social culture, learning experience, scientific theory, etc. In cognitive activities such as processing information and making decisions, prior knowledge can play a guiding and auxiliary role, helping active reasoning models to quickly process information and make decisions.

[0051] In an application scenario, the active inference model can infer several base station strategies based on request content, request delay, environmental data, basic data, and mutual data. For example, strategy A, strategy B, strategy C, and strategy D. At this time, the prior knowledge indicates that base station a in strategy A does not meet the requirements, base station b in strategy B does not meet the requirements, and base station c in strategy C does not meet the requirements. Therefore, the active inference model will use strategy D as the target base station strategy for the target vehicle.

[0052] In this embodiment, an active reasoning model is constructed using a reinforcement learning algorithm based on no reward function, and a base station strategy is selected for the target vehicle's request content in combination with prior knowledge, and a target base station strategy corresponding to the target vehicle is selected. In this process, the active reasoning model does not need to adopt a reward function, thereby reducing the flexibility limitations caused by over-reliance on the reward function. The active reasoning model is combined with prior knowledge, which helps the active reasoning model identify and focus on important perceptual features, thereby reducing uncertainty and improving the accuracy of resource allocation.

[0053] See also Figure 2 , Figure 2 1 is a flow chart of another embodiment of a method for allocating resources of an in-vehicle network provided by the present application. The method comprises:

[0054] Step 21: Receive the request content and request delay of the target vehicle, and obtain the environmental data corresponding to the vehicle network, the basic data of each base station in the vehicle network, and the mutual data between each base station and the target vehicle according to the request content.

[0055] Step 22: Input the request content, request delay, environmental data, basic data and mutual data into the active reasoning model to obtain several base station strategies to be determined.

[0056] In some embodiments, see Figure 3 , step 22 may be the following process:

[0057] Step 221: Input the request content, request delay, environmental data, basic data and mutual data as the state space into the active inference model, and prune the state space in combination with prior knowledge.

[0058] State space is a term in control engineering. State refers to an ordered set of the minimum number of variables that can determine the state of the system. The so-called state space refers to the set of all possible states of the system. In simple terms, the state space can be regarded as a space with state variables as coordinate axes, so the state of the system can be represented as a vector in this space. State space representation is a mathematical model that represents a physical system as a set of inputs, outputs and states, and the relationship between inputs, outputs and states can be described by many first-order differential equations. In order to make the mathematical model not affected by the number of inputs, outputs and states, the inputs, outputs and states are all represented in the form of vectors, and the differential equations (if it is a linear non-time-varying system, the differential equations can be converted into algebraic equations) are represented in the form of matrices. State space representation provides a convenient and simple method to analyze and model systems with multiple inputs and multiple outputs. The general frequency domain system processing method is limited to systems with constant coefficients and initial conditions of 0. The state space representation has no restrictions on the coefficients and initial conditions of the system.

[0059] In some embodiments, the state space may be pruned using prior knowledge to obtain a corresponding new state space and actual observation values.

[0060] Pruning the state space refers to the process of screening several states in the state space. For example, the state space includes four states ABCD. Using prior knowledge, the state A is pruned to obtain a state space with only three states BCD.

[0061] In quantitative forecasting analysis, actual observations refer to data records about the predicted object and related factors. In forecasting work, whether it is the identification of mathematical models, the estimation and testing of parameters, or the calculation of predicted values ​​and forecast errors, they must be based on actual observations. Therefore, the quantity and quality of actual observations have a great impact on the accuracy of forecasts.

[0062] Step 222: Use the pruned state space to perform reasoning to obtain several base station strategies to be determined.

[0063] See also Figure 4 , step 222 may be the following process:

[0064] Step 41: Use the pruned state space for reasoning to obtain the KL divergence between the future observation value distribution and the actual observation value distribution, which is represented as the perception part, and use the KL divergence to judge the mutual information between the current strategy and the observation state, which is represented as the active part.

[0065] Future observations refer to sample values ​​about the future obtained through measurement or determination. These future observations are usually based on historical data and current status, and use prediction models to reveal possible changes or status in the future.

[0066] That is, in step 41, the entire solution is divided into a sensing part and an active part, so as to distinguish the data generated in the process and improve the accuracy of subsequent calculations combining the sensing part and the active part.

[0067] Among them, the future observation distribution is expressed as follows:

[0068]

[0069] Among them, θ represents the parameters of the active inference model, π represents the strategy, Represents environmental parameters, Indicates the status, represents the conditional observation probability.

[0070] where the actual observed distribution is expressed as follows:

[0071]

[0072] The perception part is obtained based on the state space of the external world model and the internal belief model, the likelihood and strategy between the parameters of the active inference model and the conditional observation probability. The parameters of the active inference model are the hyperparameters of the active inference model.

[0073] Among them, the distribution corresponding to the perception part can be expressed by the following formula:

[0074]

[0075] The active part is obtained based on the likelihood and mutual information between the state space of the internal belief model, future observations of the model, parameters of the active inference model and the strategy.

[0076] Among them, the distribution corresponding to the active part can be expressed by the following formula:

[0077]

[0078] KL divergence is a measure of the asymmetry of the difference between two probability distributions.

[0079] Step 42: According to the sensing part and the active part, several base station strategies to be determined are obtained.

[0080] In some embodiments, see Figure 5 , step 42 may be the following process:

[0081] Step 421: Based on the sensing part and the active part, obtain some expected future free energies.

[0082] In some embodiments, the free energy can be determined using KL divergence. Since there are several states in the state space, several expected future free energies can be obtained in this way.

[0083] Step 422: construct a diagonal Gaussian distribution using a number of expected future free energies to obtain a number of base station strategies to be determined.

[0084] Step 23: Select a base station strategy from several base station strategies to be determined in combination with prior knowledge, and assign a suitable target base station to the target vehicle to handle the unloading task.

[0085] Among them, the active reasoning model is built based on the reinforcement learning algorithm without reward function and the free energy principle.

[0086] In this embodiment, an active reasoning model is constructed using a reinforcement learning algorithm based on no reward function, and a base station strategy is selected for the target vehicle's request content in combination with prior knowledge, and a target base station strategy corresponding to the target vehicle is selected. In this process, the active reasoning model does not need to adopt a reward function, thereby reducing the flexibility limitations caused by over-reliance on the reward function. The active reasoning model is combined with prior knowledge, which helps the active reasoning model identify and focus on important perceptual features, thereby reducing uncertainty and improving the accuracy of resource allocation.

[0087] Furthermore, the state space is pruned in combination with prior knowledge to reduce the complexity of the state space, thereby improving the accuracy of the active inference model.

[0088] In all scenarios, the state space includes random parameters, such as vehicle request delays, which vary between vehicles. These unpredictable variables further reduce the accuracy of the algorithm. To overcome these challenges, the state space is pruned and relevant environmental random variables e are incorporated. v (t). Therefore, the state space of base station u~(u∈U) at time t can be expressed as:

[0089]

[0090] where cont m Indicates the size of the request content for vehicle m. represents the signal-to-noise ratio between vehicle m and base station u, P u represents the computing power provided by the edge computing server to base station u, C u Indicates the remaining cache space of base station u. u represents the reputation of base station u, B val represents the remaining allocatable total bandwidth shared by all base stations. Finally, delay m represents the delay requested by vehicle m.

[0091] Strategy selection aims to exploit prior knowledge and outcome-oriented decision making to maximize success rate and minimize expected free energy. m (t), where a m ∈{1,2,…,U}, represents the index of the base station assigned to vehicle request m at time t.

[0092] At time t, the channel is in a stable state, and there is a certain degree of correlation between the channel states at different time points. This correlation is determined by the channel fading coefficient characterization, which has an impact on the signal-to-noise ratio. The channel correlation between time t-1 and t follows the Jakes model. The downlink correlation coefficient of vehicle m is expressed as D m (t), channel correlation coefficient It can be expressed as:

[0093]

[0094] where σ m represents a complex Gaussian random variable with zero mean and unit variance, that is, Then the signal-to-noise ratio at time t can be expressed as:

[0095]

[0096] in represents the long-term average signal-to-noise ratio between vehicle m and base station u. According to Shannon’s theorem and the environment state, at time t, the time from sending a vehicle request to receiving the content can be expressed by the time model Calculation, expressed as:

[0097]

[0098] in represents the bandwidth that can be allocated according to the current reputation of base station u. In order to process a single request, a certain number of CPU cycles are required, denoted as v m (t). The computation time can be expressed as follows:

[0099]

[0100] Total delay It is the sum of transmission time and computation time.

[0101] The technical solution of this application is further described below:

[0102] To minimize free energy, it is necessary to enhance the cognition of the intrinsic belief model and adopt appropriate strategies to achieve goals and requirements consistent with human priors. However, after examining the key factors, it is clear that purely data-driven active inference models are not sufficient. Challenges for this application include poor generalization, difficulty in capturing variables in dynamic environments, and accurate prediction in high-dimensional state spaces. Random variables further exacerbate prediction errors and reduce the accuracy of the overall model.

[0103] If the agent module’s preferences are completely determined by specific data and reward functions, its generalization ability and intelligence will be affected, limiting its adaptability and flexibility.

[0104] Therefore, the present application proposes to further enhance the active inference model of the present application by incorporating valuable human prior knowledge to minimize free energy. By leveraging this knowledge, key variables about the dynamic environment can be extracted and integrated. This integration enables the active inference model of the present application to effectively handle the inherent uncertainty of the world, laying the foundation for an in-vehicle network that can predict future events and develop appropriate action plans. In addition, by leveraging prior knowledge and acquired data, irrelevant information in the high-dimensional state space can be intelligently pruned. This pruning process not only reduces the computational burden of the model, but also improves the accuracy of its predictions.

[0105] In some embodiments, in combination Figure 6 Explanation: The active reasoning model includes: intrinsic belief model, agent module, prior knowledge module, transition model and generation model.

[0106] The framework can be divided into the external environment part, the variational free energy decomposition part, the model reasoning and behavior selection part, and the quantitative intelligence part. Among them, the model reasoning and behavior selection part is the core component. This part needs to obtain sensory observations from the external environment and infer the state of the actual external environment by combining prior knowledge and internal belief models. The external environment part can include Figure 6 Transition models, generative models, and environments in .

[0107] It then decomposes the free energy into an active part and a sensing part using the principle of minimizing variational free energy factorization. By calculating the free energy, it selects the most appropriate strategy to minimize the free energy while being consistent with human preferences.

[0108] In addition, through the perception part and the quantitative intelligence part, indicators such as the intelligence acquisition or intelligence acquisition rate of the current model can be calculated to evaluate the performance of the agent module. The external environment part represents the actual environment in which the agent operates, while the variational free energy decomposition part provides a mathematical expression of the free energy function, integrating components such as sensory observation, prior knowledge, and intrinsic belief models. This overall framework enables the agent module to actively infer based on perception, prior knowledge, and intrinsic belief models, select the best strategy to adapt to the external environment, and quantify and evaluate its intelligence level.

[0109] Different from the traditional reinforcement learning architecture, the active inference model proposed in this application is oriented towards the future expected free energy and optimizes the model from the perspective of state and energy.

[0110] The goal of the active inference model is to minimize the surprise of future observations. Therefore, the agent module uses the state transfer function p(s t |s t-1 ,θ,π) and the likelihood mapping p(o t |s t,π) constructs a generative model of observations.

[0111] The posterior distribution over the states is approximately the variational distribution q(s t |o t ,π). By decomposing the free energy into expected information gain and external terms, the agent can optimize its behavior to minimize surprises about future observations, allowing the agent to balance exploration and exploitation.

[0112] Crucially, the agent needs to have a prior distribution p(o t ). It allows to view action selection as an inference problem, where the goal is to minimize the expected free energy F by inferring actions that will lead to the desired outcome. To put it into practice, it is necessary to define the preferred outcome p for the agent module that matches human expectations. Φ (o t ), learn an intrinsic belief model q(o) that can capture the underlying dynamics of the environment t |s t , θ, π), make predictions about future observations, and learn a behavior model q(π) that makes appropriate decisions consistent with the agent’s preferences, thereby minimizing the free energy.

[0113] Prior knowledge is guided and embedded into the state space, using dynamic environmental variables as additional information to enrich the intrinsic belief model. Through this integration, the belief in future observations is improved, ensuring compliance with the principle of minimizing free energy while being consistent with human prior knowledge. After embedding prior knowledge and pruning, the resulting state and observation values ​​can be expressed as s k t and k t .

[0114] In addition, we consider the v t The environmental random variables are used to capture the inherent uncertainty in a dynamic environment. The process of minimizing the free energy can be rewritten as follows:

[0115]

[0116] In order to determine the best action, this application focuses on optimizing the distribution q(π) at each time step and selecting the action that best matches the prior knowledge. It is worth noting that the free energy term can be decomposed into a single term at the planning time H The sum of This decomposition enables the present application to analyze the specific contribution of each term in the overall free energy minimization process. It can be expressed as follows:

[0117]

[0118] By incorporating prior knowledge and pruned state information into the input, the active reasoning model of the present application leverages human prior knowledge and reduces the dimensionality of the state space. In addition, including environmental random variables in the input provides a more comprehensive understanding of the uncertain dynamic factors of the environment. Therefore, the active reasoning model of the present application promotes state-to-state mapping that is very similar to human reasoning. By obtaining a model that accurately predicts the consequences of its behavior, the vehicle network gains the ability to predict its future course of action and predict potential outcomes. This ability significantly improves the reliability and effectiveness of the active reasoning model.

[0119] In an application scenario, the external environment consists of U base stations (or U roadside units) that can receive and process vehicle requests, U edge computing servers that provide computing power, and M heterogeneous vehicles. After the vehicle sends a request, the method based on this application needs to find a suitable base station to effectively process the request and receive the request content back. The base station is then the resource to be allocated, which can also be regarded as a task offloading scenario. The signal-to-noise ratio between vehicle m (m∈m) and base station u (u∈u) can be represented by a random variable, denoted as

[0120] The channel between the vehicle and the base station follows a time-varying correlated quasi-stationary and frequency-flat fading model. It is assumed that data is transmitted in segments at regular time intervals. Within each time slot, the channel state remains stable, while there is a certain correlation between the channel states of different time slots.

[0121] To ensure the integrity of the base station's mission within the specified time and prevent potential negative consequences such as impaired vehicle user experience or accidents, a reputation system is introduced. Each base station is assigned a reputation score that is proportional to the success rate of its mission execution. A feedback mechanism is used to update the reputation score. Positive feedback from successful mission completion increases the reputation of the base station, while negative feedback decreases it. Base stations with high reputation scores are granted certain privileges, such as access to a larger pool of available resources. Conversely, when the reputation of a base station drops below a predetermined threshold, penalties are imposed, resulting in a reduction in available resources until the reputation score is restored.

[0122] To guide the decision-making process, the reputation score is incorporated into the state space as prior knowledge. It is worth noting that the reputation model is different from a complex reward function. Instead, it involves simple numerical addition and subtraction operations and does not rely heavily on environmental factors. This formulation facilitates stronger generalization and makes implementation across different scenarios easier.

[0123] Vehicular environments are characterized by high mobility and extensive interactions between vehicles, resulting in rapid dynamics. Realistic scenarios, especially those with drastic changes, require consideration of time-varying statistical laws. For example, vehicles may encounter various situations during operation, such as unexpectedly entering a congested road or being affected by changing weather conditions. Traditional resource management schemes usually assume that the dynamic changes of the environment follow a specific pattern, such as a Markov decision process (MDP), and train the model by setting specific hyperparameters and carefully designing reward functions.

[0124] However, in realistic vehicular environments, these static approaches may prove to be insufficient as designing specific parameters and reward functions for each environment becomes impractical.

[0125] Therefore, this application adopts the concept of a continuous dynamic environment, in which no hyperparameter is a fixed value, but varies within a specific range. These hyperparameters include key information such as signal-to-noise ratio, state transition probability, allocatable bandwidth of the base station, available cache space of the base station, type and size of vehicle request content, unit variation range of reputation, and downlink correlation coefficient between the base station and the user. By operating in such a realistic and dynamic environment, this application has two goals:

[0126] Allocate resources efficiently in this dynamic and realistic environment and rigorously test active inference models under challenging conditions.

[0127] The resource allocation process can be summarized as follows:

[0128] First, the system senses the state from the external environment and obtains environmental random variables.

[0129] These variables are embedded in the current state space together with reputation to form sensory input. The state perception is then fed back into the intrinsic belief model, where the current input state is pruned based on the variational free energy principle and prior knowledge guidance to remove redundant factors. Based on minimizing the active and perceptual parts of free energy, the state at the next moment is derived and predicted, and an appropriate policy distribution q(π) is selected to interact with the environment. Finally, the brain's intrinsic belief model is updated based on the KL divergence between the predicted distribution and the true distribution. The model training and prediction process is guided by prior knowledge macro to make it consistent with human preferences, accelerate the decision-making process, and ensure consistency with human cognition. The following flow outlines the complete reasoning process of q(π).

[0130] First, initialize:

[0131] 1: Initialize U base stations, M vehicles, dynamic environment Env, and prior knowledge Er.

[0132] 2: Initialize training iterations Q, maximum step size T, and number of candidate strategies C.

[0133] 3: Initialize the likelihood p(ot|st), parameter distribution p(θ), and global prior pΦ(ot).

[0134] 4: Initialize the factorized belief about the action sequence q(π)←N(0,I).

[0135] Secondly, loop according to the number of iterations, during which sensory input is obtained: s, e v ←Env.

[0136] Embed and trim:s k t ←(st,Er,e v t ).

[0137] Then the candidate strategies are iterated, and in this process, the candidate state strategies are obtained: π(c)~q(π).

[0138] Update free energy: Fcπ.

[0139] Then infer the policy distribution: q(π)←refit(Fcπ)

[0140] Sample strategy: π~q(π)

[0141] st+1←Interact with the dynamic environment

[0142] Get the observed value: ot←O(O|st+1,π)

[0143] Update status: st←st+1.

[0144] In some embodiments, the KL divergence of the perception part and the information entropy of the active reasoning model can be used to obtain a quantized value; the quantized value is used to characterize the intelligence level of the active reasoning model.

[0145] See also Figure 7 , Figure 7 1 is a schematic diagram of the structure of an embodiment of the vehicle network server provided by the present application. The vehicle network server 70 includes a processor 71, a memory 72 coupled to the processor 71, and a communication module 73; wherein the memory 72 is used to store a computer program, and the processor 71 is used to execute the computer program to implement the following method:

[0146] Receive the request content and request delay of the target vehicle, and obtain the environmental data corresponding to the vehicle network, the basic data of each base station in the vehicle network, and the mutual data between each base station and the target vehicle according to the request content; input the request content, request delay, environmental data, basic data and mutual data into the active reasoning model, so that the active reasoning model combines prior knowledge to perform task offloading and allocates a suitable target base station to the target vehicle to handle the offloading task; wherein the active reasoning model is constructed based on the reinforcement learning algorithm without reward function and the free energy principle.

[0147] It can be understood that the processor 71 is also used to execute computer programs to implement the method of any of the above embodiments, which will not be elaborated here.

[0148] See also Figure 8 , Figure 8 The vehicle network system 80 includes a vehicle network server 70 and a plurality of base stations 81. The vehicle network server 70 is the vehicle network server 70 provided by the above technical solution.

[0149] See also Fig. 9 , Fig. 9 1 is a schematic diagram of a computer readable storage medium according to an embodiment of the present application. The computer readable storage medium 90 is used to store a computer program 91. When the computer program 91 is executed by a processor, it is used to implement the following method:

[0150] Receive the request content and request delay of the target vehicle, and obtain the environmental data corresponding to the vehicle network, the basic data of each base station in the vehicle network, and the mutual data between each base station and the target vehicle according to the request content; input the request content, request delay, environmental data, basic data and mutual data into the active reasoning model, so that the active reasoning model combines prior knowledge to perform task offloading and allocates a suitable target base station to the target vehicle to handle the offloading task; wherein the active reasoning model is constructed based on the reinforcement learning algorithm without reward function and the free energy principle.

[0151] It can be understood that when the computer program 91 is executed by the processor, it is also used to implement the method of any of the above embodiments, which will not be elaborated here.

[0152] To summarize, the vehicle network resource allocation method, vehicle network server, system, and medium provided in the present application utilize a reinforcement learning algorithm based on no reward function and the free energy principle to construct an active reasoning model that combines prior knowledge to offload tasks from the target vehicle's request content, and allocates a suitable target base station to the target vehicle to handle the offloading task. In this process, the active reasoning model does not need to adopt a reward function, thereby reducing the flexibility and generalization limitations caused by over-reliance on the reward function. The active reasoning model is combined with prior knowledge, which helps the active reasoning model identify and focus on important perception features and environmental change features, thereby reducing uncertainty and improving the accuracy of resource allocation.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only illustrative, for example, the division of the circuit or unit is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0155] In addition, each functional unit in each embodiment of the present application may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0156] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made according to the description and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A resource allocation method for an in-vehicle network, characterized in that: The method comprises: Receive the request content of the target vehicle and the request delay of the target vehicle, and obtain the environmental data corresponding to the vehicle network, the basic data of each base station in the vehicle network, and the mutual data between each base station and the target vehicle according to the request content; The request content, the request delay, the environmental data, the basic data and the mutual data are input into an active reasoning model, so that the active reasoning model combines prior knowledge to perform task offloading and allocates a suitable target base station to the target vehicle to handle the offloading task; wherein the active reasoning model is constructed based on a reinforcement learning algorithm without a reward function and a free energy principle.

2. The method according to claim 1, characterized in that The step of inputting the request content, the request delay, the environmental data, the basic data and the mutual data into an active reasoning model, so that the active reasoning model combines prior knowledge to perform task offloading and allocates a suitable target base station to the target vehicle to process the offloading task, comprises: Inputting the request content, the request delay, the environmental data, the basic data and the mutual data into an active reasoning model to obtain a plurality of base station strategies to be determined; A base station strategy is selected for the plurality of base station strategies to be determined in combination with the priori knowledge, and a suitable target base station is allocated to the target vehicle to process the unloading task.

3. The method according to claim 2, characterized in that The step of inputting the request content, the request delay, the environmental data, the basic data and the mutual data into an active reasoning model to obtain a plurality of base station strategies to be determined, including: The request content, the request delay, the environmental data, the basic data and the mutual data are input into an active inference model as a state space, and the state space is pruned in combination with the prior knowledge; The pruned state space is used for reasoning to obtain several base station strategies to be determined.

4. The method according to claim 3, characterized in that The pruned state space is used for reasoning to obtain several base station strategies to be determined, including: The pruned state space is used for reasoning to obtain the KL divergence between the future observation distribution and the actual observation distribution, which is represented as the perception part; and the KL divergence is used to judge the mutual information between the current strategy and the observation state, which is represented as the active part; wherein the perception part is obtained based on the likelihood and strategy between the state space of the external world model and the internal belief model, the parameters of the active reasoning model and the conditional observation probability; the active part is obtained based on the likelihood and mutual information between the state space of the internal belief model, the future observations of the model, the parameters of the active reasoning model and the strategy; According to the sensing part and the active part, several base station strategies to be determined are obtained.

5. The method according to claim 4, characterized in that The method of obtaining a plurality of base station strategies to be determined based on the sensing part and the active part includes: According to the sensing part and the active part, the expected future free energy and several base station strategies to be determined are obtained.

6. The method according to claim 4, characterized in that The method further comprises: A quantized value is obtained by using the prediction error obtained by the perception part and the information entropy of the active part; the quantized value is used to characterize the intelligence level of the active inference model.

7. The method according to claim 1, characterized in that The basic data includes: the computing capacity of each base station, the remaining cache space of each base station, and the reputation of each base station; the mutual data includes the signal-to-noise ratio between the target vehicle and each base station.

8. The method according to claim 1, characterized in that: The reputation of the base station is adjusted by: In response to any base station successfully completing the request within the expected delay, enhancing its reputation; In response to any base station failing to successfully complete the request within the expected delay, its reputation is reduced.

9. A vehicle-mounted network server, characterized in that: The in-vehicle network server includes a processor and a memory and a communication module coupled to the processor; wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the method described in any one of claims 1-8.

10. A vehicle network system, characterized in that: The in-vehicle network system comprises an in-vehicle network server and several base stations, and the in-vehicle network server is the in-vehicle network server as claimed in claim 9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the computer program is used to implement the method according to any one of claims 1 to 8.

Images