Internet of vehicles optimization method based on blockchain, digital twin and edge computing

Abstract

The application belongs to the technical field of wireless communication, and discloses an Internet of Vehicles (IoV) optimization method based on a blockchain, a digital twin (DT) and edge computing (Mobile Edge Computing, MEC), wherein a binary task offloading decision is jointly optimized, an improved enhanced practical Byzantine fault tolerance (EPBFT) consensus algorithm is disclosed in the blockchain system to enhance the consensus procedure, then the selection of a consensus master node is performed, so that the consensus delay and failure rate are minimized to the greatest extent. The system is jointly optimized under the influence of factors such as edge node computing capacity, EPBFT node number limitation, task processing and blockchain consensus delay. The base stations (BSs) in the application not only serve as edge nodes in the MEC system, but also serve as consensus nodes for block consensus in the blockchain system. Due to the uncertainty of the task and the dynamic nature of the network, the task is converted into a Markov decision process, and Proximal Policy Optimization (PPO) is used to dynamically learn the optimal joint solution.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication technology and relates to a vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing. Background Technology

[0002] Currently, the widespread adoption of Internet of Vehicles (IoV) has impacted our daily lives, bringing significant convenience and attracting considerable attention from researchers. Due to the sheer volume of data, vehicles often cannot process computationally intensive or latency-sensitive tasks within a short timeframe. Therefore, Multi-access Edge Computing (MEC) technology has been proposed and widely applied in vehicle-to-everything (V2X) scenarios. By offloading computational tasks from the requesting vehicle to edge computing servers, network latency is significantly reduced, energy consumption is lowered, computational efficiency is improved, and the performance of IoV systems is enhanced. Simultaneously, as network scale expands, the complexity of current networks is also increasing. To improve network autonomy and reduce negative impacts on the physical network, Data Optimization (DT) technology is introduced. DT is an innovative technology that optimizes and verifies strategies through real-time interactive mapping of physical entities. Furthermore, because infrastructure is vulnerable to threats and control, information data can easily be tampered with by attackers during the offloading process. Therefore, ensuring the security of mobile offloading data is crucial. Integrating blockchain, DT, and MEC networks into a unified system can help create a task processing service environment characterized by trustworthiness, transparency, security, immutability, and automation.

[0003] However, applying deep reinforcement learning (DRL) to multi-access mobile edge computing (MEC) systems based on improved practical Byzantine fault-tolerant (EPBFT) consensus still faces some challenges and problems:

[0004] 1) Network architecture integration issues

[0005] First, a multi-system architecture needs to be designed, including edge computing and blockchain systems, to ensure efficient data and task transmission and processing between these systems. Second, the integration of consensus mechanisms and computing resources is also crucial. The Practical Byzantine Fault Tolerance (PBFT) consensus algorithm requires frequent communication and information exchange between nodes, which consumes significant network bandwidth and computing resources. Therefore, the consensus algorithm needs to be optimized to reduce resource consumption and improve efficiency.

[0006] In particular, PBFT faces the following challenges: (1) Due to the single-point full node and two full node broadcasts, PBFT incurs high bandwidth overhead, limiting its application in large-scale networks; (2) Arbitrary selection of the master node carries the risk of selecting a malicious or faulty node, leading to consensus failure. Although the protocol modification solved this problem, its high communication complexity leads to resource waste, increased latency, and reduced system stability.

[0007] 2) Real-time and latency requirements

[0008] In MEC systems based on EPBFT consensus, real-time performance and latency requirements are paramount. MEC systems aim to provide low-latency, high-real-time services, thus demanding extremely fast data processing and task response speeds. However, the EPBFT consensus mechanism involves complex inter-node communication and multiple message passes, increasing network latency. This poses a significant challenge to applications requiring immediate feedback and processing, such as autonomous driving, real-time monitoring, and augmented reality. Therefore, optimizing the consensus algorithm to reduce communication overhead, improving consensus-reaching efficiency, and designing efficient data transmission and processing mechanisms are crucial for meeting real-time and latency requirements. Only by ensuring real-time system response and minimal latency can the advantages of MEC systems be fully realized in various applications.

[0009] 3) Trust and security issues

[0010] Due to the distributed and heterogeneous nature of MEC systems, reliable trust relationships need to be established between nodes to ensure the security of data transmission and processing. While the EPBFT consensus mechanism improves system reliability through fault-tolerant design, it also faces the risk of selecting malicious or faulty nodes, which could lead to consensus failure or system attacks. Therefore, it is necessary to combine encryption technology, access control, and node reputation assessment mechanisms to ensure that only trusted nodes can participate in the consensus process. Simultaneously, efficient security protocols must be designed to prevent data tampering, unauthorized access, and network attacks, ensuring the overall security and stability of the system. Only when trust and security issues are effectively resolved can MEC systems provide reliable computing and services, meeting the high security requirements of users and applications.

[0011] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0012] (1) Insufficient integration of network architecture: Existing technologies are difficult to efficiently integrate MEC systems and blockchain systems, resulting in low efficiency in the transmission and processing of data and tasks.

[0013] (2) Real-time and latency requirements are difficult to meet: The EPBFT consensus mechanism involves complex inter-node communication and multiple message transmissions, which leads to increased network latency in application scenarios that require low latency and high real-time performance (such as autonomous driving and real-time monitoring), making it difficult to meet real-time requirements and affecting the response speed and overall performance of the MEC system.

[0014] (3) Trust and security issues: Although the EPBFT consensus mechanism improves the reliability of the system through fault-tolerant design, the risk of selecting malicious or faulty nodes still exists in dynamic, distributed MEC systems, which may lead to consensus failure or system attacks. Existing technologies have not yet fully solved the problem of how to effectively assess node reputation and establish a reliable trust mechanism. Summary of the Invention

[0015] To address the problems existing in the prior art, this invention provides a vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing.

[0016] This invention is implemented as follows: a vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing. The application agent's policy actor and critic network employ a deep reinforcement learning (DRL) method for decision-making, including:

[0017] Initialize the parameters of the policy actor network and the critic network, and set the training-related hyperparameters; the agent interacts with the environment based on the current policy, performs actions, and transitions between states; the entire segment in the environment is sampled using the parameters of the sampled policy actor network, and the trajectory is stored in memory; the discounted reward, advantage function, and objective function are calculated; the parameters of the policy actor network and the critic network, as well as the parameters of the sampled policy actor and critic networks, are updated; training is repeated until the policy converges, and the trained policy is used to compute unloading and select the master node.

[0018] Furthermore, the following steps are included:

[0019] S101. Initialize the parameters θ and ω of the agent's policy actor network and critic network, sample the parameters θ′ and ω′ of the policy actor network and critic network, the number of episodes, initialize the learning rates μ and σ of the policy actor network and critic network, the discount factor γ, and initialize the experience pool; initialize the network layout parameters, such as the input task data size D. i (t), workload C i Parameters such as (t);

[0020] S102. Initialize the state of the agent, the agent interacts with the environment, and the policy network generates actions based on the current policy;

[0021] S103. The agent executes the action generated by the actor, receives a reward, and updates its state.

[0022] S104. The agent executes the generated action, receives an immediate reward based on the executed action, and transfers the environmental state to the next state.

[0023] S105, Calculate the discounted reward;

[0024] S106. Calculate the advantage function, add the pruning factor to constrain the update rate, and calculate the objective function at the same time.

[0025] S107, Update the parameters of the policy actor and critic network;

[0026] S108. Update the sampling policy actor and critic network parameters according to the updated policy actor and critic network parameters.

[0027] S109. Repeatedly iterate the training, select the optimal action according to each state to obtain the maximum benefit, and finally obtain the optimal master node selection and calculation unloading strategy.

[0028] Furthermore, in S102, the agent interacts with the environment; at the beginning of each round, the system state s(t) is initialized, as follows:

[0029]

[0030] Among them, the amount of input data for the task workload of the task The link availability between moving vehicles (MVs) and parked vehicles (PVs) is The link connection rate between MVs and PVs is Computing resources allocated to each task Indicators of whether node k exhibits malicious behavior

[0031] Furthermore, in S103: the agent executes the generated action, receives an immediate reward based on the executed action, and transitions the environment state to the next state. The consensus for calculating the reward is as follows:

[0032]

[0033] In the above formula, Object(t) represents our objective function.

[0034] Furthermore, in S106: Calculate the advantage function:

[0035]

[0036] At this point:

[0037] δ t =r(t)+γV(s) t+1 ;w)-V(s t ;w).

[0038] Proximal Policy Optimization (PPO) introduces a J-based approach. θ′ Further improvement of the actor objective function (θ); by adding a pruning factor to constrain the update rate, and by maximizing the objective function to update the PPO actor, the formula is as follows:

[0039]

[0040] Where ∈ is a hyperparameter, the clip function will convert (π(a) to (π(a)). t |s t ;θ)) / π(a t |s t The value of θ′ is restricted to the range [1-, 1+] to ensure that the two distributions remain relatively close after minimizing the clip function.

[0041] Furthermore, in S107: the parameters of the policy actor and the critic network are updated; the parameters of the actor are updated by the following formula:

[0042]

[0043] Considering the mean squared error function of the value estimation, the loss function of the critic network is given: L critic (w)=[V(s t+1 ;w)-V(s t ;w)] 2 ,

[0044] Update using this formula:

[0045]

[0046] Where, δ t This indicates the TD error.

[0047] Another objective of this invention is to provide a vehicle-to-everything (V2X) optimization system based on blockchain, digital twins, and edge computing, characterized in that it includes:

[0048] The system initialization module initializes all necessary parameters, including the initialization of experience memory, policy actor network parameters θ and critic network parameters w, and sampling policy actor parameters θ' and critic network parameters w'.

[0049] The configuration module is used to set parameters for specific applications, including task parameters such as the size of the input task data;

[0050] The agent module is used to generate actions based on the current network state at the beginning of each cycle; it is also used to collect data samples, calculate the advantage function, and update the policy and value network during the policy evaluation process.

[0051] The action execution module is used to perform task unloading and master node selection during the consensus process.

[0052] The reward acquisition module is used to execute actions and calculate immediate rewards. This reward acquisition module is designed based on whether the system's constraints are met. If all constraints are met, a reward is obtained; otherwise, a penalty is imposed.

[0053] The state transition module is used to transition the system state from the current state to the next state.

[0054] The experience replay module is used to store the experience tuples for each system state, action, reward, and next state;

[0055] The data sampling module is used to extract a certain segment from the experience stored in the internal storage for learning.

[0056] The network update module is used to update the strategy actor network and critic network based on data from the experience replay module.

[0057] The parameter update module is used for updating the parameters of the policy actor network and the critic network, as well as the parameters of the sampling policy actor network and the critic network.

[0058] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform steps of a vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing.

[0059] Another object of the present invention is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of a vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing.

[0060] First, addressing the technical problems existing in the prior art and the difficulty in solving them, this paper closely analyzes, in conjunction with the technical solution to be protected by this invention and the results and data obtained during the research and development process, how the technical solution of this invention solves the technical problems, and the inventive technical effects brought about by solving these problems. The specific description is as follows:

[0061] Firstly, efficiency and security are paramount. This invention considers a vehicle-to-everything (V2X) optimization method based on blockchain, TD (Transmission Control), and MEC (Multi-access Edge Computing). It comprises two systems: an MEC system and a blockchain system. The MEC system is designed for task execution, while the blockchain system plays a crucial role in establishing a secure and reliable transaction platform for the MEC system. In the MEC system, PVs (Virtual Vehicles) form a parking cluster, acting as virtual edge servers, auxiliary MVs (Mobile Virtual Machines), base stations, and the blockchain system. BSs (Balanced Nodes) not only possess available computing resources but also act as relay nodes, enabling information exchange between MVs and PVs within the communication coverage area of ​​auxiliary devices with different computing resources. Each PV is equipped with an MEC server, providing specific computing resources to its associated MVs. Besides serving as edge nodes for task processing within the MEC system, each PS also functions as a consensus node for block consensus, possessing a dual function.

[0062] Secondly, considering the dynamic nature of the task, the volatility of network conditions, and the random behavior of nodes in consensus, we adopt PPO to learn the environment state and obtain the optimal joint decision-making strategy for unloading and master node selection.

[0063] Second, considering the technical solution as a whole or from a product perspective, the technical effects and advantages of the technical solution to be protected by this invention are specifically described as follows:

[0064] 1) Precise task unloading

[0065] This method significantly improves resource utilization and system performance in edge computing by introducing a precise task offloading strategy. The MEC system provides computing power to user devices to process tasks, while the consortium blockchain system provides trust and security guarantees for user devices during the task offloading process.

[0066] 2) High-performance consensus mechanism

[0067] An improved Practical Byzantine Fault-Tolerant (EPBFT) algorithm is adopted. This integrated EPBFT method balances scalability and complexity, overcomes the limitations of the PBFT algorithm, and significantly improves consensus performance.

[0068] 4) Minimize consensus latency and failure rate

[0069] Consensus latency refers to the time required to reach a consensus, and consensus failure can be caused by various reasons, such as unmet constraints or a malicious master node. This method minimizes consensus latency and failure rate by selecting the master node, thereby improving the stability and efficiency of the system.

[0070] 5) Integration of deep reinforcement learning

[0071] Combining DRL with MEC systems enables the system to learn and adjust itself based on real-time data. By employing the PPO algorithm, the system can still make optimal decisions even in the absence of explicit instructions.

[0072] 6) Improved system stability and reliability

[0073] By accurately calculating and optimizing the immediate rewards and state transitions after an action is executed, this method significantly enhances the stability and reliability of the system, especially when dealing with large amounts of data and high-concurrency requests.

[0074] 7) The network's self-learning and optimization capabilities

[0075] This method enables the system to continuously optimize its decision-making process through continuous iterative training and experience-based network updates, thereby improving overall performance.

[0076] 8) Joint optimization problem

[0077] This method jointly optimizes task offloading decisions and master node selection to improve the overall performance and efficiency of the system. This comprehensive optimization strategy not only improves resource utilization but also enhances the system's adaptability and stability in complex network environments.

[0078] 9) Complexity and dynamism

[0079] The complexity of networks and the dynamic nature of environments make traditional optimization methods ineffective in addressing these challenges. To address this issue, this method employs the PPO algorithm to improve the system's adaptability and performance in complex and dynamic environments.

[0080] 10) Real-time performance and efficiency

[0081] This method can improve the system's real-time response speed and resource utilization efficiency when processing tasks.

[0082] Third, the core of the vehicle-to-everything (V2X) optimization method based on blockchain, TD, and MEC provided by this invention lies in using mathematical models to guide the system's behavior and learning process. The technical effects brought about by these mathematical models can be explored based on their characteristics:

[0083] 1) Even the calculation of rewards

[0084] Even though the consensus calculation of rewards focuses on the overall network cost, consensus latency, and long failure rate.

[0085] Cost savings: By directly linking rewards to costs, this approach encourages cost reduction across the network and effectively improves resource utilization efficiency.

[0086] 2) Updates to the policy master actor and criterion network

[0087] The policy network is updated by randomly sampling small batches of empirical data and using gradient ascent to update the current policy network.

[0088] Strategy optimization: By continuously adjusting the policy network parameters, the system can learn and adopt more effective decision-making strategies.

[0089] Improved responsiveness: Using small batches of data enables the network to quickly adapt to environmental changes, enhancing the system's dynamic adjustment capabilities.

[0090] 3) Parameter update formula

[0091] This paper describes a method for updating the target network using policy Actor and Critic network parameters, involving the parameters of the current network and the target network.

[0092] Gradual strategy approximation: By introducing a pruning function, the system can smoothly transition to a new strategy, preventing performance fluctuations caused by abrupt changes.

[0093] Continuous learning and adaptation: This continuous parameter update mechanism ensures that the system can adapt to long-term environmental changes.

[0094] Fourth, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects:

[0095] The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0096] This invention, by introducing the PPO algorithm, implements a vehicle-to-everything (V2X) optimization method based on blockchain, TD (Transmission Control), and MEC (Multi-access Edge Computing), significantly improving network performance and service quality. This not only meets the growing demand for high-performance computing but also optimizes resource utilization and reduces network costs. In terms of commercial applications, this invention can be widely applied to scenarios requiring efficient processing of large amounts of data while ensuring data security and trust, bringing significant economic and social benefits to related industries. Therefore, this invention has broad market prospects and enormous commercial value. Attached Figure Description

[0097] Figure 1 This is a flowchart of a vehicle-to-everything (V2X) optimization method driven by MEC, based on blockchain, TD, and MEC.

[0098] Figure 2 This is a flowchart illustrating the implementation of a blockchain-driven vehicle-to-everything (V2X) optimization method based on blockchain, TD, and MEC.

[0099] Figure 3 This is a comparison chart of the overall convergence performance provided in the embodiments of the present invention. Detailed Implementation

[0100] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0101] Example 1

[0102] To address the problems existing in current technologies, this implementation provides a joint optimization method for MEC networks based on improved practical Byzantine fault tolerance (EPBFT) consensus. First, it comprises two systems: a multi-access edge computing system and a blockchain system. The MEC system is designed for task execution, while the blockchain system plays a crucial role in establishing a secure and reliable transaction platform for the MEC system.

[0103] In the network architecture under consideration, there are I mobile vehicles (MVs). stationary vehicles (PVs) and There are 1, 2, 3 base stations (BSs). The corresponding set of MVs, PVs, and BSs is represented as I = {1,...i,...Ι}. and Separately, PVs form a parking cluster, acting as virtual edge servers, auxiliary MVs, base stations, and a blockchain system. BSs themselves not only possess available computing resources but also act as relay nodes, enabling information exchange between MVs and PVs within the communication coverage area of ​​auxiliary devices with different computing resources. Each PV is equipped with an MEC server, providing specific computing resources to its associated MVs. Besides serving as an edge node for task processing in the MEC system, each PS also functions as a consensus node for block consensus in the blockchain system, possessing a dual function. Our system operates within discrete time slots, dividing the entire time range into intervals T, using... The duration of each time slot is represented by Δt.

[0104] Example 2

[0105] The dynamic nature of networks and the uncertainty of information acquisition make this problem quite complex. To effectively address this challenge, we remodel the problem as a Markov Decision Process (MDP). Since the variables involved are discontinuous, we choose to use the PPO algorithm, a DRL algorithm designed to solve reinforcement learning problems in continuous action spaces, which can support real-time online decision-making.

[0106] like Figure 1 As shown in the figure, this embodiment of the invention provides a flowchart of a vehicle-to-everything (V2X) optimization method based on blockchain, TD, and MEC, which specifically includes the following steps:

[0107] S101. Initialize the parameters θ and ω of the agent's policy actor network and critic network, sample the parameters θ′ and ω′ of the policy actor network and critic network, the number of episodes, initialize the learning rates μ and σ of the critic network and policy network, the discount factor γ, and initialize the experience pool; initialize the network layout parameters, such as the input task data size D. i (t), workload C i Parameters such as (t);

[0108] S102. Initialize the state of the agent, the agent interacts with the environment, and the main policy network generates actions based on the current policy;

[0109] S103. The agent executes the generated action, receives an immediate reward based on the executed action, and transfers the environmental state to the next state.

[0110] S104. Sample the entire segment in the environment according to the parameters of the actor network of the sampling strategy, and store the trajectory in memory;

[0111] S105, Calculate the discounted reward;

[0112] S106. Calculate the advantage function, add the pruning factor to constrain the update rate, and calculate the objective function at the same time.

[0113] S107, Update the parameters of the policy actor and critic network;

[0114] S108. Update the sampling policy actor and critic network parameters according to the updated policy actor and critic network parameters.

[0115] S109. Repeatedly iterate the training, select the optimal action according to each state to obtain the maximum benefit, and finally obtain the optimal computing resource allocation and computing offloading strategy.

[0116] Furthermore, in S102, the agent interacts with the environment; at the beginning of each round, the system state s(t) is initialized. Specifically:

[0117]

[0118] Among them, the amount of input data for the task workload of the task The link availability between moving vehicles (MVs) and parked vehicles (PVs) is [missing information]. The link connection rate between MVs and PVs is Computing resources allocated to each task Indicators of whether node k exhibits malicious behavior

[0119] Furthermore, in S103: the agent executes the generated action, receives an immediate reward based on the executed action, and transitions the environment state to the next state. The consensus for calculating the reward is as follows:

[0120]

[0121] In the above formula, Object(t) represents our objective function.

[0122] Furthermore, in S106: Calculate the dominance function:

[0123]

[0124] At this point:

[0125] δ t =r(t)+γV(s) t+1 ;w)-V(s t ;w).

[0126] To improve performance, PPO introduced a J-based approach. θ′ Further improvement of the actor objective function (θ). By incorporating a pruning factor to constrain the update rate, the PPO actor can be updated by maximizing the objective function, as shown in the formula:

[0127]

[0128] Where ∈ is a hyperparameter, the clip function will convert (π(a) to (π(a)). t |s t ;θ)) / π(a t |s tThe value of θ′ is restricted to the range [1-, 1+]. This method ensures that the two distributions remain relatively close after minimizing the clip function, avoiding significant differences.

[0129] Furthermore, in S107: update the parameters of the policy actor and the critic network; the actor parameters are updated by the following formula:

[0130]

[0131] This algorithm considers the mean squared error function of value estimation and gives the loss function of the critic network:

[0132] L critic (w)=[V(s t+1 ;w)-V(s t ;w)] 2 ,

[0133] It can also be updated using this formula:

[0134]

[0135] Where, δ t This indicates the TD error.

[0136] To elaborate on the vehicle-to-everything (V2X) optimization method based on blockchain, TD, and MEC, this invention provides two specific application embodiments, including key details of the implementation scheme.

[0137] Application Example 1: Intelligent City Monitoring System

[0138] 1) System Initialization

[0139] Deploy the main Actor and Critic networks in the smart city monitoring center. Initialize network parameters, including the number of rounds, training steps, and learning rate. Deploy multiple IoT devices (such as surveillance cameras and sensors) and base stations for data collection and transmission.

[0140] 2) Interaction between intelligent agents and the environment

[0141] Each IoT device acts as an intelligent agent, generating actions based on the state of its monitored environment (such as adjusting sensor sampling frequency, edge node data processing priority, or data upload frequency). Action generation is based on the current policy.

[0142] 3) Action execution and reward acquisition

[0143] After performing an action, the agent receives an immediate reward based on the action's effect (e.g., data transmission rate, data accuracy). The system then transitions to the next state, which may be due to changes in the environment or variations in the data flow over different time periods.

[0144] 4) Optimization and Updates

[0145] Based on the collected data and rewards, the policy actor and critic networks are updated to optimize the sensor control policy and the data processing priority of edge nodes. Reinforcement learning algorithms are used to progressively improve resource allocation, such as data transmission rate and computational resource allocation.

[0146] 5) Latency and performance optimization

[0147] The performance of the PPO algorithm is evaluated through simulation experiments or field tests. Key indicators such as communication network coverage, signal quality, and data transmission rate are analyzed and compared with traditional methods. Based on the evaluation results, the algorithm parameters are adjusted and optimized to further improve the performance and stability of the communication network. Through the above steps, the vehicle-to-everything (V2X) optimization method based on blockchain, TD, and MEC proposed in this invention can monitor a large number of IoT devices in real time in smart cities, such as intelligent lighting control and environmental monitoring.

[0148] Application Example 2: Government Service System

[0149] 1) System Initialization

[0150] Within the government service system, the center deploys a PPO network and initializes network parameters, while also deploying necessary infrastructure such as servers and databases. These facilities not only support real-time processing and data storage of payment transactions but also ensure the system operates securely and efficiently, meeting the security, stability, and performance requirements of government operations.

[0151] 2) Interaction between intelligent agents and the environment

[0152] Each server acts as an intelligent agent, continuously interacting with its environment and generating corresponding actions based on the current environmental state. These actions may include resource allocation, data processing optimization, and security controls to ensure the stability and performance optimization of financial and payment systems.

[0153] 3) Action execution and reward acquisition

[0154] After performing the current action, an immediate reward is awarded based on key indicators such as the rationality of resource allocation, data transmission rate, and security. The system then transitions to the next state depending on changes in the environment or the time period.

[0155] 4) Optimization and Updates

[0156] The parameters of the PPO network are updated using collected data and rewards. Through continuous iteration and training, the algorithm gradually improves the stability and efficiency of the communication system, thereby ensuring the security and reliability of the government service system.

[0157] 5) Reliability of government service systems

[0158] The PPO algorithm of this invention enables intelligent management and optimization of government service systems. This algorithm focuses on improving the reliability and stability of communication links, ensuring the security and integrity of data during transmission. Furthermore, the PPO algorithm optimizes the allocation and utilization efficiency of system resources, significantly improving the overall system's response speed and processing capacity, thereby enhancing user experience and system reliability. Through the above steps, this invention provides an efficient and intelligent solution for ensuring the smooth operation of government service systems.

[0159] In these two embodiments, the vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing provides an efficient, reliable, and energy-saving solution suitable for various application scenarios, from smart cities to government service systems, demonstrating its broad application potential and technological advantages.

[0160] To comprehensively evaluate the performance of this invention, we compared it with several representative benchmark algorithms. These benchmark algorithms each have their own characteristics and advantages, and are capable of effectively handling similar problems. By comparing the performance of these algorithms under the same conditions, we can more clearly understand the superiority of this invention and its potential for improvement.

[0161] 1) Random Unloading: The unloading decision for each MV is determined randomly. Each MV's task can choose to be processed independently on itself or on the BS. However, a task can only be processed in one location.

[0162] 2) Master node selection: Select a suitable master node from the consensus nodes.

[0163] 3) This invention: It represents the proposed PPO-based joint task offloading and consensus master node selection algorithm.

[0164] exist Figure 3 The figure presents the convergence performance of the proposed scheme and the scheme based on the random unloading algorithm. As can be seen from the figure, the scheme based on the random unloading algorithm achieves basic convergence on the 230th iteration, with a reward of 1.6 × 10⁻⁶. 4 The time fluctuations are minimal, but the scheme converges around 150 times, with a reward of 4.2 × 10⁻⁶. 4 It fluctuates left and right. Therefore, compared with algorithms based on random unloading, this scheme has a faster learning speed, better convergence, and better learning efficiency.

[0165] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing, characterized in that, The policy actor network and value critic network applied to the agent employ deep reinforcement learning (DRL) methods for decision-making, including: Initialize the parameters of the actor and critic networks, and set the training-related hyperparameters; the agent interacts with the environment based on the current policy, executes actions, and performs state transitions; the entire segment in the environment is sampled using the parameters of the sampling actor network, and the trajectory is stored in memory; the discounted reward, advantage function, and objective function are calculated; the parameters of the actor and critic networks, as well as the parameters of the sampling actor and sampling critic networks, are updated; training is repeated until the policy converges, and the trained policy is used to compute unloading and select the master node; The method specifically includes the following steps: S101. Initialize the parameters θ and ω of the actor network and critic network of the agent, the parameters θ′ and ω′ of the sampling actor network and the sampling critic network, the number of episodes, the learning rates μ and σ of the actor network and critic network, the discount factor γ, and the experience pool; initialize the network layout parameters, including the data size D of the input task. i (t), workload C i (t); S102. Initialize the state of the agent, the agent interacts with the environment, and the policy network generates actions based on the current policy; S103. The agent executes the generated action, receives an immediate reward based on the executed action, and transfers the environmental state to the next state. S104. Calculate the discounted reward; S105. Calculate the advantage function, add the pruning factor to constrain the update rate, and calculate the objective function at the same time. S106. Update the parameters of the actor network and the critic network; S107. Update the parameters of the sampling actor network and the sampling critic network based on the updated parameters of the actor network and the critic network. S108. Repeatedly iterate the training, select the optimal action according to each state to obtain the maximum benefit, and finally obtain the optimal master node selection and calculation unloading strategy.

2. The vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing as described in claim 1, characterized in that, In step S102, the agent interacts with the environment; at the beginning of each round, the system state s(t) is initialized, as follows: Among them, the amount of input data for the task workload of the task The link availability between moving vehicles (MVs) and parked vehicles (PVs) is [missing information]. The link connection rate between MVs and PVs is Computing resources allocated to each task Indicators of whether node k exhibits malicious behavior 3. The vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing as described in claim 1, characterized in that, S103: The agent executes the generated action, receives an immediate reward based on the executed action, and transitions the environment state to the next state. The reward calculation formula is as follows: In the above formula, Object(t) represents the objective function.

4. The vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing as described in claim 1, characterized in that, S105: Calculate the dominance function: At this point: δ t =r(t)+γV(s t+1 ;w)-V(s t ;w). Proximity Policy Optimization (PPO) introduces a J-based approach. θ′ Further improvement of the actor objective function (θ); by adding a pruning factor to constrain the update rate, and by maximizing the objective function to update the PPO actor, the formula is as follows: The clip function will (π(a) t |s t ;θ)) / π(a t |s t The value of θ′ is restricted to the range of [-1, +1] to ensure that the two distributions remain relatively close after minimizing the clip function.

5. The vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing as described in claim 1, characterized in that, S106: Update the parameters of the actor network and the critic network; the parameters of the actor network are updated by the following formula: Considering the mean squared error function of the value estimation, the loss function of the critic network is given: L critic (w)=[V(s t+1 ;w)-V(s t ;w)] 2 , Update using this formula: Where, δ t This represents the time difference (TD) error.

6. A vehicle-to-everything (V2X) optimization system based on blockchain, digital twins, and edge computing as described in claim 1, characterized in that, include: The system initialization module initializes all necessary parameters, including initializing the experience memory, actor network parameters θ and critic network parameters w, sampling actor network parameters θ' and sampling critic network parameters w'; The configuration module is used to set parameters for specific applications, including the data size of the input task; The agent module is used to generate actions based on the current network state at the beginning of each cycle; Used to collect data samples, calculate the advantage function, and update the policy network and value network during the policy evaluation process; The action execution module is used to perform task unloading and master node selection during the consensus process; The reward acquisition module is used to execute actions and calculate immediate rewards. This reward acquisition module is designed based on whether the system's constraints are met. If all constraints are met, a reward is obtained; otherwise, a penalty is imposed. The state transition module is used to transition the system state from the current state to the next state. The experience replay module is used to store the experience tuples for each system state, action, reward, and next state; The data sampling module is used to extract a certain segment from the experience stored in the internal storage for learning. The network update module is used to update the actor network and critic network based on data from the experience replay module. The parameter update module is used for updating the parameters of the actor network and the critic network, as well as the parameters of the sampling actor network and the sampling critic network.

7. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing as described in claim 1.

8. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the vehicle-to-everything (V2X) optimization method based on blockchain, digital twins, and edge computing as described in claim 1.

Images