Internet of vehicles optimization method based on block chain, digital twinning and edge computing

Abstract

The invention belongs to the technical field of wireless communication, and discloses an Internet of Vehicles (IoT) optimization method based on a block chain, digital twins (DT) and edge computing (MEC), a binary task unloading decision is jointly optimized, an improved practical Byzantine fault tolerance (EPBFT) consensus algorithm is also disclosed in a block chain system to enhance a consensus program, and the Byzantine fault tolerance (EPBFT) consensus algorithm is applied to the block chain system, so that the Byzantine fault tolerance (EPBFT) and the Byzantine fault tolerance (EPBFT) of the Byzantine fault tolerance (EPBFT) of the Byzantine fault tolerance (EPBFT) of the Byzantine fault tolerance (EPBFT) of the Byzantine fault tolerance (EPBFT) is improved. And then consensus main nodes are selected, so that consensus delay and failure rate are reduced to the greatest extent. The system is subjected to joint optimization under the influence of factors such as edge node computing power, EPBFT node number limitation, task processing and block chain consensus delay. According to the invention, a base station (BSS) not only serves as an edge node in an MEC system, but also serves as a consensus node of a block consensus in a block chain system. The method comprises the following steps of: converting a task into a Markov decision process due to the uncertainty of the task and the dynamic nature of the network, and dynamically learning an optimal joint solution by using near-end policy optimization (PPO).

Description

Technical Field

[0001] The present invention belongs to the field of wireless communication technology and relates to a vehicle network optimization method based on blockchain, digital twins and edge computing. Background Art

[0002] At present, the popularity of IoV has affected our daily life, brought great convenience, and attracted the attention of many researchers. Due to the large amount of data, vehicles are often unable to process computationally intensive or delay-sensitive computing tasks in a short time. Therefore, MEC technology has been proposed and widely used in vehicle networking scenarios. By offloading computing tasks from requesting vehicles to edge computing servers, network latency is greatly reduced, energy consumption is reduced, computing efficiency is improved, and the performance of IoV systems is improved. At the same time, as the scale of the network expands, the complexity of the current network is also increasing. In order to improve the autonomy of the network and reduce the negative impact on the physical network. DT technology is introduced, which is an innovative technology that optimizes and verifies policies through real-time interactive physical entity interaction mapping. In addition, since the infrastructure is vulnerable to threat attacks and control, information data can be easily tampered by attackers during the offloading process. Therefore, how to ensure the security of mobile offloaded data is very important. Integrating blockchain, DT, and MEC networks in a unified system can help create a task processing service environment with trustworthy, transparent, secure, immutable, and automated characteristics.

[0003] However, there are still some challenges and problems when applying deep reinforcement learning (DRL) to mobile edge computing (MEC) systems based on improved practical Byzantine fault tolerance (EPBFT) consensus:

[0004] 1) Network architecture integration issues

[0005] First, it is necessary to design a multi-system architecture, including edge computing systems and blockchain systems, to ensure that data and tasks between systems can be efficiently transmitted and processed. Secondly, the integration of consensus mechanisms and computing resources is also a key issue. The Practical Byzantine Fault Tolerance (PBFT) consensus algorithm requires frequent communication and information exchange between nodes, which consumes a lot of network bandwidth and computing resources. Therefore, it is necessary to optimize the consensus algorithm to reduce resource consumption and improve efficiency.

[0006] In particular, PBFT faces the following challenges: (1) Due to the single point full node and two full node broadcasts, PBFT brings high bandwidth overhead, limiting its application in large-scale networks; (2) There is a risk of selecting malicious or faulty nodes when arbitrarily selecting the master node, causing consensus failure. Although changing the protocol solves this problem, its high communication complexity will lead to resource waste, extended latency, and reduced system stability.

[0007] 2) Real-time and latency requirements

[0008] In the MEC system based on EPBFT consensus, real-time and latency requirements are crucial. MEC systems are designed to provide low-latency, high-real-time services, so they have extremely high requirements for the speed of data processing and task response. However, the EPBFT consensus mechanism involves complex inter-node communication and multiple message delivery, which increases network latency. This poses severe challenges to application scenarios that require instant feedback and processing, such as autonomous driving, real-time monitoring, and augmented reality. Therefore, optimizing the consensus algorithm to reduce communication overhead, improve the efficiency of consensus, and design efficient data transmission and processing mechanisms are the key to meeting real-time and latency requirements. Only under the premise of ensuring real-time response and minimal latency of the system can the advantages of MEC systems in various applications be fully utilized.

[0009] 3) Trust and security issues

[0010] Due to the distributed and heterogeneous nature of the MEC system, a reliable trust relationship needs to be established between nodes to ensure the security of data transmission and processing. The EPBFT consensus mechanism improves the reliability of the system through fault-tolerant design, but it also faces the risk of selecting malicious nodes or faulty nodes, which may lead to consensus failure or system attacks. To this end, it is necessary to combine encryption technology, access control, and node reputation evaluation mechanisms to ensure that only trusted nodes can participate in the consensus process. At the same time, efficient security protocols are designed to prevent data tampering, unauthorized access, and network attacks to ensure the overall security and stability of the system. Only when trust and security issues are effectively resolved can the MEC system provide reliable computing and services to meet the high security requirements of users and applications.

[0011] Through the above analysis, the problems and defects of the prior art are as follows:

[0012] (1) Insufficient network architecture integration: Existing technologies make it difficult to efficiently integrate MEC systems and blockchain systems, resulting in inefficient transmission and processing of data and tasks.

[0013] (2) Real-time and latency requirements are difficult to meet: The EPBFT consensus mechanism involves complex inter-node communications and multiple message transmissions, which results in increased network latency in the system in application scenarios that require low latency and high real-time performance (such as autonomous driving and real-time monitoring). This makes it difficult to meet real-time requirements and affects the response speed and overall performance of the MEC system.

[0014] (3) Trust and security issues: Although the EPBFT consensus mechanism improves the reliability of the system through fault-tolerant design, the risk of selecting malicious or faulty nodes still exists in the dynamic and distributed MEC system, which may lead to consensus failure or system attacks. Existing technologies have not yet fully solved the problem of how to effectively evaluate node reputation and establish a reliable trust mechanism. Summary of the invention

[0015] In response to the problems existing in the prior art, the present invention provides a vehicle networking optimization method based on blockchain, digital twins and edge computing.

[0016] The present invention is implemented as follows: a vehicle networking optimization method based on blockchain, digital twins and edge computing, wherein the strategy actor and critic network of the application agent adopts a deep reinforcement learning (DRL) method for decision making, including:

[0017] Initialize the parameters of the policy actor network and the critic network, and set the training-related hyperparameters; the agent interacts with the environment based on the current strategy, performs actions and performs state transitions; samples the entire segment in the environment using the parameters of the sampling policy actor network, and stores the trajectory in memory; calculates the discounted reward, advantage function, and objective function; updates the parameters of the policy actor network and the critic network as well as the sampling policy actor and critic network parameters; repeats the training until the strategy converges, and uses the trained strategy for computational offloading and master node selection.

[0018] Further, the following steps are included:

[0019] S101, initialize the parameters θ and ω of the agent's strategy actor network and critic network, sample the parameters θ′ and ω′ of the strategy actor network and critic network, the number of episodes, initialize the learning rates μ and σ corresponding to the strategy actor network and critic network, the discount factor γ, initialize the experience pool; initialize the network layout parameters, such as the data size D of the input task i (t), task workload C i (t) and other parameters;

[0020] S102, initializing the state of the agent, the agent interacts with the environment, and the policy network generates actions based on the current policy;

[0021] S103, the agent executes the action generated by the actor, obtains the reward, and updates the state;

[0022] S104, the agent executes the generated action, obtains an immediate reward according to the executed action, and transfers the environment state to the next state;

[0023] S105, calculating the discount reward;

[0024] S106, calculating the advantage function, adding the clipping factor to constrain the update rate, and calculating the objective function at the same time;

[0025] S107, update the parameters of the strategy actor and critic network;

[0026] S108, updating the sampling strategy actor and critic network parameters according to the updated strategy actor and critic network parameters;

[0027] S109, repeat iterative training, select the optimal action according to each state, obtain the maximum benefit, and finally obtain the optimal master node selection and calculation offloading strategy.

[0028] Furthermore, in S102, the agent interacts with the environment; at the beginning of each round, the system state s(t) is initialized, as follows:

[0029]

[0030] Among them, the input data volume of the task The workload of the task The link availability between moving vehicles (MVs) and parked vehicles (PVs) is The link connection rate between MVs and PVs is Computing resources allocated to each task Indicator of whether node k has malicious behavior

[0031] Furthermore, the calculation consensus of the reward in S103: the agent executes the generated action, obtains an immediate reward according to the executed action, and transfers the environment state to the next state is as follows:

[0032]

[0033] In the above formula, Object(t) represents our objective function.

[0034] Further, the S106: calculates the advantage function:

[0035]

[0036] At this time there are:

[0037] δ t =r(t)+γV(s t+1 ; w)-V(s t ;w).

[0038] Proximal Policy Optimization (PPO) introduces a J-based θ′ Further improvement of the actor objective function of (θ); by adding a clipping factor to constrain the update rate, the PPO actor is updated by maximizing the objective function, the formula is:

[0039]

[0040] Where ∈ is a hyperparameter, the clip function converts (π(a t ∣s t ;θ)) / π(a t ∣s t ; θ′) is restricted to the range of [1-,1+], ensuring that the two distributions remain relatively close after minimizing the clip function.

[0041] Further, the S107: update the parameters of the strategy actor and the critic network; the parameters of the actor are updated by the following formula:

[0042]

[0043] Considering the mean square error function of the value estimation, the loss function of the critic network is given: L critic (w)=[V(s t+1 ; w)-V(s t ;w)] 2 ,

[0044] Updated by this formula:

[0045]

[0046] Among them, δ t Indicates TD error.

[0047] Another object of the present invention is to provide a vehicle networking optimization system based on blockchain, digital twins and edge computing, characterized in that it includes:

[0048] System initialization module, which initializes all necessary parameters, including initializing experience memory, policy actor network parameters θ and critic network parameters w, sampling policy actor parameters θ' and critic network parameters w';

[0049] Configuration module, used to set application-specific parameters, including task parameters such as the data size of the input task;

[0050] The agent module is used to generate actions based on the current network state at the beginning of each cycle; it is used to collect data samples, calculate the advantage function, and update the policy and value network during the policy evaluation process;

[0051] Action execution module, used to perform task offloading and master node selection during consensus process

[0052] The reward acquisition module is used to execute actions and calculate instant rewards. The reward acquisition module is designed based on whether the constraints of the system are met. If all constraints are met, rewards are obtained, otherwise penalties are obtained;

[0053] The state transfer module is used to transfer the system state from the current state to the next state;

[0054] The experience replay module is used to store the experience tuples of each system state, action, reward, and next state;

[0055] The data sampling module is used to extract certain fragments from the stored experience for learning;

[0056] The network update module is used to update the actor network and the critic network based on the data of the experience playback module;

[0057] Parameter update module, used for parameter update of policy actor network and critic network, as well as parameter update of sampling policy actor and critic network.

[0058] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of a vehicle network optimization method based on blockchain, digital twins, and edge computing.

[0059] Another object of the present invention is to provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of a vehicle network optimization method based on blockchain, digital twins and edge computing.

[0060] First, in view of the technical problems existing in the above-mentioned prior art and the difficulty of solving the problems, the technical solutions to be protected by the present invention and the results and data during the research and development process are closely combined to analyze in detail and deeply how the technical solutions of the present invention solve the technical problems, and some creative technical effects brought about after solving the problems. The specific description is as follows:

[0061] The first is efficiency and security. The present invention considers a method for optimizing the Internet of Vehicles based on blockchain, TD and MEC, which includes two systems: a MEC system and a blockchain system. The MEC system is designed for task execution, while the blockchain system plays a vital role in establishing a secure and reliable transaction platform for the MEC system. In the MEC system, PVs form a parking cluster, acting as a virtual edge server, auxiliary MVs, base stations and blockchain systems. BSs themselves not only have available computing resources, but also can act as relay nodes to realize information interaction between MVs and PVs within the communication coverage of auxiliary devices with different computing resources. Each PV is equipped with a MEC server to provide specific computing resources for its associated MVs. In addition to being an edge node for task processing in the MEC system, each PS also serves as a consensus node for block consensus, with dual functions.

[0062] Secondly, considering the dynamics of the task, the volatility of network conditions and the random behavior of nodes in consensus, we use PPO to learn the environment state and obtain the optimal joint decision-making strategy for offloading and master node selection.

[0063] Second, considering the technical solution as a whole or from the perspective of the product, the technical effects and advantages of the technical solution to be protected by the present invention are described in detail as follows:

[0064] 1) Accurate task offloading

[0065] This method significantly improves the resource utilization and system performance of edge computing by introducing precise task offloading strategies. The MEC system provides computing power for user devices to process tasks, while the consortium blockchain system provides trust and security for user devices during the task offloading process.

[0066] 2) Efficient consensus performance

[0067] The improved practical Byzantine fault tolerance algorithm (EPBFT) is adopted. This integrated EPBFT method takes into account both scalability and complexity, solves the limitations of the PBFT algorithm, and significantly improves consensus performance.

[0068] 4) Minimize consensus delays and failure rates

[0069] Consensus delay refers to the time required to complete consensus, and consensus failure may be caused by many reasons, such as unsatisfied constraints, malicious master nodes, etc. This method minimizes consensus delay and failure rate through the selection of master nodes, etc., to improve the stability and efficiency of the system.

[0070] 5) Integration of Deep Reinforcement Learning

[0071] Combining DRL with the MEC system enables the system to self-learn and adjust based on real-time data. By adopting the PPO algorithm, the system can still make the best decision in the absence of clear instructions.

[0072] 6) Improvement of system stability and reliability

[0073] By accurately calculating and optimizing the immediate rewards and state transitions after action execution, this method significantly enhances the stability and reliability of the system, especially when processing large amounts of data and high concurrent requests.

[0074] 7) The network’s autonomous learning and optimization capabilities

[0075] This method enables the system to continuously optimize its decision-making process through continuous iterative training and experience-based network updates, thereby improving overall performance.

[0076] 8) Joint Optimization Problem

[0077] This method jointly optimizes the task offloading decision and the selection of the master node to improve the overall performance and efficiency of the system. This comprehensive optimization strategy not only improves resource utilization, but also enhances the system's adaptability and stability in complex network environments.

[0078] 9) Complexity and dynamism

[0079] The complexity of the network and the dynamic nature of the environment make it difficult for traditional optimization methods to effectively deal with these challenges. To address this problem, this method uses the PPO algorithm to improve the adaptability and performance of the system when facing complex and dynamic environments.

[0080] 10) Real-time and efficiency

[0081] This method can improve the real-time response speed and resource utilization efficiency of the system when processing tasks.

[0082] Third, the core of the vehicle networking optimization method based on blockchain, TD and MEC provided by the present invention is to use mathematical models to guide the behavior and learning process of the system. The technical effects brought by these mathematical models can be discussed based on their characteristics:

[0083] 1) Even if the reward is calculated

[0084] Even the computational consensus of rewards focuses on the overall network cost, consensus delays and long failure rates.

[0085] Cost savings: By directly linking rewards to costs, this approach encourages reducing costs across the network and effectively improves resource utilization efficiency.

[0086] 2) Update of the strategy master actor and critic network

[0087] Update the current policy network by randomly sampling small batches of experience data and using the gradient ascent method.

[0088] Policy optimization: By continuously adjusting the policy network parameters, the system can learn and adopt more effective decision-making strategies.

[0089] Improved responsiveness: Using small batches of data enables the network to quickly adapt to environmental changes, enhancing the system's dynamic adjustment capabilities.

[0090] 3) Parameter update formula

[0091] The method of updating the target network parameters by the strategy actor and critic network parameters is described, involving the current network and the target network parameters.

[0092] Policy gradually approaches: By introducing the clipping function, the system can smoothly transition to the new policy and prevent performance fluctuations due to drastic changes.

[0093] Continuous learning and adaptation: This continuous parameter update mechanism ensures that the system can adapt to long-term environmental changes.

[0094] Fourth, as auxiliary evidence of the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0095] The expected benefits and commercial value of the technical solution of the present invention after transformation are:

[0096] The present invention realizes the optimization method of Internet of Vehicles based on blockchain, TD and MEC by introducing the PPO algorithm, which significantly improves the network performance and service quality. This can not only meet the growing demand for high-performance computing, but also optimize resource utilization and reduce network costs. In terms of commercial applications, the present invention can be widely used in application scenarios that require efficient processing of large amounts of data and ensure data security and trustworthiness, bringing significant economic and social benefits to related industries. Therefore, the present invention has broad market prospects and huge commercial value. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] Figure 1 It is a flowchart of the vehicle networking optimization method based on blockchain, TD and MEC driven by MEC;

[0098] Figure 2 It is a flowchart of the implementation of the Internet of Vehicles optimization method based on blockchain, TD and MEC driven by blockchain;

[0099] Figure 3 It is a comprehensive convergence performance comparison diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0100] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0101] Example 1

[0102] Aiming at the problems existing in the existing technology, this implementation provides a joint optimization method for MEC network based on enhanced practical Byzantine fault tolerance (EPBFT) consensus. First, it includes two systems: multi-access edge computing system and blockchain system. MEC system is designed for task execution, while blockchain system plays a vital role in establishing a secure and reliable transaction platform for MEC system.

[0103] In the considered network architecture, there are I mobile vehicles (MVs), stationary vehicles (PVs) and Base stations (BSs). The corresponding set of MVs, PVs and BSs is represented by I = {1,...i,...Ι}, and respectively. PVs form a parking cluster, acting as virtual edge servers, auxiliary MVs, base stations, and blockchain systems. BSs themselves not only have available computing resources, but also can act as relay nodes to realize information interaction between MVs and PVs within the communication coverage of auxiliary devices with different computing resources. Each PV is equipped with a MEC server to provide specific computing resources to its associated MVs. In addition to being an edge node for task processing in the MEC system, each PS also serves as a consensus node for block consensus in the blockchain system, with dual functions. Our system operates in discrete time slots, dividing the entire time range into T intervals, with The duration of each time slot is represented by △t.

[0104] Example 2

[0105] Considering the dynamic characteristics of the network and the uncertainty of information acquisition, this makes the problem quite complicated. In order to effectively solve this challenge, we remodel the problem as a Markov decision process (MDP). Since the variables involved are non-continuous, we choose to use the PPO algorithm, which is a DRL algorithm designed to solve reinforcement learning problems in continuous action spaces and can support real-time online decision making.

[0106] like Figure 1 As shown, the embodiment of the present invention provides a flow chart of a vehicle networking optimization method based on blockchain, TD and MEC, which specifically includes the following steps:

[0107] S101, initialize the parameters θ and ω of the agent's policy actor network and critic network, sample the parameters θ′ and ω′ of the policy actor network and critic network, the number of episodes, initialize the learning rates μ and σ corresponding to the critic network and policy network, the discount factor γ, initialize the experience pool; initialize the network layout parameters, such as the data size D of the input task i (t), task workload C i (t) and other parameters;

[0108] S102, initializing the state of the agent, the agent interacts with the environment, and the main strategy network generates actions based on the current strategy;

[0109] S103, the agent executes the generated action, obtains an immediate reward according to the executed action, and transfers the environment state to the next state;

[0110] S104, sampling the entire segment in the environment according to the parameters of the sampling strategy actor network, and storing the trajectory in the memory;

[0111] S105, calculating the discount reward;

[0112] S106, calculating the advantage function, adding the clipping factor to constrain the update rate, and calculating the objective function at the same time;

[0113] S107, update the parameters of the strategy actor and critic network;

[0114] S108, updating the sampling strategy actor and critic network parameters according to the updated strategy actor and critic network parameters;

[0115] S109, repeat iterative training, select the optimal action according to each state, obtain the maximum benefit, and finally obtain the optimal computing resource allocation and computing offloading strategy.

[0116] Furthermore, in S102, the agent interacts with the environment; at the beginning of each round, the system state s(t) is initialized. The details are as follows:

[0117]

[0118] Among them, the input data volume of the task The workload of the task The link availability between moving vehicles (MVs) and parked vehicles (PVs) is The link connection rate between MVs and PVs is Computing resources allocated to each task Indicator of whether node k has malicious behavior

[0119] Furthermore, the consensus for calculating the reward in S103: the agent executes the generated action, obtains an immediate reward according to the executed action, and transfers the environment state to the next state is as follows:

[0120]

[0121] In the above formula, Object(t) represents our objective function.

[0122] Furthermore, the S106: calculates the advantage function:

[0123]

[0124] At this time there are:

[0125] δ t =r(t)+γV(s t+1 ; w)-V(s t ;w).

[0126] In order to improve the performance, PPO introduces a J-based θ′ Further improvement of the actor objective function of (θ). By adding a clipping factor to constrain the update rate, the PPO actor can be updated by maximizing the objective function, as follows:

[0127]

[0128] Where ∈ is a hyperparameter, the clip function converts (π(a t ∣s t ;θ)) / π(a t ∣s t; θ′) is restricted to the range [1-,1+]. This approach ensures that after minimizing the clip function, the two distributions remain relatively close and avoid significant differences.

[0129] Furthermore, the S107: updating the parameters of the strategy actor and the critic network; the parameters of the actor are updated by the following formula:

[0130]

[0131] This algorithm considers the mean square error function of the value estimation and gives the loss function of the critic network:

[0132] L critic (w)=[V(s t+1 ; w)-V(s t ;w)] 2 ,

[0133] At the same time, it can be updated by this formula:

[0134]

[0135] Among them, δ t Indicates TD error.

[0136] In order to elaborate on the Internet of Vehicles optimization method based on blockchain, TD and MEC, the present invention provides two specific application embodiments, including key details of the implementation scheme.

[0137] Application Example 1: Smart City Monitoring System

[0138] 1) System initialization

[0139] Deploy the main actor and critic network in the smart city monitoring center. Initialize the network parameters, including the number of rounds, training steps, learning rate, etc. Deploy multiple IoT devices (such as surveillance cameras, sensors) and base stations to collect and transmit data.

[0140] 2) Agent-environment interaction

[0141] Each IoT device acts as an intelligent agent and generates actions based on the state of its monitored environment (such as adjusting the sensor's acquisition frequency, the edge node's data processing priority, or the frequency of uploading data, etc.). The generation of actions is based on the current policy.

[0142] 3) Action execution and reward acquisition

[0143] After performing an action, the agent receives an immediate reward based on the effect of the action (e.g., data transfer rate, data accuracy). The system then moves to the next state, which may be due to changes in the environment or changes in the data stream at different time periods.

[0144] 4) Optimization and update

[0145] Based on the collected data and rewards, the policy actor and critic network are updated to optimize the control strategy of the sensor and the data processing priority of the edge node. The resource allocation, such as data transmission rate and allocation of computing resources, is gradually improved using reinforcement learning algorithms.

[0146] 5) Latency and performance optimization

[0147] The performance of the PPO algorithm is evaluated through simulation experiments or field tests. Key indicators such as the coverage, signal quality, and data transmission rate of the communication network are analyzed and compared with traditional methods. According to the evaluation results, the algorithm parameters are adjusted and optimized to further improve the performance and stability of the communication network. Through the above steps, the Internet of Vehicles optimization method based on blockchain, TD and MEC proposed in the present invention can monitor a large number of IoT devices in real time in smart cities, such as smart lighting control, environmental monitoring, etc.

[0148] Application Example 2: Government Service System

[0149] 1) System initialization

[0150] In the government service system, the center deploys the PPO network and initializes the network parameters, and also deploys the necessary infrastructure such as servers and databases. These facilities not only support the real-time processing and data storage of payment transactions, but also ensure that the system can operate safely and efficiently, meeting the security, stability and performance requirements of government services.

[0151] 2) Agent-environment interaction

[0152] Each server acts as an intelligent agent, continuously interacting with the environment and generating corresponding actions based on the current state of the environment. These actions may include resource allocation, data processing optimization, and security control to ensure the stability and performance optimization of financial and payment systems.

[0153] 3) Action execution and reward acquisition

[0154] After executing the current action, immediate rewards are obtained based on key indicators such as rationality of resource allocation, data transmission rate, and security. The system then moves to the next state based on changes in the environment or time periods.

[0155] 4) Optimization and update

[0156] The collected data and rewards are used to update the parameters of the PPO network. Through continuous iteration and training, the algorithm gradually improves the stability and efficiency of the communication system, thereby ensuring the security and reliability of the government service system.

[0157] 5) Reliability of government service systems

[0158] Through the PPO algorithm of the present invention, intelligent management and optimization of the government service system are realized. The algorithm focuses on improving the reliability and stability of the communication link to ensure the security and integrity of data during transmission. In addition, the PPO algorithm also optimizes the allocation and utilization efficiency of system resources, significantly improves the response speed and processing capacity of the overall system, thereby enhancing the user experience and system reliability. Through the above steps, the present invention provides an efficient and intelligent solution for the government service system security.

[0159] In these two embodiments, the Internet of Vehicles optimization method based on blockchain, digital twins, and edge computing provides an efficient, reliable, and energy-saving solution suitable for different application scenarios, from smart cities to government service systems, demonstrating its wide application potential and technical advantages.

[0160] In order to comprehensively evaluate the performance of the present invention, we compared it with several representative benchmark algorithms. These benchmark algorithms have their own characteristics and advantages and can effectively handle similar problems. By comparing the performance of these algorithms under the same conditions, we can more clearly understand the advantages of the present invention and the potential room for improvement.

[0161] 1) Random offloading: The offloading decision of each MV is determined randomly. Each MV's task can choose to be processed independently on itself or on the BS. However, a task can only be processed at one location.

[0162] 2) Masternode selection: Select a suitable masternode from the consensus nodes.

[0163] 3) The present invention: It represents the proposed PPO-based joint task offloading and consensus master node selection algorithm.

[0164] exist Figure 3 In the figure, the convergence performance of the scheme and the random unloading algorithm is given. As can be seen from the figure, the scheme based on the random unloading algorithm reaches basic convergence at the 230th iteration, and its reward is 1.6×10 4 The scheme converges after about 150 times, and its reward is 4.2×10 4 Therefore, compared with the algorithm based on random unloading algorithm, this scheme has faster learning speed, better convergence and learning efficiency.

[0165] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A vehicle networking optimization method based on blockchain, digital twins and edge computing, characterized in that: The strategy actor and critic network of the application agent adopts the deep reinforcement learning (DRL) method to make decisions, including: Initialize the parameters of the policy actor network and the critic network, and set the training-related hyperparameters; the agent interacts with the environment based on the current strategy, performs actions and performs state transitions; samples the entire segment in the environment using the parameters of the sampling policy actor network, and stores the trajectory in memory; calculates the discounted reward, advantage function, and objective function; updates the parameters of the policy actor network and the critic network as well as the sampling policy actor and critic network parameters; repeats the training until the strategy converges, and uses the trained strategy for computational offloading and master node selection.

2. The vehicle networking optimization method based on blockchain, digital twins and edge computing as claimed in claim 1, characterized in that: The steps include: S101, initialize the parameters θ and ω of the agent's strategy actor network and critic network, sample the parameters θ′ and ω′ of the strategy actor network and critic network, the number of episodes, initialize the learning rates μ and σ corresponding to the strategy actor network and critic network, the discount factor γ, initialize the experience pool; initialize the network layout parameters, such as the data size D of the input task i (t), task workload C i (t) and other parameters; S102, initializing the state of the agent, the agent interacts with the environment, and the policy network generates actions based on the current policy; S103, the agent executes the action generated by the actor, obtains the reward, and updates the state; S104, the agent executes the generated action, obtains an immediate reward according to the executed action, and transfers the environment state to the next state; S105, calculating the discount reward; S106, calculating the advantage function, adding the clipping factor to constrain the update rate, and calculating the objective function at the same time; S107, update the parameters of the strategy actor and critic network; S108, updating the sampling strategy actor and critic network parameters according to the updated strategy actor and critic network parameters; S109, repeat iterative training, select the best action according to each state, obtain the maximum benefit, and finally obtain the optimal master node selection and calculation offloading strategy.

3. The vehicle networking optimization solution based on blockchain, digital twin and edge computing as claimed in claim 1 is characterized in that: In S102, the agent interacts with the environment; at the beginning of each round, the system state s(t) is initialized, as follows: Among them, the input data volume of the task The workload of the task The link availability between moving vehicles (MVs) and parked vehicles (PVs) is The link connection rate between MVs and PVs is Computing resources allocated to each task Indicator of whether node k has malicious behavior 4. The vehicle networking optimization method based on blockchain, digital twins and edge computing as claimed in claim 1, characterized in that: S103: The agent executes the generated action, obtains an immediate reward according to the executed action, and transfers the environment state to the next state. The calculation consensus of the reward is as follows: In the above formula, Object(t) represents our objective function.

5. The vehicle networking optimization method based on blockchain, digital twins and edge computing as claimed in claim 1, characterized in that: S106: Calculate the advantage function: At this time there are: δ t =r(t)+γV(s t+1 ;w)-V(s t ;w). Proximal policy optimization (PPO) introduces a J-based θ′ Further improvement of the actor objective function of (θ); by adding a clipping factor to constrain the update rate, the PPO actor is updated by maximizing the objective function, the formula is: Where ∈ is a hyperparameter, the clip function converts (π(a t ∣s t ;θ)) / π(a t ∣s t ; θ′) is restricted to the range of [1-,1+], ensuring that the two distributions remain relatively close after minimizing the clip function.

6. The vehicle networking optimization method based on blockchain, digital twins and edge computing as claimed in claim 1, characterized in that: S107: Update the parameters of the strategy actor and critic network; the parameters of the actor are updated by the following formula: Considering the mean square error function of the value estimation, the loss function of the critic network is given: L critic (w)=[V(s t+1 ; w)-V(s t ;w)] 2 , Updated by this formula: Among them, δ t Indicates TD error.

7. A vehicle networking optimization system based on blockchain, digital twins and edge computing according to the method of claim 1, characterized in that: include: System initialization module, which initializes all necessary parameters, including initializing experience memory, policy actor network parameters θ and critic network parameters w, sampling policy actor parameters θ' and critic network parameters w'; Configuration module, used to set application-specific parameters, including task parameters such as the data size of the input task; The agent module is used to generate actions based on the current network state at the beginning of each cycle; Used to collect data samples, calculate advantage functions, and update policies and value networks during policy evaluation; Action execution module, used to perform task offloading and master node selection during consensus process The reward acquisition module is used to execute actions and calculate instant rewards. The reward acquisition module is designed based on whether the constraints of the system are met. If all constraints are met, rewards are obtained, otherwise penalties are obtained; The state transfer module is used to transfer the system state from the current state to the next state; The experience replay module is used to store the experience tuples of each system state, action, reward, and next state; The data sampling module is used to extract certain fragments from the stored experience for learning; The network update module is used to update the actor network and the critic network based on the data of the experience playback module; Parameter update module, used for parameter update of policy actor network and critic network, as well as parameter update of sampling policy actor and critic network.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the Internet of Vehicles optimization method based on blockchain, digital twins and edge computing as claimed in claim 1.

9. A computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the vehicle network optimization method based on blockchain, digital twins and edge computing as claimed in claim 1.

Images