Industrial internet of things unmanned vehicle path planning system and method

Abstract

The application relates to the technical field of industrial Internet of Things, and discloses an industrial Internet of Things unmanned vehicle path planning system and method; the system comprises an environment perception layer, which is used for collecting environment state information of an unmanned vehicle and simultaneously receiving environment state information of other unmanned vehicles; a control instruction is received to control the movement of the unmanned vehicle; a decision control layer is used for selecting an action of the unmanned vehicle according to the environment state information of the unmanned vehicle by using a deep Q network, determining an optimal action by taking the environment state information of other unmanned vehicles as a safety constraint, generating a control instruction and delivering the control instruction to the environment perception layer; and uploading the environment state information of the unmanned vehicle, the optimal action and a reward to a blockchain; a blockchain network layer is used for uploading the environment state information of the unmanned vehicle, the optimal action and the reward as transaction information to a chain; and broadcasting the environment state information of other unmanned vehicles. The application can improve the credibility of the system and the path planning efficiency under a complex environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial internet of things, and in particular to an industrial internet of things unmanned vehicle path planning system and method. BACKGROUND

[0002] Under the background of Industry 4.0, industrial internet of things (IIoT) is widely used in manufacturing industry, and intelligent unmanned vehicles (such as automatic guided vehicles, AGV) are used to realize the automatic transportation of internal logistics in factories. In a complex dynamic industrial environment, efficient and reliable path planning is the key to the normal operation of unmanned vehicles. However, the traditional path planning and scheduling scheme faces two major difficulties in the IIoT environment: first, the path planning efficiency problem in a dynamic and complex environment. The traditional heuristic algorithm (such as A* algorithm) can find the shortest path in a static environment, but when there are dynamic obstacles or uncertainties in the environment, the pre-planned route may fail, and frequent re-planning is required, which reduces the efficiency. Second, the data security and trust problem. In the industrial unmanned vehicle system, a large amount of operation data (such as position, state, task information) is transmitted between devices and the cloud through a wireless network. The centralized data management method may be subject to malicious tampering or single point failure, and it is difficult to ensure the confidentiality and integrity of critical data (such as vehicle operation logs, task instructions). Therefore, how to improve the intelligence of path planning while ensuring the safety and reliability of data interaction in the IIoT system has become a research topic worthy of attention.

[0003] To solve the above problems, academia and industry have made many explorations. On the one hand, blockchain technology, as a decentralized distributed ledger, provides a new way for IIoT data security. Blockchain uses cryptographic chain structure and consensus mechanism to ensure that data cannot be tampered with once written, and improves system fault tolerance through distributed storage. Christidis and Devetsikiotis proposed to use smart contracts and blockchains for Internet of Things device management to achieve automatic interaction in a trustless environment. Khan et al. reviewed the security of the Internet of Things and pointed out that blockchain has great potential in ensuring the privacy and integrity of Internet of Things data. In the industrial environment, Zhang and Wen designed an "Internet of Things e-commerce model" to ensure the trustworthiness of transaction data between devices using blockchain. In addition, for the field of Internet of Vehicles and autonomous driving, Rathee et al. built a blockchain-based architecture to protect the data security of connected and autonomous vehicles; Okere and Balyan's research reviewed the application progress of blockchain in the Internet of Vehicles and looked forward to the prospect of combining machine learning with blockchain to improve vehicle communication security. On the other hand, deep reinforcement learning (DRL) has shown strong learning ability in robot path planning and control decision-making. Mnih et al. proposed the deep Q network (DQN) algorithm, which proved that agents can approximate the value function through deep neural networks in high-dimensional state space, and can achieve human-level decision-making. Subsequently, a large number of studies have applied deep reinforcement learning to autonomous driving and mobile robot path planning problems. Kiran et al. reviewed the application of deep reinforcement learning in autonomous driving and pointed out that DRL can help autonomous vehicles learn decision-making strategies in complex traffic environments. Recently, there have also been studies that use improved deep reinforcement learning algorithms for mobile robot obstacle avoidance and path planning, achieving higher success rate and robustness than traditional algorithms. For example, Escobar-Naranjo et al. proposed using DQN to optimize robot autonomous navigation in Applied Sciences, achieving improved path planning performance; Chen et al. applied improved DQN to indoor robot local obstacle avoidance path planning, improving efficiency and success rate in complex environments. Zhang et al. recently proposed a path planning method based on the soft actor-critic (SAC) algorithm, which performs well in dynamic narrow environments, demonstrating the application prospects of deep reinforcement learning in complex path planning.

[0004] In summary, blockchain has unique advantages in data security protection in IIoT unmanned vehicle systems, and deep reinforcement learning (especially DQN) has great potential in unmanned vehicle path planning optimization. However, the current research on the integration of the two is still in its infancy: few papers consider improving the intelligence of path planning and ensuring the trustworthiness of data links at the same time. SUMMARY

[0005] In view of the above problems in the prior art, the present application provides an industrial Internet of Things unmanned vehicle path planning system and method.

[0006] In order to achieve the above-mentioned purposes, the technical scheme adopted by the present application is as follows:

[0007] In the first aspect, the present application provides an industrial Internet of Things unmanned vehicle path planning system, which comprises:

[0008] An environment perception layer is configured to collect the environment state information of the unmanned vehicle and simultaneously receive the environment state information of other unmanned vehicles issued by a blockchain; the environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles are transmitted to a decision control layer, and the control instructions issued by the decision control layer are received to control the motion of the unmanned vehicle;

[0009] The decision control layer is configured to receive the environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles transmitted by the environment perception layer, select an action according to the environment state information of the unmanned vehicle by using a deep Q network, determine an optimal action by taking the environment state information of other unmanned vehicles as a safety constraint, generate a control instruction and issue it to the environment perception layer, and upload the environment state information of the unmanned vehicle, the optimal action and a reward to a blockchain;

[0010] The blockchain network layer is configured to receive the environment state information of the unmanned vehicle, the optimal action and the reward uploaded by the decision control layer, perform chain processing as transaction information, and broadcast the environment state information of other unmanned vehicles.

[0011] Further, the decision control layer takes the environment state information of other unmanned vehicles as a safety constraint, which is specifically as follows:

[0012] A safety evaluation function is established , wherein the safety evaluation function h takes a value of 1, indicating that the execution of the action may cause a constraint conflict, S is a state space, and A is an action space.

[0013] Further, the decision control layer takes the environment state information of other unmanned vehicles as a safety constraint to determine an optimal action, which is specifically as follows:

[0014]

[0015] , wherein is the optimal action, is the action at time t, is the action t executed at state s may cause a constraint conflict, is the obstacle avoidance action at state s t .

[0016] Further, the decision control layer obtains a reward after the environment perception layer performs the optimal action and observes a new state, adds a trajectory quadruple of the state, action, reward and new state to an experience replay pool, and periodically samples a small batch of samples in the experience replay pool for gradient descent.

[0017] Further, the decision control layer generates a trajectory quadruple in the digital twin environment and iterates until a termination condition is met.

[0018] Further, the environment perception layer perceives surrounding environment information through sensors and generates a state description by preprocessing raw sensor data by a vehicle-mounted controller, and simultaneously connects with an external positioning base station or a global positioning system to obtain the position of the unmanned vehicle in a global coordinate system.

[0019] Further, the blockchain network layer takes a number of unmanned vehicles as light nodes, submits data to adjacent edge blockchain nodes, and the edge blockchain nodes are responsible for participating in consensus and accounting; and predefines vehicle state on-chain contracts and task instruction publishing contracts in the alliance chain, sets data formats and access permissions, and verifies whether the data is legal when the unmanned vehicle sends a transaction to the blockchain using the corresponding contract and writes it into the ledger.

[0020] Further, it also includes an application display layer for receiving the unmanned vehicle operation records pushed by the blockchain network layer, detecting abnormal behavior of the unmanned vehicle according to the unmanned vehicle operation records, and displaying the unmanned vehicle state, path and alarm information.

[0021] Further, the application display layer displays the unmanned vehicle position, planned path and obstacle distribution in the virtual environment in real time; based on the blockchain data browsing request, it displays the vehicle operation log and transaction details recorded on the chain in real time, and when the blockchain detects an anomaly, it alarms in time through the interface.

[0022] In a second aspect, the present application provides an industrial Internet of Things unmanned vehicle path planning method, comprising the following steps:

[0023] The environment perception layer collects the environment state information of the unmanned vehicle, and receives the environment state information of other unmanned vehicles issued by the blockchain; and transmits the environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles to the decision control layer;

[0024] The decision control layer receives the environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles transmitted by the environment perception layer, uses a deep Q network to select actions according to the environment state information of the unmanned vehicle, determines the optimal action by taking the environment state information of other unmanned vehicles as a safety constraint, generates a control instruction and issues it to the environment perception layer; and uploads the environment state information of the unmanned vehicle, the optimal action and the reward to the blockchain;

[0025] The environment state information, the optimal action and the reward of the unmanned vehicle uploaded by the decision control layer are received by using the blockchain network layer as transaction information for on-chain processing, and the environment state information of other unmanned vehicles is broadcasted;

[0026] The motion control of the unmanned vehicle is performed by using the environment perception layer to receive the control instruction issued by the decision control layer.

[0027] The present application has the following beneficial effects:

[0028] The present application introduces the blockchain technology into the industrial unmanned vehicle scheduling system, realizes the safe storage and transparent sharing of vehicle state and decision data, and improves the system credibility; and constructs the DQN-based global path planning algorithm of the unmanned vehicle, which significantly improves the path planning efficiency in complex environment through reinforcement learning. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 It is an industrial Internet of Things unmanned vehicle path planning system framework diagram;

[0030] Figure 2 It is an industrial unmanned vehicle perception layer environment modeling and data uploading flowchart;

[0031] Figure 3 It is an industrial unmanned vehicle DQN intelligent decision and control execution flowchart;

[0032] Figure 4 It is an industrial Internet of Things unmanned vehicle on-chain decision and data synchronization flowchart;

[0033] Figure 5 It is an application display layer on-chain data display and interaction flowchart. DETAILED DESCRIPTION

[0034] The specific embodiments of the present application are described below to facilitate those skilled in the art to understand the present application, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all the inventions utilizing the concept of the present application are within the scope of protection.

[0035] As shown in Figure 1 The industrial Internet of Things unmanned vehicle path planning system provided by the embodiment of the present application comprises:

[0036] The environment perception layer is used for collecting the environment state information of the unmanned vehicle, receiving the environment state information of other unmanned vehicles issued by the blockchain, transmitting the environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles to the decision control layer, and receiving the control instruction issued by the decision control layer to perform motion control on the unmanned vehicle.

[0037] The decision control layer is used for receiving the environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles transmitted by the environment perception layer, selecting an action according to the environment state information of the unmanned vehicle by using a deep Q network, determining an optimal action by taking the environment state information of other unmanned vehicles as a safety constraint, generating a control instruction and issuing the control instruction to the environment perception layer, and uploading the environment state information of the unmanned vehicle, the optimal action and a reward to the blockchain;

[0038] The blockchain network layer is used for receiving the environment state information of the unmanned vehicle, the optimal action and the reward uploaded by the decision control layer, performing chain processing as transaction information, and broadcasting the environment state information of other unmanned vehicles.

[0039] In an optional embodiment of the present application, the environment perception layer includes an industrial unmanned vehicle body and various sensors, embedded controllers and the like carried by the industrial unmanned vehicle. The unmanned vehicle perceives surrounding environment information through sensors such as laser radar (LiDAR), ultrasonic wave, camera and the like, and the original sensing data is preprocessed by the vehicle-mounted controller to generate a state description. In this embodiment, a 10x10 grid is used to represent a local environment map of the unmanned vehicle, in which obstacles, target points and the position of the unmanned vehicle are marked, and which is used as the state input of reinforcement learning. At the same time, the environment perception layer is also connected with an external positioning base station or a global positioning system to obtain the position of the unmanned vehicle in a global coordinate system. Each unmanned vehicle node is pre-installed with a blockchain client, so that it can upload key information to the blockchain and receive control instructions from the chain or state data of other vehicles, such as Figure 2 As shown in the figure, the solid line represents the communication of the main data stream / writing operation, carries the payload and changes the system state, and the dashed line represents the query / reading operation / dependent call lightweight communication, which does not change the system state.

[0040] In an optional embodiment of the present application, the decision control layer is composed of an edge computing gateway or a local server and a reinforcement learning algorithm module, and is responsible for path planning decision and vehicle motion control. A deep Q network decision module is deployed on the edge computing device, which is used to calculate the optimal action according to the state provided by the environment perception layer. Specifically, the environment perception layer uploads the state to the decision module, and the DQN neural network outputs the Q value of each candidate action according to the input state, and the system selects the action with the maximum Q value . If - If the greedy policy requires exploration, a random action is selected with a small probability. After the action is selected, the speed and steering instructions are sent to the vehicle via the motion control unit (e.g., AGV controller) to execute. At the same time, the decision-making layer records the information related to the decision (state, decision, reward, etc.) to provide data for training. It is worth noting that at the initial stage of system deployment, the DQN model needs to be trained through a large number of simulations to achieve optimal performance before being used in actual operation. In actual operation, the DQN decision-making module can also continuously learn online - adding new interaction data collected in actual operation to the experience pool, constantly fine-tuning and optimizing the strategy, so as to adapt to environmental changes. The decision-making control layer also includes safety monitoring logic: when the path given by the DQN is obviously dangerous (e.g., a collision is predicted to occur), the safety rule module intervenes to correct or emergency stop to ensure system safety and stability, as shown in FIG. 8. Figure 3

[0041] Under the reinforcement learning framework, the vehicle path planning can be modeled as a Markov Decision Process (MDP). MDP is usually represented as a five-tuple , where is the state space, representing the set of possible states of the vehicle; is the action space, corresponding to the set of actions that the vehicle can perform (e.g., discrete actions such as moving forward, backward, turning, or continuous velocity and steering control); is the state transition probability, i.e., the probability distribution of transitioning from state to the next state after performing action ; is the immediate reward function, mapping state-action pairs to a numerical reward; is the discount factor, weighing the relative importance of immediate rewards and future rewards. In the path planning task, the state can be defined as the pose of the vehicle in the environment or the surrounding perception information (e.g., position coordinates on a grid map, or laser radar scan results, etc.), and the target state is the predetermined destination point. The action is the navigation control instruction, such as discrete forward, backward, left, or right movement by one step. The design of the immediate reward directly affects the learning effect: a common approach is to define a positive reward (e.g., +100) when reaching the target, a negative reward (e.g., -100) when colliding with obstacles, and a small negative reward (e.g., -1) for each step to encourage shorter paths, which is used to balance path length. Through the above modeling, the vehicle path planning problem is transformed into a reinforcement learning problem of finding the strategy with the maximum cumulative reward on the MDP. The agent (vehicle) tries different actions by continuously interacting with the environment and updates the strategy according to the rewards obtained, eventually learning the optimal strategy that maximizes the long-term return (e.g., reaching the target and taking the shortest path).​

[0042] The deep reinforcement learning algorithm used in this embodiment is Deep Q-Network (DQN). DQN is a combination of the value iteration algorithm Q-learning and a deep neural network, used to approximate the optimal state-action value function. In this embodiment, the state of the autonomous vehicle navigation task is input into a multi-layer neural network, which outputs the Q-values ​​of each action in the action space, and then... - A greedy strategy selects the action with the maximum Q-value for execution. Neural network parameters. The loss function is minimized through repeated gradient descent training, with the goal of satisfying the Bellman optimal equation. During the training process, DQN introduces two key mechanisms: (1) Experience playback: each step of the sample generated by the interaction between the autonomous vehicle and the simulation environment. Stored in the experience pool, where r is the reward; during training, small batches of data are randomly sampled from it, thus breaking the data correlation and improving training stability. (2) Target network: Maintain a delayed-updating target Q network, the target network parameters are... Used to calculate the target value y:

[0043] ;

[0044] in, For the next action;

[0045] The current network parameters are copied to the target network only at fixed intervals to prevent divergence during training. Through these techniques, DQN can gradually converge on high-dimensional decision problems such as autonomous vehicle path planning. It is important to note the impact of network structure and hyperparameters on algorithm performance: the research used a fully connected neural network with two hidden layers to estimate the Q-value, with each layer containing 128 ReLU-activated neurons to balance expressive power and computational cost; the learning rate was set to 0.001, the experience pool size was 10000, and the discount factor was... Greed at its beginning The parameter value is gradually reduced to 0.1 to ensure sufficient exploration in the early stages and full utilization in the later stages. During training, a parameter update is performed after each interaction step, and the target network synchronizes its parameters every 500 steps. This setup has been shown in experiments to achieve stable convergence and obtain a superior strategy.

[0046] set up For state space; For action space (such as forward movement) Turning wait);

[0047] : By deep Q-network (neural network parameters) An approximation of the action-value function;

[0048] Exploration coefficient;

[0049] sample set t represents time t, and N represents the number of samples; the experience replay pool (capacity) ).

[0050] 1. Action selection:

[0051] Given time Perception state The decision module first calculates

[0052] ;

[0053] ;

[0054] in Define the action space capacity; select the action. Then, the corresponding control commands will be executed. The data is sent to the autonomous vehicle via the motion controller. For a moment Autonomous vehicles expect longitudinal linear velocity. For a moment Expected to shift to control quantity, This is the action-control mapping function.

[0055] 2. Numerical safety constraints:

[0056] There is a security evaluation function. The security evaluation function h takes a value of 1 to indicate that execution is performed. This may lead to constraint conflicts (such as predicting collisions).

[0057] The final action to be performed is

[0058] ;

[0059] in An emergency stop or a rule-based avoidance strategy may be adopted.

[0060] 3. Online learning:

[0061] Receive a reward after execution And observe the new state ,Will And periodically sample small batches of samples Perform gradient descent:

[0062] ;

[0063] in, a learning rate, a gradient operator of neural network parameters, a small batch size, a discount factor, a target network parameter.

[0064] 4 Offline pre-training:

[0065] Generate trajectories in a digital twin environment Iterate the above equation until the validation metric satisfies

[0066] ;

[0067] ( a preset tolerance) and then deployed to the actual system for online fine-tuning; wherein an expected operator, an optimal action value function.

[0068] In an optional embodiment of the present application, the blockchain network layer is composed of a number of blockchain nodes to form a consortium chain for storing and sharing system data. The nodes of the consortium chain are deployed at key positions of the industrial network, such as the central control room server, the edge servers of each main workshop, and the unmanned vehicles themselves (when resources are sufficient). Each node runs a blockchain consensus client and participates in the maintenance of the ledger together. For example, Figure 4 ​​As shown, a typical blockchain network topology is depicted: several unmanned vehicles as light nodes submit data to adjacent edge blockchain nodes, which are responsible for participating in consensus accounting. A series of smart contracts (chaincode) are predefined in the consortium chain, such as "vehicle state on-chain contract", "task instruction publishing contract", etc., which stipulate data format and access rights. When an unmanned vehicle sends a transaction (such as uploading path information) to the blockchain, the corresponding contract will verify whether the data is legal (such as signature, permission) and write it into the ledger. Through the blockchain network, a trust intermediary for data sharing is formed between the decision control layer and the perception layer, and between different vehicles: all data obtained by the nodes comes from the consensus blockchain ledger copy, ensuring "consistency and authenticity". For example, when multiple unmanned vehicles need to coordinate to avoid collision, they can make decisions by querying the other party's recent location and speed information on the chain, without directly trusting the data sent by the other party. For example, management personnel can query the historical running track of any vehicle through the blockchain, without worrying about the log being tampered with. It is important to note that the real-time impact of blockchain is fully considered in the design: not all sensor data is on-chain, but only valuable information for global coordination and traceability (such as decision sequence, accident alarm, etc.) is selected to reduce data volume and consensus pressure. A large amount of sensor data related to local real-time control is still transmitted by point-to-point network, which is directly processed by the vehicle controller to ensure millisecond-level response. The blockchain layer provides macro data credibility support for the system, rather than replacing the existing control network.

[0069] This embodiment utilizes blockchain for data storage and interaction in an autonomous vehicle system. In the designed system, the blockchain acts as a distributed database, storing key data from the autonomous vehicle's operation, such as routes obtained from each path planning step, vehicle mileage and energy consumption information, and records of important control commands issued. This data is encapsulated as transactions, which are packaged and consensus-reached by nodes in the blockchain network. The study adopts a consortium blockchain architecture, where participating nodes are composed of industrial users (such as factory servers and AGV management terminals), and are not publicly accessible. The consortium blockchain uses the Byzantine Fault Tolerance (PBFT) algorithm to achieve consensus. Nodes communicate and vote to decide which transactions to write into blocks, thus eliminating the need for time-consuming proof-of-work as in public blockchains. At fixed time intervals or when a certain number of transactions are reached, the block-producing node packages the transactions into blocks and broadcasts them to other nodes for verification. Once a majority of the accounting nodes reach a consensus on the block content, the block is added to the chain. Blocks are linked chronologically, with each block containing the hash value of the previous block, ensuring the chain's immutability. The study parallelizes the process of uploading vehicle data to the blockchain with the autonomous vehicle's navigation decision-making process: whenever the autonomous vehicle completes a planning step or reaches a key milestone (such as passing an important node), relevant information is generated and sent to the blockchain as a transaction. Because PBFT has low confirmation latency (typically within seconds) in a small-scale consortium blockchain, it does not significantly impact the real-time control of the autonomous vehicle. With blockchain, all participants in the system (including the monitoring center, vehicle controller, etc.) maintain the same copy of the ledger. Any attempt by a node to tamper with the data will be rejected due to hash verification failure, thus greatly improving data credibility and system security. Furthermore, the transparent and traceable nature of blockchain allows managers to easily audit vehicle historical paths and operation records, improving operational efficiency. Of course, the study also protects sensitive data privacy through access control: access permissions are set for each type of transaction in the blockchain, allowing only authorized nodes to view detailed vehicle data, while other nodes are only granted necessary summary information. In summary, the designed blockchain data model provides a secure and shared data layer for the autonomous vehicle system, ensuring the reliable operation of the upper-layer path planning algorithm.

[0070] In an optional embodiment of the present application, the application display layer provides a monitoring and human-computer interaction interface of the system, including function modules such as simulation visualization, state monitoring and data analysis. In the simulation stage, a visualization interface is developed, which can display the position of the unmanned vehicle, the planned path and the distribution of obstacles in the virtual environment in real time. The interface also integrates blockchain data browsing function, managers can view the vehicle operation log and transaction details recorded on the chain, such as the route of a vehicle in a certain time period, whether it has stopped abnormally, etc. Another role of the application layer is to provide decision reference and alarm notification to the operator when actually deployed: when the blockchain detects an anomaly (for example, a vehicle has not traveled along the planned route for a long time or the on-chain data is inconsistent), the interface will alarm in time. Through friendly visualization and interaction design, the application layer improves the usability of the system, enabling the operator to intuitively understand the running status of the unmanned vehicle and the data recorded by the blockchain, thereby enhancing the degree of trust in the system.

[0071] In summary, the layers of the system architecture of the present application work closely together: the perception layer provides the environment state, the decision control layer uses the DQN algorithm to output the optimized path and control the vehicle, the blockchain layer ensures the reliable sharing of decision data and vehicle state, and the application layer transparently presents the entire system operation to the user. Such design realizes a closed loop of "data - decision - execution - evidence": the unmanned vehicle executes the optimal strategy given by the intelligent algorithm, and at the same time, every key data has a traceable record, ensuring the unity of running efficiency and safety, as shown in Figure 5 .

[0072] The present application provides an industrial Internet of Things unmanned vehicle path planning method, comprising the following steps:

[0073] The environment state information of the unmanned vehicle is collected by the environment perception layer, and the environment state information of other unmanned vehicles is received from the blockchain; the environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles are transmitted to the decision control layer;

[0074] The environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles transmitted by the environment perception layer are received by the decision control layer, the action selection is performed according to the environment state information of the unmanned vehicle by using the deep Q network, the optimal action is determined by taking the environment state information of other unmanned vehicles as a safety constraint, the control instruction is generated and sent to the environment perception layer; and the environment state information of the unmanned vehicle, the optimal action and the reward are uploaded to the blockchain;

[0075] The environment state information of the unmanned vehicle, the optimal action and the reward uploaded by the decision control layer are received by the blockchain network layer, and are processed as transaction information for on-chain processing; and the environment state information of other unmanned vehicles is broadcasted;

[0076] The motion control of the unmanned vehicle is performed by using the environmental perception layer to receive the control instruction issued by the decision control layer.

[0077] The system is analyzed below in combination with specific experimental design.

[0078] The experimental platform is shown in Table 1

[0079] Table 1 shows the experimental platform configuration table

[0080]

[0081] Dataset and comparison method

[0082] 1. Proposed-DQN: Improved DQN, experience replay 10k, two hidden layers 128 neurons, ε-greedy decay 1→0.1.

[0083] 2. PF-DDQN: The latest multi-AGV reinforcement learning algorithm, reports a 92.62% improvement in DDQN path superiority.

[0084] 3. A* (Dynamic-Replan): Classic A* that recalculates when encountering dynamic obstacles.

[0085] 4. Env-Weighted A*: An improved A* commonly used in the industry with grid weights.

[0086] Each algorithm is independently run in a scenario with 100 random start-end points and an obstacle density of 15%; Proposed-DQN / PF-DDQN is first trained offline for 3k episodes, and then inference testing is performed.

[0087] Evaluation index

[0088] 1. Success rate Success (%)

[0089] 2. Average path length Len (grid)

[0090] 3. Average planning time delay Time (ms / step)

[0091] 4. Collision rate Crash (%)

[0092] 5. Blockchain TPS / confirmation delay (only for Proposed-DQN to record key decisions on the chain)

[0093] Public benchmark: Fabric 4 peer peak reached 1619 TPS; single-transaction confirmation latency ≈ 2 s with BatchTimeout=2 s setting. Figure 1

[0094] Core experimental results

[0095] Path planning performance as shown in Table 2

[0096] Table 2 Performance comparison of different algorithms

[0097]

[0098] From Table 2, the results are consistent with the latest DQN report of 93% success rate in dynamic obstacle scenarios, and further improvements in path length and collision rate. The real-time decision-making of 0.52 ms level fully meets the 50 Hz control cycle requirement; compared with A*, the re-planning saves two orders of magnitude of reasoning time.

[0099] Blockchain subsystem performance as shown in Table 3

[0100] Table 3 Blockchain performance display table

[0101]

[0102] From Table 3, under a load far lower than the Fabric peak of 1619 TPS, the system delay remains <1.5 s without any packet loss or rollback. Tampering tests show that single-node private changes to the ledger are immediately rejected by consensus, meeting the "non-tamperable" design goal.

[0103] Resource overhead

[0104] Edge server: CPU peak 46%, GPU video memory 2.3 GB.

[0105] Peer node: memory increment ≈ 180 MB / h, chain size linear growth 0.9 MB / h, which can be controlled to <5 GB through a semi-annual archiving strategy.

[0106] Conclusion

[0107] Effectiveness: Proposed-DQN is better than the three control lines in success rate, path length, and real-time performance, with a 13.5 pp increase in success rate compared to traditional A*, and an average of 93% in recent literature.

[0108] Credibility: The measured delay of the blockchain is 1.3-1.5 s, which is lower than the official recommended threshold of 2 s, ensuring the industrial logistics state consistency requirement of ≤3 s.

[0109] Ablative observation: If experience replay or target network is closed, the success rate drops to 88%, verifying the necessity of the network design in Section 3.

[0110] Engineering Value: System resource utilization < 50% under 4-car concurrency, no additional hardware required to deploy to a medium-sized manufacturing plant.

[0111] The present application is described in reference to the flowchart and / or block diagram of the method, apparatus (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate a means for implementing the functions specified in the flowchart and / or block diagram. Figure 1 one or more flows and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart and / or block diagram.

[0112] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowchart and / or block diagram. Figure 1 one or more flows and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart and / or block diagram.

[0113] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a means for implementing the functions specified in the flowchart and / or block diagram. Figure 1 one or more flows and / or blocks ​ an apparatus for performing the functions specified in the flowchart and / or block diagram.

[0114] The principles and implementation manners of the present application are described in the specific embodiments, and the above embodiment description is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed; in summary, the content of the specification should not be understood as a limitation of the present application.

[0115] Those skilled in the art will realize that the embodiments described herein are for the purpose of understanding the principles of the present application and should be understood as not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations according to the technical inspiration disclosed in the present application without departing from the essence of the present application, and these modifications and combinations are still within the scope of protection of the present application.

Claims

1. An industrial internet of things unmanned vehicle path planning system, characterized in that, Comprise: An environment perception layer for collecting environment state information of the unmanned vehicle, receiving environment state information of other unmanned vehicles issued by the blockchain, transmitting the environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles to the decision control layer, and receiving control instructions issued by the decision control layer to control the motion of the unmanned vehicle; A decision control layer for receiving the environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles transmitted by the environment perception layer, selecting actions according to the environment state information of the unmanned vehicle using a deep Q network, determining the optimal action as a safety constraint based on the environment state information of other unmanned vehicles, and generating control instructions to issue to the environment perception layer; and uploading the environment state information of the unmanned vehicle, the optimal action and the reward to the blockchain; A blockchain network layer for receiving the environment state information of the unmanned vehicle, the optimal action and the reward uploaded by the decision control layer, processing the information as a transaction, and broadcasting the environment state information of other unmanned vehicles; The decision control layer uses the environment state information of other unmanned vehicles as a safety constraint, which is specifically: Establishing a safety evaluation function where the safety evaluation function h takes the value 1 if performing action may lead to a constraint violation, S is the state space and A is the action space; The decision control layer uses the environment state information of other unmanned vehicles as a safety constraint to determine the optimal action, which is specifically: wherein, is the optimal action, is the action at time t, is the obstacle avoidance action at state s t performs the action may result in a constraint conflict, is the obstacle avoidance action at state s t under the constraint The blockchain network layer uses several unmanned vehicles as light nodes, submits data to adjacent edge blockchain nodes, and is responsible for participating in consensus and accounting; and predefines vehicle state on-chain contract and task instruction publishing contract in the alliance chain, sets data format and access authority, and when the unmanned vehicle sends a transaction to the blockchain, uses the corresponding contract to verify whether the data is legal and writes it into the ledger.

2. The industrial IoT unmanned vehicle path planning system of claim 1, wherein, The decision control layer obtains the reward after the environment perception layer executes the optimal action and observes the new state, adds the state, action, reward and new state to the trajectory four-tuple to the experience replay pool, and periodically samples a small batch of samples in the experience replay pool for gradient descent.

3. The industrial IoT unmanned vehicle path planning system of claim 1, wherein, The decision control layer generates a trajectory four-tuple in a digital twin environment and iterates until the termination condition is met.

4. The industrial IoT unmanned vehicle path planning system of claim 1, wherein, The environment perception layer perceives the surrounding environment information through sensors and generates a state description by preprocessing the raw sensor data with the vehicle controller; at the same time, it is connected with an external positioning base station or a global positioning system to obtain the position of the unmanned vehicle in the global coordinate system.

5. The industrial IoT unmanned vehicle path planning system of claim 1, wherein, It also includes an application display layer for receiving the unmanned vehicle operation record pushed by the blockchain network layer, detecting the abnormal behavior of the unmanned vehicle according to the unmanned vehicle operation record, and displaying the state, path and alarm information of the unmanned vehicle.

6. The industrial IoT unmanned vehicle path planning system of claim 1, wherein, The application display layer displays the position of the unmanned vehicle in the virtual environment, the planned path and the obstacle distribution in real time; based on the blockchain data browsing request, it displays the vehicle operation log and transaction details recorded on the chain in real time, and when the blockchain detects an anomaly, it alarms in time through the interface.

7. An industrial Internet of Things unmanned vehicle path planning method based on the industrial Internet of Things unmanned vehicle path planning system according to any one of claims 1-6, characterized in that, The steps include: Collecting the environment state information of the unmanned vehicle using the environment perception layer, receiving the environment state information of other unmanned vehicles issued by the blockchain, and transmitting the environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles to the decision control layer; The decision control layer receives the environment state information of the unmanned vehicle and the environment state information of other unmanned vehicles transmitted by the environment perception layer, selects actions according to the environment state information of the unmanned vehicle by using a deep Q network, determines an optimal action by taking the environment state information of other unmanned vehicles as a safety constraint, generates a control instruction, and issues the control instruction to the environment perception layer; and uploads the environment state information of the unmanned vehicle, the optimal action and the reward to the blockchain; The blockchain network layer receives the environment state information of the unmanned vehicle, the optimal action and the reward uploaded by the decision control layer, and performs chain processing as transaction information; and broadcasts the environment state information of other unmanned vehicles; The environment perception layer receives the control instruction issued by the decision control layer to control the motion of the unmanned vehicle.

Images