AGV intelligent scheduling and management system based on digital twinning

Abstract

The application discloses an AGV intelligent scheduling and management system based on digital twinning, which comprises a digital twinning module: scheduling state data after fusion processing is simulated and optimized through an AGV digital twinning body, task simulation is carried out in a virtual environment according to actual workshop layout, performance indexes under different scheduling strategies are counted, optimal indexes are obtained, and an initial scheduling strategy is obtained; and a high-level scheduling module: a high-level strategy reinforcement model is constructed based on a reinforcement learning algorithm and a strategy network, and the initial scheduling strategy is taken as model input, and an optimized AGV scheduling scheme is output, so that path congestion caused by the fact that the existing mode cannot be adjusted according to dynamic changes of a workshop, and the fact that task priority is not distinguished clearly are effectively solved.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology, and in particular to an intelligent scheduling and management system for AGVs based on digital twins. Background Technology

[0002] In modern manufacturing, the application of automated guided vehicles (AGVs) is becoming increasingly widespread. They undertake key tasks such as material handling and assembly assistance within the workshop, playing a vital role in production efficiency and logistics operations. However, traditional AGV scheduling and management methods have many shortcomings.

[0003] Existing AGV scheduling and management systems struggle to effectively integrate diverse AGV data and fail to fully explore the correlations and roles among different data points. This results in a lack of comprehensive and accurate basis for scheduling decisions. Furthermore, AGV scheduling often relies on preset fixed rules, such as fixed route travel and fixed task assignments. This model cannot adaptively adjust to the real-time status and dynamic changes in the workshop, leading to problems such as path congestion and unclear task priority distinctions, which affect the overall management rhythm. Therefore, this paper proposes an intelligent scheduling and management system for AGVs based on digital twins. Summary of the Invention

[0004] To overcome the aforementioned shortcomings of the prior art and to achieve the above objectives, the present invention proposes the following technical solution: an intelligent scheduling and management system for AGV vehicles based on digital twins, comprising: Data acquisition module: Collects AGV environmental and task data based on multi-source sensors; Data fusion module: It fuses AGV environmental data and task data through an optimized deep network architecture to obtain initial fused data, calculates the frequency domain feature weights of the initial fused data and performs weighted fusion, and finally obtains the scheduling status data through inverse Fourier transform. Digital twin module: The fused state data is simulated and optimized through the AGV digital twin, and the task is simulated in a virtual environment based on the actual layout of the workshop. The performance indicators under different scheduling strategies are statistically analyzed and the optimal indicators are obtained to obtain the initial scheduling strategy. Advanced scheduling module: Based on reinforcement learning algorithms and policy networks, a high-level policy reinforcement model is constructed, and the initial scheduling policy is used as the input of the model to output the final optimized AGV scheduling scheme; AGV Management Module: Based on the AGV scheduling scheme, it provides comprehensive management of AGV vehicles.

[0005] The AGV environmental data acquisition process is as follows: LiDAR sensors are installed on the top and around the AGV (Automated Guided Vehicle). The laser beams reflect off surrounding obstacles. By measuring the round-trip time of the laser beams and calculating the relative distance between the obstacles and the AGV based on the speed of light, and combining this with the rotation angle of the LiDAR sensors, the distance information is converted into polar or Cartesian coordinates to construct a 3D point cloud map around the AGV. This 3D point cloud map forms the AGV's environmental data. ; The process of acquiring AGV task data is as follows: A task identification device is installed in the operating area or cargo-carrying part of the AGV. When the AGV approaches the task start point or cargo, it automatically scans and analyzes the key information of the task. This key information is combined to form AGV task data. ; The key information includes the task type, task destination, and task parameters.

[0006] The process of obtaining the initial fusion data is as follows: AGV environmental data AGV task data Merging input data as part of a deep network architecture ,Right now ; Convert it into matrix form for use as input to convolutional layers. The matrix form is represented as The tensor, where, here It is the number of channels. and It refers to the height and width of the data; To build an optimized deep network architecture, the construction process is as follows: Optimizing deep network architecture includes three parts: channel-specific convolution and interaction, adaptive weighted pooling, and fusion of fully connected layers; Multichannel convolution and interaction include environment data channel convolution and task data channel convolution; Environmental data channel convolution is achieved by determining the environmental data convolution parameters; Based on the determined parameters and the initialized convolution kernel and bias, an environmental data channel is constructed, expressed by the formula: ,in, The convolution kernel for the environmental data channel convolution. The bias term for the convolution of the environmental data channels. Indicates the channel index; The environmental data channel is convolved to generate the output features of the environmental data channel convolutional layer. Task data channel convolution determines the task data convolution parameters, including the size of the first layer convolution kernel. Number of convolution kernels and step length ; Based on the determined parameters and the initialized convolution kernel and bias, the task data channel is constructed, expressed by the formula: ,in, The convolution kernel for the task data channel convolution. The bias term for the convolution of the task data channels. Indicates the channel index; Perform convolution operations on the data in the task data channel to generate the output features of the task data channel convolutional layer; By fusing and interacting the environmental data channel and the task data channel, the fused output features are obtained. , represented as ,in, The weight matrix, obtained through random initialization, is formed by concatenating the data from two channels along the channel dimension and multiplying by the weight matrix. This enables cross-channel feature fusion, which then forms the multi-channel convolutional and interactive components. Adaptive weighted pooling includes Each pooling layer has parameters including the pool size. and step length , This indicates the pool size of the first pooling layer. This indicates the step size of the first pooling layer; Define a learning weight Output features based on fusion interaction Construct the weighted pooling operation formula: Where 'a' is the channel index of the environmental data channel. This is the channel index for the task data channel. If the output of the first pooling layer in the adaptive weighted pooling process is given, then the... The output of the layer pooling layer is ; The fully connected layer unifies the output after multiple layers of adaptive weighted pooling into a one-dimensional vector. , length is ; Define the number of neurons in a fully connected layer as And initialize the weight matrix of the fully connected layer. and bias terms The formula for a fully connected layer is then expressed as: Where i is the index, ranging from 0 to 1. between, This represents the output of the fully connected layer, i.e., the initial fused data.

[0007] The process of obtaining the scheduling status data is as follows: Perform a Discrete Fourier Transform on the initial fused data, and let... The discrete values ​​of the initial fused data in the time domain are represented by the Discrete Fourier Transform formula: ,in, Represents the frequency components in the frequency domain. It refers to the number of data points; The initial fused data is transformed from the time domain to the frequency domain to obtain frequency domain data. The purpose of this transformation is to analyze the data in the frequency domain in order to better uncover the characteristics and patterns of the data; For each frequency component Define its energy Then calculate the frequency domain feature weights: ,in, For the first Energy of each frequency component After obtaining the frequency domain feature weights, the frequency domain data is weighted and fused. Let the weighted and fused frequency domain data be... The weighted and fused frequency domain data is subjected to an inverse discrete Fourier transform, and finally the frequency domain data is converted back to the time domain to obtain the final scheduling state data. .

[0008] The environmental data convolution parameters include the size of the first layer convolution kernel. Number of convolution kernels and step length ; The task data convolution parameters include the size of the first convolution kernel. Number of convolution kernels and step length .

[0009] The process of obtaining the initial scheduling policy is as follows: The status data is input into the AGV digital twin system, and a digital twin is constructed based on the AGV parameters in the status data; A virtual environment model is created based on the actual layout of the workshop. Tasks are defined in the virtual environment model. Each task includes the starting position, target position, and priority information. Tasks are initially assigned to the digital twin AGV according to specified rules, and the position and status of the AGV in the virtual environment are updated at each time step. After assigning tasks, performance metrics are defined, and several scheduling strategies to be verified are designed. A simulation is then performed in a virtual environment to calculate the average performance metrics under different scheduling strategies. The strategy with the best performance on key metrics is selected as the initial scheduling strategy, expressed as: .

[0010] The construction process of the high-level strategy reinforcement model is as follows: Define the state space of the high-level policy reinforcement model in the reinforcement learning process as follows: ; state space Includes task distribution within the workshop and traffic flow conditions ,in, for The matrix, row vectors Represents the number of task sites, a column vector. The number of task types is represented by d, and the data points within the matrix are denoted as d, indicating traffic flow conditions. It is in vector form, representing the AGV flow rate of each main channel; The actions of high-level strategies are defined as macro-dispatch instructions, which include regional task allocation strategies and overall traffic control strategies. These macro-dispatch instructions constitute an action space. ; Define a high-level policy reward function The reward function is based on the workshop's production efficiency and task completion rate indicators, and is expressed as: ,in, It is the number of tasks completed in the workshop. It refers to workshop production efficiency (such as output per unit of time). It refers to the degree of collision in the workshop. , , These are weighting coefficients; Based on state space From the action space through the policy network Output a macro-scheduling action with an exploration rate. Make action selection, in order to The probability is randomly selected from all possible macro-scheduling actions, so that... The probability of selecting an action is based on the output of the policy network; After executing the action, observe the changes in the workshop environment, obtain the macroscopic information status of the workshop at the next time step, and simultaneously calculate the immediate reward based on the reward function, and update the current state. The selected action and instant rewards The policy gradient is calculated using the backpropagation algorithm. The specific calculation process is as follows: First, obtain the probability distribution of the output action, represented as: Then calculate the model's Obtain the policy gradient through the loss function The parameters of the high-level policy network are updated using the obtained policy gradient and learning rate. The updated formula is: ,in, Indicates the learning rate; Observe the changes in the reward function value and calculate the continuous The average value of the reward function over each time step When the average value of the reward function If the change is less than the preset threshold within consecutive time steps, the model is considered to have converged, the parameter update of the model is stopped, the model has completed convergence, and the high-level policy reinforcement model is obtained.

[0011] The process of obtaining the AGV scheduling scheme is as follows: Initial scheduling policy As input to the high-level strategy reinforcement model, the model outputs several optimal macro-scheduling actions, which constitute the final optimized AGV scheduling scheme.

[0012] The present invention has the following beneficial effects: 1. In this invention, firstly, data acquisition and fusion technology is used to integrate various aspects of AGV data, including environmental data and task data. A three-dimensional point cloud map is constructed using environmental data collected by LiDAR, and task data is obtained through a task recognition device. These data are then integrated through an optimized deep network architecture. Firstly, through channel-wise convolution and interactive operations, the deep network can extract and analyze different features of environmental data and task data respectively, while also allowing them to influence and fuse with each other at different levels of the network. This achieves comprehensive integration of various aspects of data, enabling the scheduling system to simultaneously consider the physical environment of the workshop and task requirements. Secondly, during the fusion process, components such as the adaptive weighted pooling and fusion fully connected layer of the deep network architecture can automatically learn the correlation and weight between different data. By discovering the potential relationship between features in environmental data and certain factors in task data (such as the frequency and urgency of tasks passing through the area), it ensures that scheduling decisions are based on complete and accurate information and fully consider the mutual influence and constraints between various factors. In this way, subsequent scheduling decisions can better adapt to the actual situation in the workshop. Finally, based on the combination of digital twins and reinforcement learning, it no longer relies on preset fixed rules. In a virtual environment, a model is created and tasks are defined according to the actual layout of the workshop. The initial scheduling strategy is determined through multiple simulations and performance index evaluations. Then, combined with high-level strategy reinforcement model, the optimized AGV scheduling scheme is automatically output according to the real-time dynamic changes in task distribution and traffic flow in the workshop, including regional task allocation ratio and channel passage priority. This enables AGV scheduling to adaptively adjust according to the real-time status of the workshop, effectively solving the problems of path congestion and unclear task priority due to the inability of the existing mode to adjust according to the dynamic changes of the workshop, and improving the overall operating efficiency and smoothness of the workshop. Attached Figure Description

[0013] Figure 1 This is a system block diagram of the AGV intelligent scheduling and management system based on digital twin proposed in this invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] Example 1, as Figure 1 As shown, the intelligent scheduling and management system for AGVs based on digital twins proposed in this invention includes a data acquisition module that collects AGV environmental data and task data based on multi-source sensors. The AGV environmental data acquisition process is as follows: LiDAR sensors are installed on the top and around the AGV (Automated Guided Vehicle). When the laser beam encounters an obstacle, it reflects back. By measuring the round-trip time of the laser beam and calculating the relative distance between the obstacle and the AGV based on the speed of light, and combining this with the LiDAR's own rotation angle, the distance information is converted into polar or Cartesian coordinates to construct a 3D point cloud map of the AGV's surroundings. This map accurately reflects the distribution of obstacles; this 3D point cloud map constitutes the AGV's environmental data. ; The process of acquiring AGV task data is as follows: Task identification devices, such as barcode scanners or electronic tag readers, are installed in the operating area or cargo-carrying part of the AGV. When the AGV approaches the task start point or cargo, the device automatically scans the barcodes, electronic tags, and other identifiers attached to the task sheet or cargo packaging, and parses out the key task information, including the task type (such as handling, assembly, sorting, etc.), task destination, and task parameters (such as the weight and size requirements of the goods to be handled). This data is combined to form the AGV task data. .

[0016] Data fusion module: AGV environment data and task data are fused through an optimized deep network architecture to obtain initial fused data. The frequency domain feature weights of the initial fused data are calculated and weighted fused. Then, inverse Fourier transform is performed to obtain scheduling status data. The process of obtaining the initial fused data is as follows: AGV environmental data AGV task data Merging input data as part of a deep network architecture ,Right now ; Convert it into matrix form for use as input to convolutional layers. The matrix form is represented as The tensor, where, here This refers to the number of channels (e.g., if environmental data and task data are used as different channels). and It is the height and width of the data ( That is, the dimensions of the data. Width (i.e., data length); Specifically, will Convert to matrix form The rationale is that environmental data and task data can be viewed as different "channels," so the number of channels can be set here. (One channel is for environmental data, and the other is for task data), and the environmental data is one-dimensional, let's say... The data length W is the sum of the lengths of the environment data and the task data. Here, we assume there are a total of 100 data vectors, i.e. Therefore, input data The matrix form is In a deep network architecture, this tensor will be used as the input to the convolutional layer. The convolutional kernel will slide on this tensor to perform convolution operations and extract features. In this way, the deep network can process environmental data and task data at the same time and extract meaningful features for subsequent AGV scheduling and management. To build an optimized deep network architecture, the construction process is as follows: Optimizing deep network architecture includes three parts: channel-specific convolution and interaction, adaptive weighted pooling, and fusion of fully connected layers; Multichannel convolution and interaction include environment data channel convolution and task data channel convolution; Environmental data channel convolution determines the environmental data convolution parameters, including the size of the first-layer convolution kernel. Number of convolution kernels and step length These parameters are used to define how the convolution operation is performed; Based on the determined parameters and the initialized convolution kernel and bias, an environmental data channel is constructed, expressed by the formula: ,in, The convolution kernel for the environmental data channel convolution. The bias term for the convolution of the environmental data channels. Indicates the channel index; The environmental data channel is convolved to generate the output features of the environmental data channel convolutional layer. Task data channel convolution is achieved by determining the convolution parameters, which include the size of the first-layer convolution kernel. Number of convolution kernels and step length These parameters are used to define how the convolution operation is performed; Based on the determined parameters and the initialized convolution kernel and bias, the task data channel is constructed, expressed by the formula: ,in, The convolution kernel for the task data channel convolution. The bias term for the convolution of the task data channels. Indicates the channel index; Perform convolution operations on the data in the task data channel to generate the output features of the task data channel convolutional layer; By fusing and interacting the environmental data channel and the task data channel, the fused output features are obtained. , represented as ,in, The weight matrix, obtained through random initialization, is formed by concatenating the data from two channels along the channel dimension and multiplying by the weight matrix. By achieving cross-channel feature fusion, a multi-channel convolution and interaction part is formed; Adaptive weighted pooling includes Each pooling layer (the first pooling layer is used as an example here) has parameters including the pool size. and step length , This indicates the pool size of the first pooling layer. This indicates the step size of the first pooling layer; Define a learning weight Output features based on fusion interaction Construct the weighted pooling operation formula: Where 'a' is the channel index of the environmental data channel. This is the channel index for the task data channel. If the output of the first pooling layer in the adaptive weighted pooling process is given, then the... The output of the layer pooling layer is ; The fully connected layer unifies the output after multiple layers of adaptive weighted pooling into a one-dimensional vector. , length is ; Define the number of neurons in a fully connected layer as And initialize the weight matrix of the fully connected layer. and bias terms The formula for a fully connected layer is then expressed as: Where i is the index, ranging from 0 to 1. between, This represents the output of the fully connected layer, i.e., the initial fused data; Specifically, the purpose of the data fusion module is to fuse effective features from AGV environmental data and task data to obtain state data that reflects the overall state of the AGV. This process is achieved through an optimized deep network architecture. In this architecture, environmental data and task data are first input into the network. The role of the fully connected layer in the deep network is to integrate features. After the preceding channel-wise convolution and interaction, and adaptive weighted pooling operations, the features of the data are gradually extracted and abstracted. The fully connected layer integrates these features and adds a bias term to obtain a new vector. Channel-wise convolution and interaction, and adaptive weighted pooling operations are for feature extraction and preliminary fusion of the original environmental data and task data. The fully connected layer performs the final integration processing based on these features, and finally outputs the feature information that integrates the environmental data and task data extracted and processed in the network. Therefore, it can be regarded as a fused representation of environmental data and task data, i.e., the initial fused data. The process of obtaining scheduling status data is as follows: Perform a Discrete Fourier Transform on the initial fused data, and let... The discrete values ​​of the initial fused data in the time domain are represented by the Discrete Fourier Transform formula: ,in, Represents the frequency components in the frequency domain. It refers to the number of data points; The initial fused data is transformed from the time domain to the frequency domain to obtain frequency domain data. The purpose of this transformation is to analyze the data in the frequency domain in order to better uncover the characteristics and patterns of the data; For each frequency component Define its energy Then calculate the frequency domain feature weights: ,in, For the first Energy of each frequency component Specifically, the weight is calculated by dividing the energy of each frequency component by the sum of the energies of all frequency components. The resulting weight reflects the relative importance of each frequency component in the entire frequency domain. In this way, important frequency features can be highlighted and relatively unimportant features can be suppressed, thereby enabling better data fusion and feature extraction. After obtaining the frequency domain feature weights, the frequency domain data is weighted and fused. Let the weighted and fused frequency domain data be... The weighted and fused frequency domain data is subjected to an inverse discrete Fourier transform, and finally the frequency domain data is converted back to the time domain to obtain the final scheduling state data. ; Specifically, the time-domain information is restored from the weighted fusion frequency-domain information to obtain the scheduling status data. By integrating information from AGV environmental and task data, and through frequency domain analysis and weighted processing, it can more accurately reflect the status of AGV vehicles, providing more valuable data support for subsequent AGV scheduling and management.

[0017] Digital twin module: The fused state data is simulated and optimized through the AGV digital twin, and the task is simulated in a virtual environment based on the actual layout of the workshop. The performance indicators under different scheduling strategies are statistically analyzed and the optimal indicators are obtained to obtain the initial scheduling strategy. The process of obtaining the initial scheduling policy is as follows: The status data is input into the AGV digital twin system, and a digital twin is constructed based on the AGV parameters in the status data; Specifically, the AGV parameters in the status data include information such as the AGV's position coordinates, speed, and final destination obtained from the status data of the environment and task data. These information are all data points in the status data obtained from the environment and task data. A virtual environment model is created based on the actual layout of the workshop. Tasks are defined in the virtual environment model. Each task includes the starting position, target position, and priority information. Tasks are initially assigned to the digital twin AGV according to specified rules, and the position and status of the AGV in the virtual environment are updated at each time step. Specifically, the rule is to allocate tasks according to the order in which they arrive, with the first task to arrive being assigned to an idle AGV. Implementation steps: Maintain a task queue. When a new task arrives, add it to the end of the queue. When an AGV is available, retrieve the task from the head of the task queue and assign it. For example, if a task arrives first, it enters the task queue; if another task arrives next, it also enters the task queue. At this time, the task is at the head of the task queue and the task is at the tail of the task queue. When an AGV is available, assign the task to that AGV, and then move the task to the head of the queue to wait for the next available AGV. After task allocation, performance metrics are defined, including task completion time, path length, energy consumption, and task delay rate. Then, several scheduling strategies to be verified are designed, including shortest path selection, task priority, energy consumption control, and task delay control. For each strategy, multiple simulations are performed in a virtual environment, and the performance metric values ​​from each simulation are recorded and averaged. Average values ​​of various performance metrics under different strategies The strategy with the best performance on key metrics is selected as the initial scheduling strategy, expressed as: ; Specifically, Different initial scheduling strategies, for example, Represents the shortest path strategy. Represents task priority strategy, This indicates the number of initial scheduling policies, i.e., the number of policies contained in the set; By performing multiple simulations of these different initial scheduling strategies in a virtual environment, recording the performance metrics (such as task completion time, path length, energy consumption, task delay rate, etc.) of each simulation, and calculating the average value of each performance metric, then comparing the average values ​​of each performance metric under different strategies, the strategy that performs best in key metrics is selected as the final initial scheduling strategy. This set covers all available and evaluable initial scheduling strategies and is the foundation of the entire scheduling strategy selection and optimization process.

[0018] Advanced scheduling module: Based on reinforcement learning algorithms and policy networks, a high-level policy reinforcement model is constructed, and the initial scheduling policy is used as the input of the model to output the final optimized AGV scheduling scheme; The process of obtaining the AGV scheduling scheme is as follows: Construct a high-level policy reinforcement model. The construction process of the high-level policy reinforcement model is as follows: Define the state space of the high-level policy reinforcement model in the reinforcement learning process as follows: ; state space Includes task distribution within the workshop and traffic flow conditions ,in, for The matrix, row vectors Represents the number of task sites, a column vector. The number of task types is represented by d, and the data points within the matrix are denoted as d, indicating traffic flow conditions. It is in vector form, representing the AGV flow rate of each main channel; For example, if a workshop has 2 task stations and 3 task types, then... The first task site indicates there is one type 1 task, one type 2 task, and no type 3 task. The second task site indicates there is one type 1 task, no type 2 task, and one type 3 task. This indicates that the AGV flow rates for the three main channels are 5, 3, and 2 vehicles / minute, respectively. The actions of high-level strategies are defined as macro-level scheduling instructions. Macro-level scheduling instructions include regional task allocation strategies (such as dividing the workshop into several regions and determining the proportion of tasks allocated to each region) and overall traffic control strategies (such as setting the passage priority for certain channels). All macro-level scheduling instructions constitute an action space. ; Define a high-level policy reward function The reward function is based on the workshop's production efficiency and task completion rate indicators, and is expressed as: ,in, It is the number of tasks completed in the workshop. It refers to workshop production efficiency (such as output per unit of time). It refers to the degree of collision in the workshop. , , These are weighting coefficients; Specifically, , , All of these are obtained through the process of digital twin simulation; For example: In the process of acquiring data, various tasks are defined in the virtual environment during digital twin simulation, such as handling and assembly. Each task has a clear starting position, target position, and priority information. When the AGV digital twin executes tasks according to preset rules, the system tracks the status of each task in real time and counts the number of completed tasks to form the data. ; Similarly, In the process of acquiring information, during digital twin simulation, for different types of tasks, taking material handling tasks as an example, output efficiency is defined as the number of goods handled or the total volume of goods handled per unit time; for assembly tasks, output efficiency is defined as the number of assembled products completed per unit time. In the process of acquiring data, during the digital twin simulation, for each AGV digital twin, the distance between it and other AGV digital twins and obstacles in the virtual environment is calculated. If the distance is less than the preset safe distance (determined according to the size of the AGV and safety requirements), a collision is considered to have occurred. Based on state space From the action space through the policy network Output a macro-scheduling action with an exploration rate. Make action selection, in order to The probability is randomly selected from all possible macro-scheduling actions, so that... The probability of selecting an action is based on the output of the policy network; After executing the action, observe the changes in the workshop environment, obtain the macroscopic information status of the workshop at the next time step, and simultaneously calculate the immediate reward based on the reward function, and update the current state. The selected action and instant rewards The policy gradient is calculated using the backpropagation algorithm. The specific calculation process is as follows: First, obtain the probability distribution of the output action, represented as: Then calculate the model's The policy gradient is obtained through the loss function. The parameters of the high-level policy network are updated using the obtained policy gradient and learning rate. The updated formula is: ,in, Indicates the learning rate; Observe the changes in the reward function value and calculate the continuous The average value of the reward function over each time step If the change is less than a preset threshold within multiple consecutive time windows (e.g., 20 windows), the model is considered to have converged, the parameter update of the model is stopped, and the model has completed convergence. Once the model converges, training stops. The resulting high-level policy reinforcement model has learned a better policy under the current workshop environment and reward function settings. Based on the initial scheduling policy input, the model can output several optimal macro-scheduling actions (such as regional task allocation ratio, channel passage priority, etc.). These outputs constitute the final optimized AGV scheduling scheme, which can be used to guide the scheduling of AGVs in the actual workshop to achieve efficient production and logistics operations and improve the overall performance and resource utilization of the workshop.

[0019] AGV Management Module: Based on the AGV scheduling scheme, this module provides comprehensive management of AGV vehicles. When a new task arrives at the workshop, the model is reinforced by the pre-trained high-level strategy. The model outputs a task allocation scheme and outputs the regional task allocation ratio based on the current workshop status (including task distribution and traffic flow). For example: The model output divides the workshop into 3 areas. The current new task should be performed according to... , , The tasks are allocated proportionally to these three areas. For each area, based on the current status of the AGVs (such as location, battery level, task progress, etc., which can be obtained through the AGV's positioning system and status monitoring system), the tasks assigned to that area are further distributed to suitable AGVs. Priority is given to AGVs that are closer to the task's starting position, have sufficient battery power, and whose task progress allows them to take on new tasks. For example, if there are 5 AGVs in area 1, the second AGV... If the location is closest to the starting point of the new mission, has sufficient power, and the mission progress allows, then allocate a portion of the missions in the assigned area to [the new mission's starting point]. ; Path planning: Based on the AGV scheduling scheme output by the model, the path planning of the AGVs is adjusted in real time. When multiple AGVs may pass through the same channel at the same time, the AGVs with higher priority are given priority. The positioning system in the workshop is used to obtain the location information of the AGVs in real time. Combined with the AGV scheduling scheme, the driving path of the AGVs is dynamically planned to avoid congested areas and obstacles, so as to ensure smooth and efficient path.

[0020] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An intelligent scheduling and management system for AGV vehicles based on digital twins, characterized in that, include: Data acquisition module: Collects AGV environmental and task data based on multi-source sensors; Data fusion module: It fuses AGV environmental data and task data through an optimized deep network architecture to obtain initial fused data, calculates the frequency domain feature weights of the initial fused data and performs weighted fusion, and finally obtains the scheduling status data through inverse Fourier transform. Digital twin module: The fused scheduling status data is simulated and optimized through the AGV digital twin, and the task is simulated in a virtual environment based on the actual layout of the workshop. The performance indicators under different scheduling strategies are statistically analyzed and the optimal indicators are obtained to obtain the initial scheduling strategy. Advanced scheduling module: Based on reinforcement learning algorithms and policy networks, a high-level policy reinforcement model is constructed, and the initial scheduling policy is used as the input of the model to output the final optimized AGV scheduling scheme; AGV Management Module: Based on the AGV scheduling scheme, manages the AGV vehicles.

2. The AGV intelligent scheduling and management system based on digital twins according to claim 1, characterized in that, The AGV environmental data acquisition process is as follows: LiDAR sensors are installed on the top and around the AGV (Automated Guided Vehicle). The laser beams reflect off surrounding obstacles. By measuring the round-trip time of the laser beams and calculating the relative distance between the obstacles and the AGV based on the speed of light, and combining this with the rotation angle of the LiDAR sensors, the distance information is converted into polar or Cartesian coordinates to construct a 3D point cloud map around the AGV. This 3D point cloud map forms the AGV's environmental data. ; The process of acquiring AGV task data is as follows: A task identification device is installed in the operating area or cargo-carrying part of the AGV. When the AGV approaches the task start point or cargo, it automatically scans and analyzes the key information of the task and combines it into AGV task data. ; The key information includes the task type, task destination, and task parameters.

3. The AGV intelligent scheduling and management system based on digital twins according to claim 1, characterized in that, The process of obtaining the initial fusion data is as follows: AGV environmental data AGV task data Merging input data as part of a deep network architecture ,Right now ; Convert it into matrix form for use as input to convolutional layers. The matrix form is represented as The tensor of, where, It is the number of channels. and It refers to the height and width of the data; To build an optimized deep network architecture, the construction process is as follows: Optimizing deep network architecture includes three parts: channel-specific convolution and interaction, adaptive weighted pooling, and fusion of fully connected layers; Multichannel convolution and interaction include environment data channel convolution and task data channel convolution; Environmental data channel convolution is achieved by determining the environmental data convolution parameters; Based on the determined parameters and the initialized convolution kernel and bias, an environmental data channel is constructed, expressed by the formula: ,in, The convolution kernel for the environmental data channel convolution. The bias term for the convolution of the environmental data channels. Indicates the channel index; The environmental data channel is convolved to generate the output features of the environmental data channel convolutional layer. Task data channel convolution determines the task data convolution parameters, including the size of the first layer convolution kernel. Number of convolution kernels and step length ; Based on the determined parameters and the initialized convolution kernel and bias, the task data channel is constructed, expressed by the formula: ,in, The convolution kernel for the task data channel convolution. The bias term for the convolution of the task data channels. Indicates the channel index; Perform convolution operations on the data in the task data channel to generate the output features of the task data channel convolutional layer; By fusing and interacting the environmental data channel and the task data channel, the fused output features are obtained. , represented as ,in, The weight matrix, obtained through random initialization, is formed by concatenating the data from two channels along the channel dimension and multiplying by the weight matrix. This enables cross-channel feature fusion, which then forms the multi-channel convolutional and interactive components. Adaptive weighted pooling includes Each pooling layer has parameters including the pool size. and step length , This indicates the pool size of the first pooling layer. This indicates the step size of the first pooling layer; Define a learning weight Output features based on fusion interaction Construct the weighted pooling operation formula: Where 'a' is the channel index of the environmental data channel. This is the channel index for the task data channel. If the output of the first pooling layer in the adaptive weighted pooling process is given, then the... The output of the layer pooling layer is ; The fully connected layer unifies the output after multiple layers of adaptive weighted pooling into a one-dimensional vector. The length is the number of pooling layers. ; Define the number of neurons in a fully connected layer as And initialize the weight matrix of the fully connected layer. and bias terms The formula for a fully connected layer is then expressed as: Where i is the index, ranging from 0 to 1. between, This represents the output of the fully connected layer, i.e., the initial fused data.

4. The AGV intelligent scheduling and management system based on digital twins according to claim 3, characterized in that, The process of obtaining the scheduling status data is as follows: Perform a Discrete Fourier Transform on the initial fused data, and let... The discrete values ​​of the initial fused data in the time domain are represented by the Discrete Fourier Transform formula: ,in, Represents the frequency components in the frequency domain. It refers to the number of data points; The initial fused data is transformed from the time domain to the frequency domain to obtain frequency domain data. ; For each frequency component Define its energy Then calculate the frequency domain feature weights: ,in, For the first Energy of each frequency component After obtaining the frequency domain feature weights, the frequency domain data is weighted and fused. Let the weighted and fused frequency domain data be... The weighted and fused frequency domain data is subjected to an inverse discrete Fourier transform, and finally the frequency domain data is converted back to the time domain to obtain the final scheduling state data. .

5. The AGV intelligent scheduling and management system based on digital twins according to claim 3, characterized in that, The environmental data convolution parameters include the size of the first convolution kernel. Number of convolution kernels and step length ; The task data convolution parameters include the size of the first convolution kernel. Number of convolution kernels and step length .

6. The AGV intelligent scheduling and management system based on digital twins according to claim 1, characterized in that, The process of obtaining the initial scheduling policy is as follows: The status data is input into the AGV digital twin system, and a digital twin is constructed based on the AGV parameters in the status data; A virtual environment model is created based on the actual layout of the workshop. Tasks are defined in the virtual environment model. Each task includes the starting position, target position, and priority information. Tasks are initially assigned to the digital twin AGV according to specified rules, and the position and status of the AGV in the virtual environment are updated at each time step. After assigning tasks, performance metrics are defined, and several scheduling strategies to be verified are designed. Simulations are performed in a virtual environment, and the average values ​​of each performance metric under different scheduling strategies are calculated. The strategy with the best performance on key metrics is selected as the initial scheduling strategy, expressed as: .

7. The AGV intelligent scheduling and management system based on digital twins according to claim 1, characterized in that, The construction process of the high-level strategy reinforcement model is as follows: Define the state space of the high-level policy reinforcement model in the reinforcement learning process as follows: ; state space Includes task distribution within the workshop and traffic flow conditions ,in, for The matrix, row vectors Represents the number of task sites, a column vector. The number of task types is represented by d, and the data points within the matrix are denoted as d, indicating traffic flow conditions. It is in vector form, representing the AGV flow rate of each main channel; The actions of high-level strategies are defined as macro-dispatch instructions. Macro-dispatch instructions include regional task allocation strategies and overall traffic control strategies. All macro-dispatch instructions constitute an action space. ; Define a high-level policy reward function The reward function is based on the workshop's production efficiency and task completion rate indicators, and is expressed as: ,in, It is the number of tasks completed in the workshop. It's about workshop production efficiency. It refers to the degree of collision in the workshop. , , These are weighting coefficients; Based on state space From the action space through the policy network Output a macro-scheduling action with an exploration rate. Make action selection, in order to The probability is randomly selected from all possible macro-scheduling actions, so that... The probability of selecting an action is based on the output of the policy network; After executing the action, observe the changes in the workshop environment, obtain the macroscopic information status of the workshop at the next time step, and simultaneously calculate the immediate reward based on the reward function, and update the current state. The selected action and instant rewards The policy gradient is calculated using the backpropagation algorithm. The specific calculation process is as follows: First, obtain the probability distribution of the output action, represented as: Then calculate the model's The policy gradient is obtained through the loss function. The parameters of the high-level policy network are updated using the obtained policy gradient and learning rate. The updated formula is: ,in, Indicates the learning rate; Observe the changes in the reward function value and calculate the continuous The average value of the reward function over each time step When the average value of the reward function If the change is less than the preset threshold within consecutive time steps, the model is considered to have converged, the parameter update of the model is stopped, the model has completed convergence, and the high-level policy reinforcement model is obtained.

8. The AGV intelligent scheduling and management system based on digital twins according to claim 7, characterized in that, The process of obtaining the AGV scheduling scheme is as follows: Initial scheduling policy As input to the high-level strategy reinforcement model, the model outputs several optimal macro-scheduling actions, which together constitute the final optimized AGV scheduling scheme.

Images