AGV intelligent scheduling and management system based on digital twinning

Abstract

The invention discloses an AGV intelligent scheduling and management system based on digital twinning, and the system comprises a digital twinning module which carries out the simulation optimization of the scheduling state data after fusion processing through an AGV digital twinning body, and carries out the task simulation in a virtual environment according to the actual layout of a workshop. Counting performance indexes under different scheduling strategies and obtaining an optimal index, and obtaining an initial scheduling strategy; and the advanced scheduling module is used for constructing a high-level strategy enhancement model based on a reinforcement learning algorithm and a strategy network, taking the initial scheduling strategy as model input, and outputting a final optimized AGV scheduling scheme. The problems of path congestion, unclear task priority distinguishing and the like caused by the fact that an existing mode cannot be adjusted according to workshop dynamic changes are effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and in particular to an AGV (Automated Guided Vehicle) intelligent scheduling and management system based on digital twins. Background Art

[0002] Automated Guided Vehicles (AGVs) are increasingly used in modern manufacturing. They perform critical tasks within the shop floor, such as material handling and assembly assistance, and play a significant role in production efficiency and logistics operations. However, traditional AGV scheduling and management methods have numerous shortcomings.

[0003] The existing AGV scheduling management system is difficult to effectively integrate various data of AGVs and cannot fully explore the correlation and role of different data, resulting in a lack of comprehensive and accurate basis for scheduling decisions. In addition, AGV scheduling mostly relies on preset fixed rules, such as fixed route driving and fixed task assignment. This model cannot be adaptively adjusted according to the real-time status and dynamic changes of the workshop, resulting in path congestion, unclear distinction of task priorities and other problems, affecting the overall management rhythm. Therefore, an AGV intelligent scheduling and management system based on digital twins is proposed here. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned purpose, the present invention proposes the following technical solution: an AGV intelligent scheduling and management system based on digital twins, comprising: Data acquisition module: collects AGV environment data and task data based on multi-source sensors; Data fusion module: This module fuses AGV environment data and task data through an optimized deep network architecture to obtain initial fused data. It then calculates the frequency domain feature weights of the initial fused data and performs weighted fusion. Finally, it obtains scheduling status data through inverse Fourier transform. Digital twin module: This module uses the AGV digital twin to simulate and optimize the fused state data. It also simulates tasks in a virtual environment based on the actual layout of the workshop. It then calculates performance indicators under different scheduling strategies, obtains the optimal indicators, and develops an initial scheduling strategy. Advanced Scheduling Module: Builds a high-level policy reinforcement model based on reinforcement learning algorithms and policy networks, takes the initial scheduling policy as model input, and outputs the final optimized AGV scheduling solution; Car management module: Based on the AGV scheduling plan, it comprehensively manages the AGV cars.

[0005] The AGV environment data acquisition process is as follows: Laser radars are installed on the top and around the AGV. The laser is reflected back after encountering surrounding obstacles. By measuring the round-trip time of the laser, the relative distance between the obstacle and the AGV is calculated based on the speed of light. At the same time, combined with the rotation angle of the laser radar itself, the distance information is converted into polar coordinates or Cartesian coordinates to construct a three-dimensional point cloud map around the AGV. The three-dimensional point cloud map forms the AGV environmental data ; The process of obtaining AGV task data is as follows: The task identification device is installed in the operation area or cargo carrying part of the AGV. When the AGV approaches the task starting point or cargo, it automatically scans and analyzes the key information of the task. These key information are combined into the AGV task data. ; The key information includes task type, task destination, and task parameters.

[0006] The process of obtaining the initial fusion data is as follows: AGV environmental data and AGV mission data Merge as input data for deep network architectures ,Right now ; Convert it into matrix form for convolution layer input, The matrix form of A tensor of , where is the number of channels, and is the height and width of the data; Build an optimized deep network architecture. The construction process of the optimized deep network architecture is as follows: The optimized deep network architecture includes three parts: channel-wise convolution and interaction, adaptive weighted pooling, and fusion of fully connected layers; Channel-wise convolution and interaction include environment data channel convolution and task data channel convolution; The environmental data channel convolution is performed by determining the environmental data convolution parameters; According to the determined parameters and the initialized convolution kernel and bias, the environmental data channel is constructed, and the formula is expressed as: ,in, is the convolution kernel of the environment data channel convolution, is the bias term of the convolution of the environment data channel, Indicates the channel index; Perform convolution operation on the data through the environment data channel to generate the output features of the environment data channel convolution layer; The task data channel convolution is performed by determining the task data convolution parameters, which include the size of the first layer convolution kernel. , number of convolution kernels and step length ; According to the determined parameters and initialized convolution kernel and bias, the task data channel is constructed, and the formula is expressed as: ,in, is the convolution kernel of the task data channel convolution, is the bias term of the task data channel convolution, Indicates the channel index; Perform convolution operation on the data of the task data channel to generate the output features of the convolution layer of the task data channel; Fuse the environment data channel with the task data channel to obtain the fused output features , expressed as ,in, Represents the weight matrix, which is obtained by random initialization, by concatenating the data of the two channels in the channel dimension and multiplying it by the weight matrix , realize cross-channel feature fusion, and then form channel convolution and interaction parts; Adaptive weighted pooling includes Layer pooling layer, each layer pooling layer parameters include pooling pool size and step length , Indicates the pool size of the first pooling layer, Indicates the step size of the first pooling layer; Define a learning weight , output features based on fusion interaction , construct the weighted pooling operation formula: , where a is the channel index of the environment data channel, is the channel index of the task data channel, is the output of the first pooling layer in the adaptive weighted pooling process, then The output of the pooling layer is ; The fusion fully connected layer will unify the output after multi-layer adaptive weighted pooling into a one-dimensional vector , the length is ; The number of neurons in the fully connected layer is defined as , and initialize the weight matrix of the fully connected layer and bias , then the fully connected layer formula is expressed as: , where i is the index, ranging from 0 to between, Represents the output of the fully connected layer, that is, the initial fused data.

[0007] The process of obtaining the dispatch status data is as follows: Perform discrete Fourier transform on the initial fusion data, let Represents the discrete value of the initial fusion data in the time domain, through the discrete Fourier transform formula: ,in, represents the frequency components in the frequency domain, is the number of data points; Convert the initial fusion data from the time domain to the frequency domain to obtain the frequency domain data ,The purpose of this transformation is to analyze the data in the frequency ,domain in order to better explore the characteristics and regularities of the data; For each frequency component , defining its energy , and then calculate the frequency domain feature weights: ,in, For the The energy of the frequency components, After obtaining the frequency domain feature weights, the frequency domain data is weighted fused. Let the frequency domain data after weighted fusion be , perform inverse discrete Fourier transform on the frequency domain data after weighted fusion, and finally convert the frequency domain data back to the time domain to obtain the final scheduling status data .

[0008] The environmental data convolution parameters include the size of the first layer convolution kernel , number of convolution kernels and step length ; The task data convolution parameters include the size of the first layer convolution kernel , number of convolution kernels and step length .

[0009] The process of obtaining the initial scheduling strategy is as follows: Input the status data into the AGV digital twin system and build the digital twin based on the AGV parameters in the status data; Create a virtual environment model based on the actual layout of the workshop, define tasks in the virtual environment model, and each task includes the starting position, target position, and priority information. Initially assign tasks to the digital twin AGV according to the specified rules, and update the position and status of the AGV in the virtual environment at each time step. After assigning tasks, define performance indicators and design several scheduling strategies to be verified. Simulate them in a virtual environment and calculate the average value of each performance indicator under different scheduling strategies. , select the strategy with the best performance of key indicators as the initial scheduling strategy, expressed as: .

[0010] The construction process of the high-level policy reinforcement model is as follows: Define the state space of the high-level policy reinforcement model in the reinforcement learning process as ; State Space Including task distribution within the workshop and traffic flow conditions ,in, for Matrix, row vector Represents the number of task sites, column vector Indicates the number of task types, the data points in the matrix are represented by d, and the traffic flow conditions In vector form, it represents the AGV flow of each main channel; The actions of high-level strategies are defined as macro-scheduling instructions. Macro-scheduling instructions include regional task allocation strategies and overall traffic control strategies. Each macro-scheduling instruction constitutes an action space. ; Define a high-level policy reward function ,The reward function is obtained based on the workshop's production efficiency and task completion rate indicators, and is expressed as: ,in, is the number of completed workshop tasks, is the workshop production efficiency (such as output per unit time), is the degree of workshop collision, , , is the weight coefficient; Based on state space , from the action space through the policy network Output a macro scheduling action to explore the rate Select an action to The probability of randomly selecting one from all possible macro scheduling actions is The probability of selecting an action based on the output of the policy network; After executing the action, observe the changes in the workshop environment and obtain the macro information state of the workshop in the next time step. At the same time, calculate the immediate reward according to the reward function and convert the current state , selected action and instant rewards Perform policy gradient calculation and use the back propagation algorithm to calculate the policy gradient. The specific calculation process is as follows: First, obtain the probability distribution of the output action, expressed as , and then calculate the model's , obtain the policy gradient through the loss function , use the obtained policy gradient and learning rate to update the parameters of the high-level policy network , the update formula is: ,in, represents the learning rate; Observe the change of reward function value and calculate continuous The average value of the reward function within time steps , when the average value of the reward function If the change of is less than the preset threshold in consecutive time steps, the model is considered to have converged, and the parameter update of the model is stopped. The model completes convergence and obtains a high-level strategy reinforcement model.

[0011] The process of obtaining the AGV scheduling plan is as follows: The initial scheduling strategy As the input of the high-level policy reinforcement model, the model outputs several optimal macro-scheduling actions, which constitute the final optimized AGV scheduling plan.

[0012] The present invention has the following beneficial effects: 1. This invention first integrates various aspects of AGV data, including environmental data and task data, through data collection and fusion technology. A three-dimensional point cloud map is constructed using environmental data collected by a lidar, as well as task data acquired by a task recognition device. This data is then integrated through an optimized deep network architecture. First, through channel-wise convolution and interactive operations, the deep network can extract and analyze the different features of environmental data and task data separately, while allowing them to interact and fuse at different levels of the network, thereby achieving comprehensive integration of multiple data, enabling the scheduling system to simultaneously consider the physical environment of the workshop and task requirements. Secondly, during the fusion process, components such as the adaptive weighted pooling and fused fully connected layers of the deep network architecture automatically learn the associations and weights between different data. This allows them to discover the potential relationships between features in the environmental data and certain factors in the task data (such as the frequency and urgency of tasks passing through the area). This ensures that scheduling decisions are based on complete and accurate information, fully considering the interactions and constraints between various factors. In this way, subsequent scheduling decisions can better adapt to the actual situation on the shop floor. Finally, based on the combination of digital twins and reinforcement learning, it no longer relies on preset fixed rules. In a virtual environment, a model is created and tasks are defined based on the actual layout of the workshop. The initial scheduling strategy is determined through multiple simulations and performance indicator evaluations. Combined with the high-level strategy reinforcement model, it automatically outputs an optimized AGV scheduling plan based on real-time dynamic changes such as task distribution and traffic flow conditions in the workshop, including regional task allocation ratios and channel traffic priorities. This enables AGV scheduling to be adaptively adjusted according to the real-time status of the workshop, effectively solving problems such as path congestion and unclear task priority distinction caused by the inability of the existing model to adjust according to dynamic changes in the workshop, thereby improving the overall operating efficiency and smoothness of the workshop. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is the system block diagram of the digital twin-based AGV intelligent scheduling and management system proposed in this invention. DETAILED DESCRIPTION

[0014] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0015] Example 1, as Figure 1 As shown in the figure, the AGV intelligent scheduling and management system based on digital twin proposed by the present invention has a data acquisition module: it collects AGV environmental data and task data based on multi-source sensors; The process of acquiring AGV environmental data is as follows: Laser radars are installed on the top and around the AGV. The laser is reflected back after encountering surrounding obstacles. By measuring the round-trip time of the laser, the relative distance between the obstacle and the AGV is calculated based on the speed of light. At the same time, combined with the rotation angle of the laser radar itself, the distance information is converted into polar coordinates or Cartesian coordinates to construct a three-dimensional point cloud map around the AGV, which accurately reflects the distribution of obstacles. This three-dimensional point cloud map is the AGV environmental data. ; The process of obtaining AGV task data is as follows: A task identification device, such as a barcode scanner or electronic tag reader / writer, is installed in the operating area or cargo carrying area of the AGV. When the AGV approaches the task starting point or cargo, the device automatically scans the barcode, electronic tag, and other identifiers attached to the task list or cargo packaging, and parses the key information of the task, including the task type (such as handling, assembly, sorting, etc.), task destination, and task parameters (such as the weight and size requirements of the cargo to be handled). These data are combined into the AGV task data. .

[0016] Data fusion module: AGV environment data and task data are fused through an optimized deep network architecture to obtain initial fused data. The frequency domain feature weights of the initial fused data are calculated and weighted fused. Then, an inverse Fourier transform is performed to obtain scheduling status data. The process of obtaining the initial fusion data is as follows: AGV environment data and AGV mission data Merge as input data for deep network architectures ,Right now ; Convert it into matrix form for convolution layer input, The matrix form of A tensor of , where is the number of channels (for example, if the environment data and task data are separate channels), and is the height and width of the data ( That is, the dimension of the data, Width is data length); Specifically, Convert to matrix form The reason is that the environment data and task data can be regarded as different "channels", so the number of channels can be set here. (One channel is environment data, one channel is task data), then the environment data is one-dimensional, let , the data length W is the sum of the lengths of the environment data and the task data. Here we assume that there are 100 data vectors, that is, So, input data The matrix form of is In the deep network architecture, this tensor will serve as the input of the convolution layer. The convolution kernel will slide on this tensor to perform convolution operations and extract features. In this way, the deep network can process environmental data and task data at the same time and extract meaningful features for subsequent AGV scheduling and management. Build an optimized deep network architecture. The construction process of the optimized deep network architecture is as follows: The optimized deep network architecture includes three parts: channel-wise convolution and interaction, adaptive weighted pooling, and fusion of fully connected layers; Channel-wise convolution and interaction include environment data channel convolution and task data channel convolution; The environmental data channel convolution is performed by determining the environmental data convolution parameters, which include the size of the first layer convolution kernel. , number of convolution kernels and step length , these parameters are used to define the way the convolution operates; According to the determined parameters and the initialized convolution kernel and bias, the environmental data channel is constructed, and the formula is expressed as: ,in, is the convolution kernel of the environment data channel convolution, is the bias term of the convolution of the environment data channel, Indicates the channel index; Perform convolution operation on the data through the environment data channel to generate the output features of the environment data channel convolution layer; The task data channel convolution is performed by determining the convolution parameters, which include the size of the first layer convolution kernel. , number of convolution kernels and step length , these parameters are used to define the way the convolution operates; According to the determined parameters and initialized convolution kernel and bias, the task data channel is constructed, and the formula is expressed as: ,in, is the convolution kernel of the task data channel convolution, is the bias term of the task data channel convolution, Indicates the channel index; Perform convolution operation on the data of the task data channel to generate the output features of the convolution layer of the task data channel; Fuse the environment data channel with the task data channel to obtain the fused output features , expressed as ,in, Represents the weight matrix, which is obtained by random initialization, by concatenating the data of the two channels in the channel dimension and multiplying it by the weight matrix , realizing cross-channel feature fusion, thus forming the channel-wise convolution and interaction parts; Adaptive weighted pooling includes Layer pooling layer, each pooling layer (here the first pooling layer is taken as an example) has parameters including pooling pool size and step length , Indicates the pool size of the first pooling layer, Indicates the step size of the first pooling layer; Define a learning weight , output features based on fusion interaction , construct the weighted pooling operation formula: , where a is the channel index of the environment data channel, is the channel index of the task data channel, is the output of the first pooling layer in the adaptive weighted pooling process, then The output of the pooling layer is ; The fusion fully connected layer will unify the output after multi-layer adaptive weighted pooling into a one-dimensional vector , the length is ; The number of neurons in the fully connected layer is defined as , and initialize the weight matrix of the fully connected layer and bias , then the fully connected layer formula is expressed as: , where i is the index, ranging from 0 to between, Represents the output of the fully connected layer, that is, the initial fusion data; Specifically, the purpose of the data fusion module is to fuse the effective features in the AGV environmental data and task data to obtain status data that can reflect the comprehensive status of the AGV. This process is achieved through an optimized deep network architecture. In this network architecture, the environmental data and task data are first input into the network. The role of the fully connected layer is that the fully connected layer plays the role of integrating features in the deep network. After the previous channel-wise convolution and interaction, and adaptive weighted pooling operations, the features of the data are gradually extracted and abstracted. The fully connected layer comprehensively processes these features and adds a bias term to obtain a new vector. The channel-wise convolution and interaction, and adaptive weighted pooling operations are to extract and initially fuse the features of the original environmental data and task data, while the fully connected layer performs the final comprehensive processing based on these features. The final output is the feature information of the environmental data and task data extracted and processed in the network. Therefore, it can be regarded as a fusion representation of the environmental data and task data, that is, the initial fusion data. The process of obtaining scheduling status data is as follows: Perform discrete Fourier transform on the initial fusion data, let Represents the discrete value of the initial fusion data in the time domain, through the discrete Fourier transform formula: ,in, represents the frequency components in the frequency domain, is the number of data points; Convert the initial fusion data from the time domain to the frequency domain to obtain the frequency domain data ,The purpose of this transformation is to analyze the data in the frequency ,domain in order to better explore the characteristics and regularities of the data; For each frequency component , defining its energy , and then calculate the frequency domain feature weights: ,in, For the The energy of the frequency components, Specifically, the weight is calculated by dividing the energy of each frequency component by the sum of the energies of all frequency components. The weight obtained in this way can reflect the relative importance of each frequency component in the entire frequency domain. In this way, important frequency features can be highlighted and relatively unimportant features can be suppressed, thereby better performing data fusion and feature extraction. After obtaining the frequency domain feature weights, the frequency domain data is weighted fused. Let the frequency domain data after weighted fusion be , perform inverse discrete Fourier transform on the frequency domain data after weighted fusion, and finally convert the frequency domain data back to the time domain to obtain the final scheduling status data ; Specifically, the frequency domain information after weighted fusion is restored to time domain information, and the obtained scheduling status data is It integrates the information of AGV environmental data and task data, and through frequency domain analysis and weighted processing, it can more accurately reflect the status of the AGV car, providing more valuable data support for subsequent AGV scheduling and management.

[0017] Digital twin module: This module uses the AGV digital twin to simulate and optimize the fused state data. It also simulates tasks in a virtual environment based on the actual layout of the workshop. It then calculates performance indicators under different scheduling strategies, obtains the optimal indicators, and develops an initial scheduling strategy. The process of obtaining the initial scheduling strategy is as follows: Input the status data into the AGV digital twin system and build the digital twin based on the AGV parameters in the status data; Specifically, the AGV parameters in the status data include information such as the position coordinates, speed, and final destination of the AGV in the status data obtained from the environmental data and the task data. These information are all data points in the status data obtained from the environmental data and the task data. Create a virtual environment model based on the actual layout of the workshop, define tasks in the virtual environment model, and each task includes the starting position, target position, and priority information. Initially assign tasks to the digital twin AGV according to the specified rules, and update the position and status of the AGV in the virtual environment at each time step. Specifically, the specified rule is to assign tasks according to the order in which they arrive, and the tasks that arrive first are assigned to the idle AGVs first; Implementation steps: Maintain a task queue. When a new task arrives, add it to the end of the queue. When an AGV is idle, take the task from the head of the task queue for assignment. For example, a task arrives first and enters the task queue; then a task arrives and also enters the task queue. At this time, the task is at the head of the task queue, and the task is at the tail of the task queue. When an AGV is idle, assign the task to the AGV, and then move the task to the head of the queue and wait for the next idle AGV. After assigning tasks, performance indicators are defined, including task completion time, path length, energy consumption, and task delay rate. Then, several scheduling strategies to be verified are designed, including the shortest path, task priority, energy consumption decision-making, and task delay decision-making. For each strategy, multiple simulations are performed in a virtual environment, and the performance indicator values of each simulation are recorded and the average value is calculated. , the average value of each performance indicator under different strategies , select the strategy with the best performance of key indicators as the initial scheduling strategy, expressed as: ; Specifically, are different initial scheduling strategies, for example, represents the shortest path strategy, represents the task priority strategy, Indicates the number of initial scheduling strategies, that is, the number of strategies included in the set; By simulating these different initial scheduling strategies multiple times in a virtual environment, recording the performance indicator values of each simulation (such as task completion time, path length, energy consumption, task delay rate, etc.), and calculating the average value of each performance indicator, and then comparing the average values of each performance indicator under different strategies, the strategy with the best performance in key indicators is selected as the final initial scheduling strategy. This set covers all the initial scheduling strategies that can be selected and evaluated, and is the basis of the entire scheduling strategy selection and optimization process.

[0018] Advanced Scheduling Module: Builds a high-level policy reinforcement model based on reinforcement learning algorithms and policy networks, takes the initial scheduling policy as model input, and outputs the final optimized AGV scheduling solution; The process of obtaining the AGV scheduling plan is as follows: Build a high-level strategy reinforcement model. The construction process of the high-level strategy reinforcement model is as follows: Define the state space of the high-level policy reinforcement model in the reinforcement learning process as ; State Space Including task distribution within the workshop and traffic flow conditions ,in, for Matrix, row vector Represents the number of task sites, column vector Indicates the number of task types, the data points in the matrix are represented by d, and the traffic flow conditions In vector form, it represents the AGV flow of each main channel; For example, a workshop has 2 task sites and 3 task types. The first task site indicates that there is 1 type 1 task, 1 type 2 task, and no type 3 task. The second task site indicates that there is 1 type 1 task, no type 2 task, and 1 type 3 task. The AGV flow rates of the three main channels are 5, 3, and 2 vehicles per minute respectively; The actions of high-level strategies are defined as macro-scheduling instructions. Macro-scheduling instructions include regional task allocation strategies (for example, dividing a workshop into several areas and determining the proportion of tasks allocated to each area) and overall traffic control strategies (such as setting the priority of certain channels). Each macro-scheduling instruction constitutes an action space. ; Define a high-level policy reward function ,The reward function is obtained based on the workshop's production efficiency and task completion rate indicators, and is expressed as: ,in, is the number of completed workshop tasks, is the workshop production efficiency (such as output per unit time), is the degree of workshop collision, , , is the weight coefficient; Specifically, , , All of them are obtained through the digital twin simulation process; For example: In the process of acquiring digital twins, various tasks are defined in the virtual environment, such as handling and assembly. Each task has clear information such as starting position, target position and priority. When the AGV digital twin performs tasks according to the preset rules, the system will track the status of each task in real time and count the number of completed tasks. ; Similarly, In the process of acquiring data, during the digital twin simulation, for different types of tasks, taking the transportation task as an example, the output efficiency is defined as the number of goods transported per unit time or the total volume of goods; for the assembly task, the output efficiency is defined as the number of assembled products completed per unit time; During the digital twin simulation, for each AGV digital twin, the distance between it and other AGV digital twins and obstacles in the virtual environment is calculated. If the distance is less than the preset safety distance (determined according to the size and safety requirements of the AGV), a collision is considered to have occurred. Based on state space , from the action space through the policy network Output a macro scheduling action to explore the rate Select an action to The probability of randomly selecting one from all possible macro scheduling actions is The probability of selecting an action based on the output of the policy network; After executing the action, observe the changes in the workshop environment and obtain the macro information state of the workshop in the next time step. At the same time, calculate the immediate reward according to the reward function and convert the current state , selected action and instant rewards Perform policy gradient calculation and use the back propagation algorithm to calculate the policy gradient. The specific calculation process is as follows: First, obtain the probability distribution of the output action, expressed as , and then calculate the model's , obtain the policy gradient through the loss function , use the obtained policy gradient and learning rate to update the parameters of the high-level policy network , the update formula is: ,in, represents the learning rate; Observe the change of reward function value and calculate continuous The average value of the reward function within time steps If the change in multiple consecutive time windows (such as 20 windows) is less than the preset threshold, the model is considered to have converged, the parameter update of the model is stopped, and the model completes convergence; When the model converges, training is stopped. At this time, the high-level policy reinforcement model obtained has learned the optimal strategy under the current workshop environment and reward function settings. The model can output several optimal macro-scheduling actions (such as regional task allocation ratio, channel access priority, etc.) based on the input initial scheduling strategy. These outputs constitute the final optimized AGV scheduling plan, which can be used to guide the scheduling of AGVs in actual workshops to achieve efficient production and logistics operations and improve the overall performance and resource utilization of the workshop.

[0019] Car management module: Based on the AGV scheduling plan, it comprehensively manages the AGV cars; When a new task arrives at the workshop, the model is strengthened by the trained high-level strategy, and the model outputs a task allocation plan. The model outputs the regional task allocation ratio based on the current workshop status (including task distribution and traffic flow). For example: The model output divides the workshop into 3 areas. The current new task should be based on 、 、 For each area, the tasks assigned to the area are further assigned to the appropriate AGVs according to the current status of the AGVs in the area (such as location, power, task progress, etc., which can be obtained through the AGV positioning system and status monitoring system). Priority is given to assigning tasks to AGVs that are closer to the starting location of the task, have sufficient power, and whose task progress allows them to take on new tasks. For example, there are 5 AGVs in area 1, of which the second The one closest to the starting location of the new task and with sufficient power and task progress allowed will be assigned to the part of the task assigned to the area. ; Path planning: Based on the AGV scheduling plan output by the model, the path planning of the AGV is adjusted in real time. When multiple AGVs may pass through the same channel at the same time, the AGV with higher priority will pass first. The positioning system in the workshop is used to obtain the location information of the AGV in real time. Combined with the AGV scheduling plan, the driving path of the AGV is dynamically planned to avoid congested areas and obstacles, ensuring a smooth and efficient path.

[0020] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. The AGV intelligent dispatching and management system based on digital twins is characterized by: include: Data acquisition module: collects AGV environment data and task data based on multi-source sensors; Data fusion module: This module fuses AGV environment data and task data through an optimized deep network architecture to obtain initial fused data. It then calculates the frequency domain feature weights of the initial fused data and performs weighted fusion. Finally, it obtains scheduling status data through inverse Fourier transform. Digital Twin Module: This module simulates and optimizes the fused scheduling status data through the AGV digital twin, and simulates tasks in a virtual environment based on the actual layout of the workshop. It then calculates performance indicators under different scheduling strategies, obtains the optimal indicators, and develops the initial scheduling strategy. Advanced Scheduling Module: Builds a high-level policy reinforcement model based on reinforcement learning algorithms and policy networks, takes the initial scheduling policy as model input, and outputs the final optimized AGV scheduling solution; Car management module: manages AGV cars based on the AGV scheduling plan.

2. The AGV intelligent dispatching and management system based on digital twin according to claim 1 is characterized in that: The AGV environment data acquisition process is as follows: Laser radars are installed on the top and around the AGV. The laser is reflected back after encountering surrounding obstacles. By measuring the round-trip time of the laser, the relative distance between the obstacle and the AGV is calculated based on the speed of light. At the same time, combined with the rotation angle of the laser radar itself, the distance information is converted into polar coordinates or Cartesian coordinates to construct a three-dimensional point cloud map around the AGV. The three-dimensional point cloud map forms the AGV environmental data ; The process of obtaining AGV task data is as follows: The task identification device is installed in the operation area or cargo carrying part of the AGV. When the AGV approaches the task starting point or cargo, it automatically scans and analyzes the key information of the task and combines it into AGV task data. ; The key information includes task type, task destination, and task parameters.

3. The AGV intelligent dispatching and management system based on digital twin according to claim 1 is characterized in that: The process of obtaining the initial fusion data is as follows: AGV environment data and AGV mission data Merge as input data for deep network architectures ,Right now ; Convert it into matrix form for convolution layer input, The matrix form of A tensor of , where is the number of channels, and is the height and width of the data; Build an optimized deep network architecture. The construction process of the optimized deep network architecture is as follows: The optimized deep network architecture includes three parts: channel-wise convolution and interaction, adaptive weighted pooling, and fusion of fully connected layers; Channel-wise convolution and interaction include environment data channel convolution and task data channel convolution; The environmental data channel convolution is performed by determining the environmental data convolution parameters; According to the determined parameters and the initialized convolution kernel and bias, the environmental data channel is constructed, and the formula is expressed as: ,in, is the convolution kernel of the environment data channel convolution, is the bias term of the convolution of the environment data channel, Indicates the channel index; Perform convolution operation on the data through the environment data channel to generate the output features of the environment data channel convolution layer; The task data channel convolution is performed by determining the task data convolution parameters, which include the size of the first layer convolution kernel. , number of convolution kernels and step length ; According to the determined parameters and initialized convolution kernel and bias, the task data channel is constructed, and the formula is expressed as: ,in, is the convolution kernel of the task data channel convolution, is the bias term of the task data channel convolution, Indicates the channel index; Perform convolution operation on the data of the task data channel to generate the output features of the convolution layer of the task data channel; Fuse the environment data channel with the task data channel to obtain the fused output features , expressed as ,in, Represents the weight matrix, which is obtained by random initialization, by concatenating the data of the two channels in the channel dimension and multiplying it by the weight matrix , realize cross-channel feature fusion, and then form channel convolution and interaction parts; Adaptive weighted pooling includes Layer pooling layer, each layer pooling layer parameters include pooling pool size and step length , Indicates the pool size of the first pooling layer, Indicates the step size of the first pooling layer; Define a learning weight , output features based on fusion interaction , construct the weighted pooling operation formula: , where a is the channel index of the environment data channel, is the channel index of the task data channel, is the output of the first pooling layer in the adaptive weighted pooling process, then The output of the pooling layer is ; The fusion fully connected layer will unify the output after multi-layer adaptive weighted pooling into a one-dimensional vector , the length is the number of pooling layers ; The number of neurons in the fully connected layer is defined as , and initialize the weight matrix of the fully connected layer and bias , then the fully connected layer formula is expressed as: , where i is the index, ranging from 0 to between, Represents the output of the fully connected layer, that is, the initial fused data.

4. The AGV intelligent dispatching and management system based on digital twin according to claim 3 is characterized in that: The process of obtaining the dispatch status data is as follows: Perform discrete Fourier transform on the initial fusion data, let Represents the discrete value of the initial fusion data in the time domain, through the discrete Fourier transform formula: ,in, represents the frequency components in the frequency domain, is the number of data points; Convert the initial fusion data from the time domain to the frequency domain to obtain the frequency domain data ; For each frequency component , defining its energy , and then calculate the frequency domain feature weights: ,in, For the The energy of the frequency components, After obtaining the frequency domain feature weights, the frequency domain data is weighted fused. Let the frequency domain data after weighted fusion be , perform inverse discrete Fourier transform on the frequency domain data after weighted fusion, and finally convert the frequency domain data back to the time domain to obtain the final scheduling status data .

5. The AGV intelligent dispatching and management system based on digital twin according to claim 3 is characterized in that: The environmental data convolution parameters include the size of the first layer convolution kernel , number of convolution kernels and step length ; The task data convolution parameters include the size of the first layer convolution kernel , number of convolution kernels and step length .

6. The AGV intelligent dispatching and management system based on digital twin according to claim 1 is characterized in that: The process of obtaining the initial scheduling strategy is as follows: Input the status data into the AGV digital twin system and build the digital twin based on the AGV parameters in the status data; Create a virtual environment model based on the actual layout of the workshop, define tasks in the virtual environment model, and each task includes the starting position, target position, and priority information. Initially assign tasks to the digital twin AGV according to the specified rules, and update the position and status of the AGV in the virtual environment at each time step. After assigning tasks, define performance indicators and design several scheduling strategies to be verified. Simulate in a virtual environment and calculate the average value of each performance indicator under different scheduling strategies. , select the strategy with the best performance of key indicators as the initial scheduling strategy, expressed as: .

7. The AGV intelligent dispatching and management system based on digital twin according to claim 1 is characterized in that: The construction process of the high-level policy reinforcement model is as follows: Define the state space of the high-level policy reinforcement model in the reinforcement learning process as ; State Space Including task distribution within the workshop and traffic flow conditions ,in, for Matrix, row vector Represents the number of task sites, column vector Indicates the number of task types, the data points in the matrix are represented by d, and the traffic flow conditions In vector form, it represents the AGV flow of each main channel; The actions of high-level strategies are defined as macro-scheduling instructions. Macro-scheduling instructions include regional task allocation strategies and overall traffic control strategies. Each macro-scheduling instruction constitutes an action space. ; Define a high-level policy reward function ,The reward function is obtained based on the workshop's production efficiency and task completion rate indicators, and is expressed as: ,in, is the number of completed tasks in the workshop, is the workshop production efficiency, is the degree of workshop collision, , , is the weight coefficient; Based on state space , from the action space through the policy network Output a macro scheduling action to explore the rate Select an action to The probability of randomly selecting one from all possible macro scheduling actions is The probability of selecting an action based on the output of the policy network; After executing the action, observe the changes in the workshop environment and obtain the macro information state of the workshop in the next time step. At the same time, calculate the immediate reward according to the reward function and convert the current state , selected action and instant rewards Perform policy gradient calculation and use the back propagation algorithm to calculate the policy gradient. The specific calculation process is as follows: First, obtain the probability distribution of the output action, expressed as , and then calculate the model's , obtain the policy gradient through the loss function , use the obtained policy gradient and learning rate to update the parameters of the high-level policy network , the update formula is: ,in, represents the learning rate; Observe the change of reward function value and calculate continuous The average value of the reward function within time steps , when the average value of the reward function If the change of is less than the preset threshold in consecutive time steps, the model is considered to have converged, and the parameter update of the model is stopped. The model completes convergence and obtains a high-level strategy reinforcement model.

8. The AGV intelligent dispatching and management system based on digital twin according to claim 7 is characterized in that: The process of obtaining the AGV scheduling plan is as follows: The initial scheduling strategy As the input of the high-level policy reinforcement model, the model outputs several optimal macro-scheduling actions, which constitute the final optimized AGV scheduling plan.

Images