Intelligent scheduling system for precise distribution of discrete workshops based on digital twins

Abstract

The present invention discloses an intelligent scheduling system for precise distribution of discrete workshops based on digital twins. The scheduling system includes a physical workshop layer; a workshop data layer, which is the data collected from entities in the workshop and transmitted to the virtual workshop layer; after data fusion, the data generated by different entities are converted into corresponding data formats and stored in a MySQL database; a virtual workshop layer, which is a virtual workshop model based on an entity. The virtual workshop layer includes an OPC UA information model, and the overall workshop information model is divided into five types of information concept models based on the MySQL database. These five types of information concept models and the virtual workshop model all have a local-to-global relationship. Then, according to the tasks and plans in the workshop, several task objects are instantiated to realize the description of the tasks in the workshop; in the application service layer, a multi-agent-based reinforcement learning logistics scheduling algorithm is established, and a real-time scheduling model for the distribution system is established to realize the generation of precise distribution paths for AGVs.

Description

Technical Field

[0001] The present invention relates to the field of digital twin technology, and in particular to a discrete workshop precise distribution intelligent scheduling system based on digital twin. Background Art

[0002] With the continuous development of next-generation information and communication technologies, production workshops are becoming increasingly intelligent. Leveraging advanced technologies such as the Internet of Things, artificial intelligence, and big data to provide intelligent scheduling support for production and distribution, ensuring timely delivery and improving customer satisfaction, has become a key issue in enhancing corporate competitiveness. Discrete workshops offer flexible production paths, effectively meeting the demands of high-variety, small-batch production, making them a widely used workshop model. However, with increasing personalization of product demands and increasing diversity in manufacturing processes, the logistics scheduling problem in discrete workshops has become increasingly complex. Manufacturing companies are placing higher demands on autonomous, intelligent, and predictive solutions to these problems. Enabling interconnectivity and intelligent operation between the physical and cyberspaces of manufacturing is key to achieving intelligent manufacturing and intelligent scheduling. Digital twins utilize digital methods to model physical objects with high fidelity, simulating their characteristics and behaviors in a virtual space. Through virtual-physical interaction, data fusion and value-added, knowledge mining, iterative optimization, and combined with reinforcement learning-based precision distribution scheduling algorithms, they enable more autonomous, predictive, and intelligent precise workshop distribution scheduling. Summary of the Invention

[0003] In response to the problems and shortcomings in the existing technology, the purpose of the present invention is to provide an intelligent scheduling system for precise distribution of discrete workshops based on digital twins, which combines reinforcement learning technology, database technology, digital twin modeling technology, etc. to realize precise distribution of materials in the self-organized production process of discrete workshops.

[0004] To this end, the present invention is implemented through the following technical solutions: a discrete workshop precise distribution intelligent scheduling system based on digital twins, wherein the scheduling system includes a physical workshop layer, and the physical workshop layer includes entities in the workshop;

[0005] The workshop data layer contains data collected from entities within the workshop. The data is obtained from the entities and transmitted to the virtual workshop layer. The data is decoded by the decoder and then handed over to the RFID middleware for processing and fusion. The data generated by different entities is then converted into the corresponding data format and stored in the MySQL database.

[0006] The virtual workshop layer is an entity-based virtual workshop model that includes an OPC UA information model. Based on a MySQL database, the overall workshop information model is divided into five types of information concept models. These five types of information concept models and the virtual workshop model all have a local-global relationship. For any processing workshop, the corresponding objects of the five types of information concept models are first instantiated based on the collected entity data to represent all production factors in the workshop. Then, based on the tasks and plans in the workshop, several task objects are instantiated to describe the tasks in the workshop.

[0007] At the application service layer, the application service layer reads the data generated by the workshop data layer and the workshop task description of the simulation workshop layer, establishes a multi-agent-based reinforcement learning logistics scheduling algorithm, establishes a real-time scheduling model for the distribution system, and realizes the generation of precise AGV distribution paths.

[0008] Furthermore, the five types of information concept models are divided into equipment information concept model, personnel information concept model, material information concept model, environment information concept model and order task information concept model.

[0009] Furthermore, the physical workshop layer includes machine tools, AGVs, robotic arms, automatic storage, sensors, and radio frequency identification (RFID) devices, which are used to receive manufacturing instructions and execute manufacturing tasks, and perceive and process heterogeneous, multi-source, and real-time data based on the Internet of Things.

[0010] Furthermore, the multi-agent reinforcement learning-based logistics scheduling algorithm model includes a path scheduling optimization model and a multi-agent reinforcement learning algorithm; wherein,

[0011] The path scheduling optimization model is established through the following process:

[0012] Set the model to a fully connected graph G(B), where the coordinate vector B = {v0, v1, ...v N ), v1, v2, …, vN are the physical coordinates of the workshop;

[0013] When an order is received, a demand instance j = (v, d, e, l, α, β) is generated, where v represents the physical coordinates of the workshop and d represents the demand instance. i , the time window is (e i , l i ), penalty coefficient α i , β i ;

[0014] Assume that M AGVs are allocated to meet the order requirements, and generate a vertex-disjoint route r with the lowest path cost for each AGV. m, m=1,2,...M, the starting point and the end point are both in warehouse v0, the path distance is expressed as the Euclidean distance on the plane, the speed of all vehicles is set to be the same, and the model goal is to find a solution r=[r1,r2,...,r M ], the total cost is defined as:

[0015] C(r)=Γ sum (r)+p sum (r) (1)

[0016] in is the total cost of all vehicles, I is the identity matrix, which represents the total penalty of the time window constraint.

[0017] Furthermore, the multi-agent reinforcement learning algorithm is implemented based on the DQN method:

[0018] First, the problem model is established according to the Markov decision process, that is, the environment state, action and reward are defined. The environment state includes a five-dimensional vector S composed of the workshop state and the AGV state. t =(N t , T, D, A st , A vt ), including the operating status of each equipment in the workshop and the operating status and location information of each AGV, N t represents the current order number, T represents the remaining time of the current task, D represents the distance from the current target point, A st Indicates the current working status of the AGV car, A vt Indicates the movement speed of the AGV car;

[0019] Define the action vector a t =(J t AGV t ), where J t Representative workstation, AGV t Represents the AGV number that performs the task. When the AGV completes one step, it compares the sample cost and gives a small reward feedback. When the AGV completes all steps, it compares the total cost and gives a large reward feedback.

[0020] Finally, the reward obtained is converted into a final reward value through a time evaluation function and fed back to the path scheduling optimization model to obtain the optimal total cost C(r).

[0021] Beneficial effects:

[0022] This paper builds a precise distribution intelligent scheduling system for discrete workshops based on reinforcement learning technology and digital twin modeling technology. Its advantages are:

[0023] 1. The system adopts a design pattern of separation of view, model and control, which facilitates the function expansion and daily maintenance of the system.

[0024] 2. Use the OPC UA information modeling method to realize data interaction, unify the protocols of various devices through the OPC UA protocol, establish a standardized interface for data transmission, and present information in different formats generated by different types of devices in a standard format to achieve effective data collection and management, so that multi-source heterogeneous data can be integrated and transmitted in the digital twin platform.

[0025] 3. Utilize the reinforcement learning method in machine learning to establish a real-time dispatching model for the distribution system, so that the model has the ability to learn, predict, and adjust. The distribution system is no longer an automated system that relies solely on rules to operate.

[0026] 4. Introduce detection and monitoring equipment to collect relevant parameters of distribution operations and relevant data on workshop operations. The data comes from real data updated in real time in the workshop. Based on relevant methods of big data analysis, the operation status of distribution operations is analyzed, which can effectively respond to disturbances such as equipment failures and emergency orders, and provide solutions for system scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of the architecture of the intelligent scheduling system for precise distribution of discrete workshops based on digital twins;

[0028] Figure 2 This is the process logistics perception flow chart based on RFID;

[0029] Figure 3 Schematic diagram of the OPC UA information model for the virtual twin workshop;

[0030] Figure 4 Dynamically drive process diagrams for virtual shop floor models;

[0031] Figure 5 Schematic diagram of the reinforcement learning workshop scheduling algorithm model for precise distribution. DETAILED DESCRIPTION

[0032] The present invention will be further described below with reference to the accompanying drawings.

[0033] This application proposes a discrete workshop precise distribution intelligent scheduling system based on digital twins. Figure 1As shown, the system architecture is divided into three layers: the first layer is the physical workshop layer, which refers to the entities in the workshop, such as machine tools, AGVs, robotic arms, automatic storage, sensors, radio frequency identification (RFID), etc., which are used to receive manufacturing instructions and perform manufacturing tasks, while sensing and processing heterogeneous, multi-source, and real-time data based on the Internet of Things; the second layer is the virtual workshop layer, which refers to the virtual workshop model, real-time workshop mapping, workshop information display, etc.; the third layer is the workshop data layer, which serves as a data source and transmission channel; the fourth layer is the application service layer, which refers to a collection of technologies that support intelligent production management and control functions and target requirements. Figures 2 to 5 The specific definition and implementation of each layer are introduced respectively.

[0034] like Figure 2 Figure 2 shows the RFID-based process logistics perception flow. Workshop data collection is achieved through RFID technology, which uses radio signals to identify targets and read the data stored on the tags. Information is transmitted through spatial coupling. In an RFID system, RFID tags store detailed information about the identified object, such as name, model, quantity, and purpose. The antenna emits a radio frequency signal, which is sensed by the reader / writer and automatically generates an electromagnetic field. When an RFID tag enters the area covered by the radio frequency signal, the reader / writer, under the influence of the electromagnetic field, automatically retrieves the information from the RFID tag through the antenna. The reader / writer then decodes the information using a decoder and passes it to the RFID middleware for processing and integration. The processed information is then sent to the data collection module, which converts the information into the appropriate data format and stores it in a MySQL database.

[0035] like Figure 3 Figure 1 shows the OPC UA information model for the virtual twin workshop. Based on the concept of physical information fusion, the overall workshop information model is divided into five information concept models: the device information concept model (Device-Inf), the personnel information concept model (Person-Inf), the material information concept model (Material-Inf), the environment information concept model (Environment-Inf), and the order task information concept model (Task-Inf). These five information concept models all have a part-of-whole relationship with the overall model. For a processing workshop, it is necessary to first instantiate several Device-Inf, Person-Inf, Material-Inf, and Environment-Inf objects based on the collected data to represent all the workshop's production factors. Then, based on the workshop's tasks and plans, several Task-Inf objects are instantiated to describe the workshop's tasks.

[0036] like Figure 4As shown in the figure, the dynamic driving process of the virtual workshop model is oriented. After real-time data is collected from physical workshop objects, the real-time database classifies, labels, and stores this data. The model then reads the corresponding data signal to perform the corresponding action. The virtual workshop model is integrated hierarchically. The same model can contain multiple sub-models. The action of the entire model is a composite action driven by all the sub-models it contains. After reading the material access signal, the warehouse model will perform the corresponding material access action. The AGV model will perform the corresponding action when driven by data signals such as the next point, target point, and material loading and unloading. The robotic arm model needs to be driven by data such as the corresponding axis angle, gripping model, and gripping position. Various processing workshop models are driven by corresponding data signals such as spindle speed, fixture signal, start and stop signal, and operation status information.

[0037] like Figure 5 As shown in the figure, a logistics scheduling algorithm model based on multi-agent reinforcement learning is presented. The algorithm model consists of a path scheduling optimization model and a multi-agent reinforcement learning algorithm. The path scheduling optimization model regards the road grid as a fully connected graph G(B), where B = {v0v1, ...v N The order is represented by a vector group j = (v, d, e, l, α, β), where v represents the physical coordinates of the workshop and the demand is d. i , the time window is (e i , l i ), and the penalty coefficient α i , β i When an order is received, an instance j = (v, d, e, l, α, β) is generated. The decision maker will allocate M AGVs to meet the order requirements and generate a vertex-disjoint route r with the lowest path cost for each AGV. m , m=1,2,...M, whose starting and ending points are both at warehouse v0, and the path distances are all expressed using the Euclidean distance on the plane, assuming that all vehicles have the same speed. The model goal is to find a solution r=[r1,r2,...,r M ], the total cost is defined as:

[0038] C(r)=Γ sum (r)+p sum (r) (1)

[0039] in is the total cost of all vehicles, represents the total penalty for the time window constraint.

[0040] The multi-agent reinforcement learning algorithm is implemented based on the DQN method. First, the problem model is established according to the Markov decision process, that is, the environment (State), action (Action) and reward (Reward) are defined. The environment state is mainly composed of the five-dimensional vector S composed of the workshop state and the AGV state. t =(N t , T, D, A st , A vt ), including the operating status of each equipment in the workshop and the operating status and location information of each AGV. t represents the current order number, T represents the remaining time of the current task, D represents the distance from the current target point, A st Indicates the current working status of the AGV car, A vt Indicates the movement speed of the AGV. Define the action vector α t =(J t AGV t ), where J t Representative workstations, there are six in total, AGV t Represents the AGV number performing the task. When the AGV completes a step, it is compared to the sample cost and given a small reward. When the AGV completes all steps, it is compared to the total cost and given a larger reward. Finally, the reward is converted into a final reward value using a time evaluation function and fed back to the model. An example reward function is: R = 1 + 1 + … + 10. This algorithm is used to perform unsupervised training on the path scheduling optimization model to obtain optimal parameters.

[0041] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of implementation of the present invention. Several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. The discrete workshop precise distribution intelligent scheduling system based on digital twin is characterized by: The scheduling system includes a physical shop floor, which includes entities within the job shop; The workshop data layer is the data collected from the entities in the workshop. The data is obtained from the entities and transmitted to the virtual workshop layer; The data is decoded by the decoder and then handed over to the RFID middleware for processing and fusion. The data generated by different entities are then converted into the corresponding data format and stored in the MySQL database; The virtual workshop layer is an entity-based virtual workshop model. It includes the OPC UA information model and divides the overall workshop information model into five types of information concept models based on the MySQL database. These five types of information concept models and the virtual workshop model all have a local-global relationship. For any processing workshop, the corresponding objects of the five types of information concept models are first instantiated based on the collected entity data to represent all production factors of the workshop; then, based on the tasks and plans in the workshop, several task objects are instantiated to describe the tasks in the workshop; At the application service layer, the application service layer reads the data generated by the workshop data layer and the workshop task description of the simulation workshop layer, establishes a multi-agent-based reinforcement learning logistics scheduling algorithm, establishes a real-time scheduling model for the distribution system, and realizes the generation of accurate AGV distribution paths; The multi-agent reinforcement learning-based logistics scheduling algorithm model includes a path scheduling optimization model and a multi-agent reinforcement learning algorithm; wherein, The path scheduling optimization model is established through the following process: Set the model to a fully connected graph G(B), where the coordinate vector B = {v0,v1,...v N }, v0, v1, ... v N is the physical coordinate of the workshop; When an order is received, a demand instance j = (v, d, e, l, α, β) is generated, where v represents the physical coordinates of the workshop and d represents the demand. i , the time window is (e i ,l i ), penalty coefficient α i ,β i ; Assume that M AGVs are allocated to meet the order requirements, and generate a vertex-disjoint route r with the lowest path cost for each AGV. m ,m=1,2,...M, the starting point and the end point are both in warehouse v0, the path distance is expressed using the Euclidean distance on the plane, and the speed of all vehicles is set to be the same. The model goal is to find a solution with the minimum total cost r=[r1,r2,...,r M ], the total cost is defined as: C(r)=Γ sum (r)+p sum (r) (1) in is the total cost of all vehicles, I is the identity matrix, representing the total penalty of the time window constraint; The multi-agent reinforcement learning algorithm is implemented based on the DQN method: First, the problem model is established according to the Markov decision process, that is, the environment state, action and reward are defined. The environment state includes a five-dimensional vector S composed of the workshop state and the AGV state. t =(N t ,T,D,A st ,A vt ), including the operating status of each equipment in the workshop and the operating status and location information of each AGV, N t represents the current order number, T represents the remaining time of the current task, D represents the distance from the current target point, A st Indicates the current working status of the AGV car, A vt Indicates the movement speed of the AGV car; Define the action vector a t =(J t AGV t ), where J t Representative workstation, AGV t Represents the AGV number that performs the task. When the AGV completes one step, it compares the sample cost and gives a small reward feedback. When the AGV completes all steps, it compares the total cost and gives a large reward feedback. Finally, the reward obtained is converted into a final reward value through a time evaluation function and fed back to the path scheduling optimization model to obtain the optimal total cost C(r).

2. The discrete workshop precise distribution intelligent scheduling system based on digital twin according to claim 1 is characterized in that: The five types of information concept models are divided into equipment information concept model, personnel information concept model, material information concept model, environment information concept model and order task information concept model.

3. The digital twin-based discrete workshop precise distribution intelligent scheduling system according to claim 1 is characterized in that: The physical workshop layer contains machine tools, AGVs, robotic arms, automatic storage, sensors, and radio frequency identification (RFID) devices, which are used to receive manufacturing instructions and execute manufacturing tasks, and perceive and process heterogeneous, multi-source, and real-time data based on the Internet of Things.

Images