An AGV scheduling system and method for 3C smart manufacturing workshops considering buffer capacity

Abstract

A 3C smart manufacturing workshop AGV scheduling system and method considering buffer capacity, the system includes: a network transmission layer, an information receiving layer, a scheduling control layer, an information sending layer, a transportation application layer and other structures; the AGV scheduling method includes: a central control unit first extracts information in the workshop environment; then extracts relevant feature values ​​according to the order information of the 3C smart manufacturing workshop; based on the extracted information, the tasks to be transported are arranged and combined according to the optimization target, and assigned to the AGVs waiting for work; after receiving the assigned tasks, the AGVs start to execute, and at the same time interact with the working environment and upload the information to the central control unit; the central control unit uses the information provided by the AGV to timely observe the AGV transportation status and machine processing status to realize real-time scheduling of workshop tasks; the present invention realizes global scheduling of AGVs and solves problems such as AGV congestion caused by insufficient buffer capacity of the production line.

Description

Technical Field

[0001] The present invention belongs to the technical field of workshop scheduling, and in particular relates to an AGV scheduling system and method for a 3C smart manufacturing workshop taking buffer capacity into consideration. Background Art

[0002] With the development of Industry 4.0, the 3C manufacturing industry is accelerating its transformation toward digital, intelligent, and networked smart workshops and factories. AGV smart logistics is a core component of these smart workshops and factories, playing a crucial role in improving production efficiency and quality. In 3C smart manufacturing workshops and factories, AGV smart logistics ensures fast, accurate, and optimal material transport based on production needs, production line layout, and warehouse planning.

[0003] Common AGV intelligent scheduling methods include integrated scheduling and distributed scheduling. In integrated scheduling, a central control unit is responsible for order acquisition, material dispatch, and coordinated scheduling, while the AGVs simply execute the tasks planned by the central control unit. This approach is easy to manage and offers strong global optimization capabilities, but it can also lead to issues such as low data transmission efficiency when handling conflicts. In distributed scheduling, each AGV is independent and can independently schedule received tasks. However, achieving coordinated scheduling of multiple AGVs and mitigating conflicts presents significant difficulties and challenges.

[0004] Currently, 3C smart production workshops mainly use integrated AGV scheduling based on scheduling rules. The invention with application (patent) number CN202211510638.0 discloses an AGV path planning method, including: setting the starting point and target point of the AGV; adding the influence function from the target point to the current point to the original repulsive potential field function of the artificial potential field method to obtain an improved repulsive potential field function; calculating the resultant force on the AGV based on the repulsive force of the obstacle on the AGV and the attractive force of the target point on the AGV, and the AGV searches for a path following the direction of the resultant force; after the path search, the AGV reaches the next position point and determines whether it has reached the target point; if it has not reached the target point, it determines whether the AGV has fallen into a local minimum; if the AGV has fallen into a local minimum, it introduces a simulated annealing method to reach the next position point, and iterates until the AGV exits the local minimum; if the AGV reaches the target point, the path is retained. However, for the scheduling method generated in this way, since there are only very limited buffer zones between the production lines in most 3C smart manufacturing workshops, once problems such as AGV and workpiece blockages caused by insufficient buffer capacity occur, the rule-based holistic scheduling method lacks real-time performance and cannot handle conflicts in a timely and effective manner, which will affect the transportation efficiency and production continuity of the entire factory. Summary of the Invention

[0005] In response to the deficiencies of the above-mentioned prior art, the purpose of the present invention is to propose an AGV scheduling system and method that takes buffer capacity into consideration. By obtaining status information from the AGVs running in the workshop, uploading it to the central control unit through the workshop local area network and the information transmission layer, and utilizing the Q learning algorithm in reinforcement learning for global scheduling and real-time monitoring, the global scheduling of AGVs can be achieved while considering the limited buffer capacity, thus solving the AGV congestion caused by insufficient buffer capacity of the production line in real time and efficiently. The present invention can effectively overcome the problems of poor real-time performance and easy falling into local optimal solutions existing in the rule-based overall scheduling.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] An AGV scheduling system considering buffer capacity includes a network transmission layer, an information receiving layer, an information receiving layer, a scheduling control layer, an information sending layer, and a transportation application layer;

[0008] The network transmission layer establishes a communication local area network according to the processing environment of the workshop, which is used for information exchange between the processing machines, AGV transport vehicles and the central control unit in the workshop;

[0009] The information receiving layer is used for the central control unit to receive order information, processing machine information and AGV transport vehicle information;

[0010] The scheduling control layer is used by the central control unit to allocate the transportation tasks of multiple AGVs based on the received information, scheduling goals and scheduling rules, using the reinforcement learning algorithm to achieve optimal scheduling under limited buffer capacity;

[0011] The information sending layer is used by the central control unit to distribute the optimal scheduling results calculated by the scheduling application layer to each AGV;

[0012] The transportation application layer is used for the AGV to enter the workshop environment for transportation after receiving the task assigned by the central control unit, and to interact with the workshop environment at the same time, and transmit the transportation information to the central control unit in real time through the information receiving layer.

[0013] The scheduling method of the AGV scheduling system based on the above-mentioned buffer capacity consideration includes the following steps:

[0014] S1. Through the information receiving layer and network transmission layer in the AGV scheduling system, the central control unit extracts workshop environment information in the form of a feature matrix, including machine occupancy, AGV working conditions, and the buffer capacity of each production line buffer;

[0015] S2. In the scheduling control layer, the central control unit extracts relevant feature values ​​based on the order information of the 3C smart manufacturing workshop. The feature values ​​include the time consumed by the workpiece production process, the machines occupied, and the number of processes;

[0016] S3. Based on the information from steps S1 and S2, the central control unit arranges and combines the required transport tasks according to the optimization goal through the algorithm module of the scheduling control layer, transmits the results to the action module, and distributes them to the waiting AGVs through the information sending layer and network transmission layer;

[0017] S4. The AGV receives the task assigned by the central control unit through step S3 and enters the task execution phase. At the same time, it interacts with the working environment in the transportation application layer and uploads information to the central control unit;

[0018] S5. The central control unit uses the information provided by the AGV in step S4 to timely observe the AGV transportation status and machine processing status through the network transmission layer and the information receiving layer, thereby realizing real-time scheduling of workshop tasks.

[0019] The step S1 is specifically as follows:

[0020] Through the information receiving layer and network transmission layer in the AGV scheduling system, the central control unit will obtain workshop environment information before receiving the first order, forming a feature matrix of AGV work status table, transportation time between AGV and each target machine, machine work status table, and buffer capacity table of buffer zones between production lines. From the start of the first order to the completion of all work, this information will be provided by each AGV in the working environment after interacting with the environment.

[0021] The step S2 is specifically as follows:

[0022] In the scheduling control layer, the central control unit integrates and extracts the order information, arranges the processed workpieces in order of processing sequence, and forms a ready schedule, waiting schedule, and processing process schedule for the workpiece processing procedures and machines.

[0023] The ready schedule refers to the time when the workpiece starts to be processed. Due to the limited capacity of the buffer, the machine may be blocked, and the ready time of the workpiece is not equal to the time when the workpiece arrives at the machine. The waiting schedule refers to the time when each workpiece waits for processing after reaching the ready time. This time is not related to machine congestion, but is related to the workpiece processed by the machine at the last moment and the previous process of the target workpiece. For example, the waiting time corresponding to the i-th process of the j-th workpiece is shown in formula (1):

[0024]

[0025] Where: is the processing time corresponding to the i-th process of the j-1-th workpiece;

[0026] is the ready time corresponding to the i-th process of the j-1-th workpiece;

[0027] is the processing time corresponding to the i-1th process of the jth workpiece;

[0028] is the ready time corresponding to the i-1th process of the jth job.

[0029] The step S3 is specifically as follows:

[0030] The optimization goal in step S3 is to arrange and combine the tasks to be transported according to the optimization goal through the algorithm module of the scheduling control layer, and transmit the results to the action module. The optimization goals include: minimizing the maximum transportation time, minimizing the delivery delay period, minimizing the AGV energy consumption, and minimizing the machine idle time;

[0031] The allocation process of the AGVs waiting for work is as follows:

[0032] S301. The central control unit creates a state space in the system. Based on the information obtained in steps S1 and S2, the central control unit arranges the tasks to be transported in a two-dimensional matrix of machines and processing steps. Similarly, the buffer capacity table, the ready schedule of each workpiece, the waiting schedule and the status table of each AGV are used as the state space matrix. The system status will change in real time as the AGV performs different actions and is continuously updated.

[0033] S302. The central control unit selects actions in the action space by observing the system status, selects the AGV to work according to the availability of the AGV, and dispatches processing tasks to each standby AGV according to the scheduling strategy. The default scheduling strategy is the first-come-first-served strategy, that is, the workpiece that is processed first is given priority to be transported by the AGV; if the buffer zone is about to be blocked, the central control unit changes to the buffer zone priority strategy in real time according to the workshop working environment, that is, the workpiece of the machine that is about to cause buffer zone congestion is given priority to be transported; if the buffer zone will not be blocked under the current strategy, the central control unit can strictly change the relaxed first-come-first-served strategy to the minimum transportation time strategy, that is, the strategy that makes the overall transportation time the shortest without causing buffer zone congestion; if an emergency order or machine failure occurs, the central control unit will change to a conflict priority strategy, that is, when an emergency order occurs, the transportation task priority of the order is adjusted to the highest, and the task is transported first. When a machine failure occurs, the transportation task of the workpiece in the machine buffer is set to the first level of priority, and the workpiece transported to the machine is set to the second level of priority.

[0034] Step S302 involves selecting actions in the action space. The central control unit uses the ε-greedy algorithm. Initially, the central control unit randomly selects actions. As the number of rounds of selection increases, the central control unit begins to select actions with higher reward values, accelerating the search for the optimal solution.

[0035] S303. After the central control unit assigns tasks to each working AGV according to different scheduling rules through the information sending layer and the network transmission layer, it calculates the reward function. The reward function corresponding to the current transportation task of each AGV is the discounted return of the future reward value. The reward function consists of two parts. The first part is the calculation of the transportation time reward. The transportation time of each AGV is calculated according to the assigned tasks, and the longest transportation time is selected. The second part is the maximum proportion of the relevant buffer zone. The calculation formula is shown in formula (2). The weighted sum of the two parts is the total reward function of the current scheduling result.

[0036]

[0037] Where: α is the buffer congestion ratio coefficient;

[0038] β0 is the buffer capacity of the buffer zone;

[0039] β ' max The maximum number of buffers in the current dispatch result buffer.

[0040] The step S4 is specifically as follows:

[0041] After receiving the task assigned by the central control unit, the AGV performs the transportation work and interacts with the environment in the transportation application layer. Since machine failure and buffer blockage are all transmitted between the machine and the central control unit, the AGV will return to the central control unit whether there is any workpiece falling or AGV failure in the working environment.

[0042] The step S5 is specifically as follows:

[0043] After receiving the information returned by the AGV, the central control unit handles any emergencies through the network transmission layer and the information receiving layer. If a workpiece falls, the central control unit will first select the nearest AGV-a with the same route and destination and assign the original transport task to another AGV. AGV-a only needs to deliver the dropped workpiece to the target machine ①, while also determining whether the transport time will make machine ① idle.

[0044] If a machine is idle, the target machine of AGV-b, which is transporting the same workpiece and whose time is later than that of the AGV transporting the dropped workpiece, is changed to machine ①, and the target machine of AGV-a is changed to machine ② of the same process;

[0045] If an AGV fails, other AGVs will be used to complete the pending transportation tasks. At the same time, the central control unit sends information to the AGV garage, calling a new AGV to join the work sequence, and rearranges the subsequent work of the faulty AGV and the subsequent work of several other AGVs that have completed the current work of the faulty AGV.

[0046] Compared with the prior art, the present invention has the following beneficial effects and advantages:

[0047] 1. In step S1 of the present invention, the central control unit extracts and represents key information from the workshop environment in the form of a feature matrix, eliminating the data processing burden caused by each machine transmitting independent information and reducing the difficulty of data processing. It can also clearly display machine occupancy, AGV operating conditions, and buffer congestion, facilitating action selection and scheduling rule selection when scheduling tasks.

[0048] 2. In step S2 of the present invention, the central control unit extracts order data in the form of feature vectors, which can obtain effective information efficiently and quickly, avoid receiving and screening large amounts of data, save data capacity, reduce processing time and data processing errors, and provide clear data samples for the subsequent allocation of scheduling tasks.

[0049] 3. In step S3 of the present invention, setting different task priorities and optimization targets for AGV scheduling can further improve the flexibility of workshop production and transportation, and is conducive to the scheduling system to better achieve different scheduling optimizations according to the different needs of the workshop.

[0050] Taking parameters such as the processing time of different processes of each workpiece, the availability of the machine, and the capacity of the buffer zone as the state space can ensure that reinforcement learning selects the state based on the advantages and disadvantages of the parameters related to the optimization goal, and more comprehensively and completely describe the actual state of workpiece processing, so that the final scheduling result can be closer to the optimization goal and more in line with reality.

[0051] 4. In step S3 of the present invention, the weighted sum of the congestion coefficient of the buffer zone and the maximum transportation time is used as one of the optimization targets, thereby ensuring that the buffer zone will not be blocked while improving the processing efficiency of the workpiece.

[0052] In summary, this invention considers the buffer's priority capacity and incorporates relevant parameters of buffer capacity as part of the optimization objective, reducing the probability of congestion in transportation and workpiece processing in the 3C intelligent manufacturing workshop and ensuring production continuity. By establishing a state space through reinforcement learning, selecting appropriate actions, and updating the state using feedback, the computational speed and accuracy of AGV scheduling are improved. Furthermore, by utilizing different optimization objectives and AGV scheduling strategies, AGV scheduling can be performed more flexibly. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is the architecture diagram of the AGV scheduling system of the present invention.

[0054] Figure 2 This is a schematic diagram of the workshop information feature matrix of the present invention.

[0055] Figure 3 This is a flow chart of information conversion in the information receiving layer of the present invention.

[0056] Figure 4 Flowchart of the scheduling process of the present invention.

[0057] Figure 5 State space diagram created for the Q-learning algorithm of the present invention.

[0058] Figure 6 The action flow chart for selecting the Q learning algorithm of the present invention.

[0059] Figure 7 This is a flow chart of the target neural network training of the Q learning algorithm of the present invention.

[0060] Figure 8 This is a flow chart of the Q learning algorithm strategy selection process of the present invention.

[0061] Figure 9 This is a flow chart of processing emergency orders according to the present invention.

[0062] Figure 10 This is a flow chart of the present invention for handling machine failures.

[0063] Figure 11 This is a schematic diagram of the workpiece falling scene of the present invention.

[0064] Figure 12 This is a flow chart of the process of handling workpiece drops according to the present invention.

[0065] Figure 13 This is a flow chart of the present invention for handling AGV faults.

[0066] Figure 14 This is a simulation diagram of the AGV scheduling system based on the consideration of buffer capacity;

[0067] Figure 15This is a training result diagram of the AGV scheduling system based on the consideration of the buffer capacity of the present invention. DETAILED DESCRIPTION

[0068] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings.

[0069] like Figure 1 The present invention constructs an AGV scheduling system that takes into account the buffer capacity. Specifically, the scheduling system includes a network transmission layer, an information receiving layer, a scheduling control layer, an information sending layer, and a transportation application layer.

[0070] Among them, the network transmission layer establishes a communication local area network based on the processing environment of the workshop, which is used for information exchange between the processing machines, AGV transport vehicles and the central control unit in the workshop;

[0071] Information receiving layer: the central control unit receives order information, processing machine information and AGV transport vehicle information through the information receiving layer;

[0072] The central control system uses reinforcement learning algorithms to allocate transportation tasks to multiple AGVs based on the received information, scheduling goals, and scheduling rules, to achieve optimal scheduling under limited buffer capacity.

[0073] At the information sending layer, the central control unit distributes the optimal scheduling results calculated by the scheduling application layer to each AGV;

[0074] At the transportation application layer, after receiving the task assigned by the central control unit, the AGV enters the workshop environment for transportation and interacts with the workshop environment, transmitting the transportation information to the central control unit in real time through the information receiving layer;

[0075] The present invention further proposes a 3C smart manufacturing workshop AGV scheduling method considering the buffer capacity. The method is applied to the above-mentioned 3C smart manufacturing workshop AGV scheduling system considering the buffer capacity, comprising the following steps:

[0076] S1. Through the information receiving layer and network transmission layer in the AGV scheduling system, the central control unit extracts workshop environment information in the form of a feature matrix, such as the occupancy of the machine, the working status of the AGV, and the buffer capacity of each production line buffer. Before receiving the first order, the central control unit will obtain the workshop environment information and form a feature matrix of the AGV working status table, the transportation time between the AGV and each target machine, the machine working status table, and the buffer capacity table between production lines. The feature matrix is ​​as follows: Figure 2 As shown in the figure, from the start of the first order to the completion of all work, this information will be provided by each AGV in the working environment after interacting with the environment.

[0077] Workshop equipment is mainly divided into processing machines, AGV transport vehicles, and processing machine input and output buffers.

[0078] Among them, processing machines are used to process or assemble workpieces and can complete one or more processes;

[0079] AGV transport vehicles are used to transfer materials between processing machines;

[0080] The input and output buffer of the processing machine is used for temporary storage and buffering of workpieces that have been processed by the machine or workpieces to be processed.

[0081] Construct the workshop layout and operation logic. The workshop layout is used to explain the relative position relationship between the processing machines, AGV transport vehicles, and the input and output buffers of the processing machines within the workshop. The workshop operation logic is used to explain the relationship between the processing machines, workpiece processing procedures, and workpiece transportation trajectories.

[0082] The information receiving, sending, and transmission architecture is used to explain how the workshop central control unit and AGV transport vehicles receive, send, and transmit information. It includes the following steps:

[0083] S101. Establishment of a workshop local area network, which consists of the central control unit, all processing machines, and all AGV transport vehicles in the workshop, and is used to transmit scheduling-related information;

[0084] Establish a workshop local area network, connecting all processing machines and AGV transport vehicles in the workshop to form a local area network. Devices in the local area network are connected to machines or AGV vehicles through wireless access points, and the other end is connected to the central control unit.

[0085] S102, an information receiving network, which is used to receive real-time workshop information transmitted by AGV transport vehicles to the central control unit, including the status of workshop processing machines and AGV transport vehicles, the capacity of each input and output buffer, etc., and initialize the information;

[0086] Build an information receiving network to organize the information transmitted by AGV transport vehicles through the workshop local area network, such as whether all processing machines in the workshop are in an available state. If one or more machines cannot work normally in the initial state, the information receiving network will change the value of the faulty machine in the feature matrix that manages the machine status to unavailable; after organizing the information, transmit the feature matrix of the processing machine, AGV transport vehicle and buffer zone to the central control unit.

[0087] S2. In the scheduling control layer, the central control unit extracts relevant feature values ​​based on the order information of the 3C production workshop. The feature values ​​include the time consumed by the workpiece production process, the machines occupied, and the number of processes;

[0088] Because 3C smart manufacturing workshops operate in a mixed-flow, parallel machine replacement process, each workpiece undergoes the same processing sequence. Thanks to the presence of parallel machines, workpieces with the same process can be processed on different machines. A central control unit integrates and extracts order information, arranges the workpieces in order of processing sequence, and generates a timeline for the workpiece processing steps and machines, including readiness, waiting, and processing schedules.

[0089] The ready schedule refers to the time when the workpiece starts to be processed. Due to the limited capacity of the buffer, the machine may be blocked, and the workpiece ready time is not equal to the time when the workpiece arrives at the machine. The waiting schedule refers to the time when each workpiece waits for processing after reaching the ready time. This time has nothing to do with machine congestion, but is related to the workpiece processed by the machine at the last moment and the previous process of the target workpiece. For example, the waiting time corresponding to the i-th process of the j-th workpiece is shown in formula (1). The waiting time corresponding to each process of each workpiece will affect the processing efficiency.

[0090]

[0091] Where: is the processing time corresponding to the i-th process of the j-1-th workpiece;

[0092] is the ready time corresponding to the i-th process of the j-1-th workpiece;

[0093] is the processing time corresponding to the i-1th process of the jth workpiece;

[0094] is the ready time corresponding to the i-1th process of the jth job.

[0095] like Figure 3 As shown, the workshop warehouse obtains information such as the total number and batch size of workpieces to be processed in real time, and uploads it to the information receiving layer through the workshop local area network. The information receiving layer extracts information of different workpieces to be processed one by one in sequence, and forms a parameter feature vector of the workpiece to be processed by combining the extracted information with the processing sequence and processing time corresponding to the workpiece to be processed. If the extracted workpiece information to be processed is the last type, the information receiving layer transmits the workpiece feature matrix containing multiple feature vectors, such as the ready schedule, waiting schedule, and processing process schedule of the workpiece processing procedures and time, to the central control unit.

[0096] The production model of a 3C smart manufacturing workshop requires a relatively fixed product line with minimal variation in workpiece types. Therefore, the processing sequence, processing time, and machine parameters for each workpiece are pre-stored in the information management layer. This allows for parameter characteristics to be transmitted based on the batch size of workpieces provided by the warehouse. This reduces the storage burden on the central control unit while ensuring data accuracy and real-time availability.

[0097] S3. Based on the information from steps S1 and S2, the central control unit arranges and combines the required transport tasks according to the optimization goal through the algorithm module of the scheduling control layer, transmits the results to the action module, and distributes them to the waiting AGVs through the information sending layer and network transmission layer;

[0098] Optimization objectives include minimizing maximum transportation time, minimizing delivery delays, minimizing AGV energy consumption, and minimizing machine idle time. The optimization objectives need to be changed according to the real-time environment of the workshop. The optimization objective of generally processed workpieces is to minimize the maximum transportation time. If there are special requirements for the workpiece order, the central control unit will change the optimization objective of the workshop in real time.

[0099] Reference Figure 4 The central control unit receives workshop environment information and order information based on S1 and S2, and uses the Q learning algorithm in the reinforcement learning algorithm to establish a state space. Then the Q learning algorithm gives the AGV an action according to the current state. After the AGV performs the action, it interacts with the environment to obtain rewards, and thus selects the action with the largest reward to execute. After executing the action, it obtains the next state. After a task cycle is cycled, all the actions performed by the AGV are output as the scheduling result.

[0100] The central control unit allocates transportation tasks as follows:

[0101] S301, reference Figure 5 The central control unit creates a state space in the system. The AGV that performs the transportation task transmits the information in the workshop environment to the information receiving layer through the workshop LAN. After sorting, the information receiving layer transmits the feature matrix of the workpiece to be processed to the central control unit. The central control unit arranges the tasks to be transported according to a two-dimensional matrix of time and processing procedures. Similarly, the buffer capacity table, the ready schedule, waiting schedule and processing process schedule of each workpiece are used as the state space matrix. The system status will change in real time as the AGV performs different actions and needs to be continuously updated.

[0102] S302, the central control unit uses the target neural network in the Q learning algorithm to select an action, such as Figure 6As shown in Figure 1, the Q-learning algorithm uses the ε-greedy algorithm to select actions. Initially, ε is large, and the Q-learning algorithm randomly selects actions, preventing the algorithm from quickly falling into local optima and premature maturity. As the number of cyclic selections increases, ε gradually decreases, and the Q-learning algorithm begins to select the action with the highest reward, accelerating the search for the optimal solution. The specific steps for selecting actions are as follows:

[0103] S3021, the Q learning algorithm initializes the ε-greedy threshold ε, and then identifies the current state;

[0104] S3022. The Q-learning algorithm generates a random number in the interval (0, 1). If the number is greater than the threshold ε, the action with the highest reward is selected for execution. If the number is less than the threshold ε, an action in the action set is randomly selected for execution.

[0105] S3023, reference Figure 7 , the Q learning algorithm stores the action corresponding to the state and the next state in the experience library for learning and training the target neural network. For the experience library, the learning step size and number of samples are first set. The Q learning algorithm saves multiple states and execution actions as experience through training, and then judges the time step size and the number of samples in the experience library. If both conditions are met, a period of experience is extracted, and the target neural network is used to calculate the maximum predicted Q value of the next state. At the same time, the Q value of the current state is calculated, and the expected Q value is calculated using the evaluation neural network. Then, the difference between the expected Q value and the current Q value is calculated and optimized. The parameters of the target neural network are updated using the expected Q value and the current Q value.

[0106] S3024: After the AGV executes the action, it interacts with the workshop environment to obtain the next state and determines whether the next state has reached the last task. If so, all actions to be executed are dispatched to the AGV.

[0107] like Figure 8 As shown in the figure, the AGV to be worked is selected according to the availability of AGVs, and tasks are issued to each standby AGV according to the scheduling strategy. If there are no special requirements, the default scheduling strategy is the first-come-first-served strategy, that is, the workpieces that are processed first are given priority to be transported by AGVs; if the buffer zone is about to be blocked, the central control unit will change to the buffer zone priority strategy in real time according to the workshop working environment, that is, the workpieces of the machines that are about to cause buffer zone congestion will be given priority to be transported; if the buffer zone will not be blocked under the current strategy, the central control unit can strictly change the relaxed first-come-first-served strategy to the minimum transportation time strategy, that is, the strategy that makes the overall transportation time the shortest without causing buffer zone congestion; if special circumstances are encountered, the central control unit will select the corresponding conflict priority strategy.

[0108] For emergency orders, refer to Figure 9 .

[0109] When a warehouse receives an urgent order, it uploads it to the information receiving layer via the workshop LAN. The information receiving layer prioritizes uploading the order's characteristic information to the central control unit, which then identifies the status of each process in the order information. The Q-learning algorithm prioritizes each process as an action, first selecting the AGV that completed its current transport task the earliest and then determining whether it has reached the last item in the transport task. If so, the corresponding task is assigned to the AGV that received the new task; if not, the AGV that completed its current transport task earlier is selected to continue the task.

[0110] For machine failures, refer to Figure 10 .

[0111] After a machine breaks down, it stops working. The AGV receives the fault signal and uploads the fault information to the information receiving layer through the workshop LAN. The information receiving layer uploads the fault information to the central control unit. The central control unit sets the status of the faulty machine to unavailable. At the same time, the central control unit identifies the status of the faulty machine's input buffer, output buffer, and the workpiece to be processed that is about to be transported to the faulty machine. The Q learning algorithm uses the subsequent process as an action. The central control unit selects the AGV and dispatches the task to the corresponding AGV after one task cycle.

[0112] S303. After dispatching tasks to each AGV according to different scheduling rules through the information transmission layer and the network transmission layer, the central control unit calculates the reward function. The reward function consists of two parts. The first part is the calculation of the transportation time reward. The transportation time of each AGV is calculated based on the dispatched tasks, and the longest transportation time is selected. The second part is the maximum proportion of the buffer zone. The calculation formula is shown in Equation (2). The weighted sum of the two parts is the total reward function of the current scheduling result.

[0113]

[0114] Where: α is the buffer congestion ratio coefficient;

[0115] β0 is the buffer capacity of the buffer zone;

[0116] β ' max The maximum number of buffers in the current scheduling result buffer

[0117] S4. The AGV receives the task assigned by the central control unit through step S3 and enters the task execution phase. At the same time, it interacts with the working environment in the transportation application layer and uploads information to the central control unit;

[0118] After receiving the task assigned by the central control unit, the AGV performs the transportation work and interacts with the environment in the transportation application layer. Due to problems such as machine failure and buffer blockage, the machine and the central control unit transmit information. Therefore, the AGV will return to the central control unit whether there are problems such as workpiece falling and AGV failure in the working environment.

[0119] S5. The central control unit processes the AGV transportation tasks and machine processing tasks in a timely manner through the network transmission layer and the information receiving layer based on the information provided by the AGV in step S4, thereby realizing real-time scheduling of workshop tasks.

[0120] After receiving the information returned by the AGV, the central control unit will handle the emergency situation.

[0121] For workpiece drops, refer to Figure 11 and Figure 12 .

[0122] The central control unit first selects the nearest AGV-a with the same route and destination, and assigns its original transport task to another AGV. AGV-a only needs to deliver the dropped workpiece to the target machine ①. At the same time, it determines whether the transport time will make machine ① idle. If so, the target machine of AGV-b, which is transporting the same workpiece at a later time, will be changed to machine ①, and the target machine of AGV-a will be changed to machine ② for the same process.

[0123] For AGV failures, refer to Figure 13 , and similarly use other AGVs to complete the pending transport tasks. At the same time, the central control unit sends information to the AGV garage, calls a new AGV to join the work sequence, and rearranges the subsequent work of the faulty AGV and the subsequent work of several other AGVs that have completed the current work of the faulty AGV.

[0124] Figure 13 This is a simulation of the 3C smart manufacturing workshop AGV scheduling system proposed by the present invention that takes into account the buffer capacity. In this model, the workshop is an 8*8 mixed flow workshop with a total of 8 processes. Each process has a processing machine tool. 64 tasks are completed at random times, and the number of workpieces in each task is also randomly distributed. The above AGV scheduling system, which has been trained with a large amount of data, is used to perform reasonable scheduling of AGVs. The training results are as follows: Figure 15 The final scheduling result is as follows Figure 14 , achieving the goal of shortest transport time for this workshop without buffer zone congestion.

[0125] In summary, the present invention sets different scheduling strategies when performing AGV task dispatching, which can realize scheduling of different complexities and different optimization targets for different working environments and working conditions, making the AGV scheduling of 3C smart manufacturing workshops more flexible and flexible. The first-come-first-served operation has the lowest complexity, and the conditions are relatively loose, with fewer factors to consider. It is suitable for 3C smart production workshops with simple structures and lower requirements. The buffer priority strategy and the minimum transportation time strategy are more complicated. The former needs to consider the AGV and workpiece processing congestion problem caused by insufficient buffer capacity, while the latter takes more time costs into account and is suitable for 3C smart production workshops with complex structures. The conflict priority strategy is an emergency strategy set up for emergencies in the workshop, which is more in line with reality. In actual 3C smart manufacturing workshops, more complex AGV scheduling strategies can be designed to meet different optimization targets and workshop production modes.

[0126] Furthermore, the central control unit selects an action based on the current state, calculates the real-time reward for that action, and then determines the next state. This real-time reward is related to the future action and state. Therefore, reinforcement learning plans future scheduling tasks, accurately estimating the current action and achieving more accurate scheduling results. After repeated training, it can quickly and accurately find the optimal solution for new scheduling tasks, demonstrating good generalization.

[0127] Furthermore, by setting up conflict resolution strategies, we can avoid situations such as urgent orders, machine failures, or AGV malfunctions that cause workpiece production halts. By elevating the priority of urgent orders or machine failures, we can reschedule subsequent transport tasks. Through extensive data training, this scheduling algorithm has achieved excellent real-time and high-accuracy results in solving emergencies. By rescheduling some AGV transport tasks to address situations such as AGV failures or dropped workpieces, we can effectively minimize the impact of emergencies on workpiece production.

[0128] Furthermore, when dealing with emergencies such as AGV failures, the central control unit needs to make changes to the scheduling tasks based on the actual situation, and thus requires status information of the workshop production environment. The traditional collection method is to obtain the status data of the global workshop only through the central control unit, make a judgment, and then schedule, which will cause a large delay in rescheduling. The present invention interacts with the working environment through AGV, and uploads information such as the workshop environment and emergencies to the central control unit. This can improve the accuracy of information transmission and reduce the capacity of a single piece of information, making it easier for the central control unit to receive it. At the same time, it can avoid the central control unit from collecting scattered workshop status data, which causes it to be overburdened.

[0129] Therefore, the present invention organically combines the central control unit, transport AGV and processing machines in the 3C smart factory, utilizes the interaction between AGV and the factory environment, and transmits workshop information in real time, so that the workpiece processing process can be completed efficiently while avoiding conflicts. This AGV scheduling method that combines the overall and distributed methods makes it easy for the central control unit to integrate the complex and large amount of data resources in the factory environment, and then quickly and accurately find the optimal scheduling solution. At the same time, it will not easily fall into the local optimal solution like the intelligent optimization algorithm or heuristic algorithm, resulting in the scheduling result not being optimal and the resources cannot be used most effectively. The 3C smart manufacturing workshop is relatively complex as a whole, with many equipment. The AGV scheduling problem cannot be accurately and comprehensively expressed using mathematical polynomials. Mathematical solution methods such as linear programming and heuristic algorithms such as intelligent algorithms cannot accurately solve the AGV scheduling results. The AGV scheduling method proposed in the present invention can solve this problem very well, and utilizes the ability of neural networks to process data and the strong generalization ability of reinforcement learning to improve the real-time and accuracy of AGV scheduling.

[0130] The above content is only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A 3C smart manufacturing workshop AGV scheduling method considering buffer capacity, characterized in that: The AGV scheduling system it is based on includes a network transmission layer, an information receiving layer, a scheduling control layer, an information sending layer, and a transportation application layer. The network transmission layer establishes a communication local area network based on the processing environment of the workshop, which is used for information exchange between the processing machines in the workshop, the AGV transport vehicles and the central control unit. The information receiving layer is used for the central control unit to receive order information, processing machine information and AGV transport vehicle information; The scheduling control layer is used by the central control unit to allocate the transportation tasks of multiple AGVs based on the received information, scheduling goals and scheduling rules, using the reinforcement learning algorithm to achieve optimal scheduling under limited buffer capacity; The information sending layer is used by the central control unit to distribute the optimal scheduling results calculated by the scheduling application layer to each AGV; The transport application layer is used for the AGV to enter the workshop environment for transportation after receiving the task assigned by the central control unit, and to interact with the workshop environment at the same time, and transmit the transportation information to the central control unit in real time through the information receiving layer; The scheduling method includes the following steps: S1. Through the information receiving layer and network transmission layer in the AGV scheduling system, the central control unit extracts workshop environment information in the form of a feature matrix, including machine occupancy, AGV working conditions, and the buffer capacity of each production line buffer; S2. In the scheduling control layer, the central control unit extracts relevant feature values ​​based on the order information of the 3C smart manufacturing workshop. The feature values ​​include the time consumed by the workpiece production process, the machines occupied, and the number of processes; S3. Based on the information from steps S1 and S2, the central control unit arranges and combines the required transport tasks according to the optimization goal through the algorithm module of the scheduling control layer, transmits the results to the action module, and distributes them to the waiting AGVs through the information sending layer and network transmission layer; S4. The AGV receives the task assigned by the central control unit through step S3 and enters the task execution phase. At the same time, it interacts with the working environment in the transportation application layer and uploads information to the central control unit; S5. The central control unit uses the information provided by the AGV in step S4 to timely observe the AGV transportation status and machine processing status through the network transmission layer and the information receiving layer, thereby realizing real-time scheduling of workshop tasks.

2. The AGV scheduling method for a 3C smart manufacturing workshop considering buffer capacity according to claim 1 is characterized in that: The step S1 is specifically as follows: Through the information receiving layer and network transmission layer in the AGV scheduling system, the central control unit will obtain workshop environment information before receiving the first order, forming a feature matrix of AGV work status table, transportation time between AGV and each target machine, machine work status table, and buffer capacity table of buffer zones between production lines. From the start of the first order to the completion of all work, this information will be provided by each AGV in the working environment after interacting with the environment.

3. The AGV scheduling method for a 3C smart manufacturing workshop considering buffer capacity according to claim 1 is characterized in that: The step S2 is specifically as follows: In the scheduling control layer, the central control unit integrates and extracts order information, arranges the workpieces to be processed in order of processing sequence, and forms a ready schedule, waiting schedule, and processing schedule for the workpiece processing procedures and machines; The ready schedule refers to the time when the workpiece starts to be processed. Due to the limited capacity of the buffer, the machine may be blocked, and the ready time of the workpiece is not equal to the time when the workpiece arrives at the machine. The waiting schedule refers to the time when each workpiece waits for processing after reaching the ready time. This time is not related to machine congestion, but is related to the workpiece processed by the machine at the last moment and the previous process of the target workpiece. For example, the waiting time corresponding to the i-th process of the j-th workpiece is shown in formula (1): Where: is the processing time corresponding to the i-th process of the j-1-th workpiece; is the ready time corresponding to the i-th process of the j-1-th workpiece; is the processing time corresponding to the i-1th process of the jth workpiece; is the ready time corresponding to the i-1th process of the jth job.

4. The AGV scheduling method for a 3C smart manufacturing workshop considering buffer capacity according to claim 1 is characterized in that: The step S3 is specifically as follows: The optimization goal in step S3 is to arrange and combine the tasks to be transported according to the optimization goal through the algorithm module of the scheduling control layer, and transmit the results to the action module. The optimization goals include: minimizing the maximum transportation time, minimizing the delivery delay period, minimizing the AGV energy consumption, and minimizing the machine idle time; The allocation process of the AGVs waiting for work is as follows: S301. The central control unit creates a state space in the system. Based on the information obtained in steps S1 and S2, the central control unit arranges the transport tasks in a two-dimensional matrix of machines and processing steps. Similarly, the buffer capacity table, the ready schedule and waiting schedule of each workpiece, and the state table of each AGV are used as the state space matrix. The system state changes in real time as the AGV performs different actions and is continuously updated. S302, the central control unit observes the system status, selects actions in the action space, selects the AGV to work according to the availability of AGVs, and dispatches processing tasks to each standby AGV according to the scheduling strategy. The default scheduling strategy is the first-come-first-served strategy, that is, the workpiece that is processed first is given priority to be transported by the AGV; if the buffer is about to be blocked, the central control unit changes to the buffer priority strategy in real time according to the workshop working environment, that is, the workpiece of the machine that is about to cause buffer congestion is given priority to be transported; if the buffer is not blocked under the current strategy, the central control unit can strictly change the relaxed first-come-first-served strategy to the minimum transportation time strategy, that is, the strategy that minimizes the overall transportation time without causing buffer congestion; if an emergency order or machine failure occurs, the central control unit will change to the conflict priority strategy, that is, when an emergency order occurs, the transportation task priority of the order is adjusted to the highest, and the task is transported first; when a machine failure occurs, the transportation task of the workpiece in the machine buffer is set to the first level of priority, and the workpiece transported to the machine is set to the second level of priority; In step S302, the central control unit selects an action in the action space using the ε-greedy algorithm. Initially, the central control unit randomly selects an action. As the number of cyclic selections increases, the central control unit begins to select actions with higher reward values, thereby accelerating the search for the optimal solution. S303. After the central control unit assigns tasks to each working AGV according to different scheduling rules through the information transmission layer and the network transmission layer, it calculates the reward function. The reward function corresponding to the current transportation task of each AGV is the discounted return of the future reward value. The reward function consists of two parts. The first part is the calculation of the transportation time reward. The transportation time of each AGV is calculated according to the assigned tasks, and the longest transportation time is selected. The second part is the maximum proportion of the buffer zone. The calculation formula is shown in formula (2). The weighted sum of the two parts is the total reward function of the current scheduling result. Where: α is the buffer congestion ratio coefficient; β0 is the buffer capacity of the buffer zone; β ' max The maximum number of buffers in the current dispatch result buffer.

5. The AGV scheduling method for a 3C smart manufacturing workshop considering buffer capacity according to claim 1 is characterized in that: The step S4 is specifically as follows: After receiving the task assigned by the central control unit, the AGV performs the transportation work and interacts with the environment in the transportation application layer. Since machine failure and buffer blockage are all transmitted between the machine and the central control unit, the AGV will return to the central control unit whether there is any workpiece falling or AGV failure in the working environment.

6. The AGV scheduling method for a 3C smart manufacturing workshop considering buffer capacity according to claim 1 is characterized in that: The step S5 is specifically as follows: After receiving the information returned by the AGV, the central control unit handles any emergencies through the network transmission layer and the information receiving layer. If a workpiece falls, the central control unit will first select the nearest AGV-a with the same route and destination and assign the original transport task to another AGV. AGV-a only needs to deliver the dropped workpiece to the target machine ①, while also determining whether the transport time will make machine ① idle. If a machine is idle, the target machine of AGV-b, which is transporting the same workpiece and whose time is later than that of the AGV transporting the dropped workpiece, is changed to machine ①, and the target machine of AGV-a is changed to machine ② of the same process; If an AGV fails, other AGVs will be used to complete the pending transportation tasks. At the same time, the central control unit sends information to the AGV garage, calling a new AGV to join the work sequence, and rearranges the subsequent work of the faulty AGV and the subsequent work of several other AGVs that have completed the current work of the faulty AGV.

Images