Multi-variety workpiece mixed flow conveyor belt feeding processing system optimization control method

Abstract

The invention belongs to the field of system optimization and production scheduling, and particularly relates to an optimization control method of a conveyor belt feeding processing system capable of performing mixed-flow production of multiple types of workpieces in consideration of product demand conditions. The multi-variety workpiece mixed flow conveying belt feeding processing system comprises a conveying belt, a photoelectric sensor, # imgabs 0 # different types of workpieces, a processing main body, # imgabs 1 # cache libraries with different capacities and corresponding finished product libraries with different capacities. Firstly, the state, action and corresponding strategy of the system are defined, and the decision moment and decision process of the system are determined according to the working mechanism of the system; and then, according to the random characteristics and the decision process of the system, establishing a semi-Markov decision process model for optimizing the control problem, and solving an optimal strategy by using a strategy iterative algorithm. According to the invention, the productivity of the multi-dimensional flexible CSPS system can be effectively improved, and the requirements of customers can be better met.

Description

Technical Field

[0001] The present invention belongs to the field of system optimization and production scheduling, and specifically is an optimization control method for a conveyor belt material processing system capable of mixed flow production of multiple varieties of workpieces taking product demand conditions into consideration. Background Art

[0002] With the rapid development of information, communication and control technologies, human society has gradually entered the era of intelligent manufacturing. The most representative of intelligent manufacturing is a modern discrete manufacturing production line composed of conveyor-serviced production stations (CSPS), such as some robot assembly lines or food and drug packaging lines. The intelligence level of the CSPS system reflects the information level of the entire production line and even the enterprise. Therefore, how to improve the flexibility and intelligence level of its work through technical means such as information, communication and control has become one of the important ways for enterprises to improve productivity and economic benefits. The scientific and technological issues involved are important research directions in the field of system optimization.

[0003] So far, domestic and foreign scholars have conducted a lot of in-depth research on traditional CSPS systems and achieved fruitful theoretical and application results. However, the existing research mainly focuses on single-variety workpieces and does not consider the impact of product demand. With the improvement of intelligent manufacturing level and the development of lean production, mixed-flow production of multi-variety workpieces will become the norm of future production mode, and the production control of processing stations can also be flexibly adjusted according to product demand. These are all lacking in existing research. Therefore, studying the optimization control method of such multi-dimensional flexible CSPS system has important theoretical research value and practical application prospects. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes an optimization control method for a conveyor belt material processing system with a mixed flow of multiple types of workpieces, thereby improving the productivity of the system and better meeting customer needs.

[0005] In order to achieve the above object, the present invention adopts the following technical scheme:

[0006] An optimization control method for a conveyor belt material processing system with a mixed flow of multiple types of workpieces, the conveyor belt material processing system with a mixed flow of multiple types of workpieces comprising: a conveyor belt, a photoelectric sensor, Different types of workpieces, processing subjects, Cache libraries of different capacities and corresponding finished product libraries of different capacities,

[0007] The processing body is located on one side of the conveyor belt, a buffer storehouse and a finished product storehouse are respectively arranged on both sides of the processing body, and the photoelectric sensor is located upstream of the fixed picking point of the conveyor belt; Different types of workpieces arrive at the processing station randomly along the conveyor belt in an independent Poisson process. The arrival rate of the workpiece type is ,in The processing subject can unload the workpiece that has arrived at the picking point and put it into the corresponding cache, or take out a workpiece from the cache for processing. The finished product will be stored in the corresponding finished product warehouse. The processing time of each type of workpiece is subject to the parameter of The photoelectric sensor can detect and record the position and time information of the workpiece within a distance from the pick-up point to a certain position upstream on the conveyor belt. The buffers of different capacities are used to store the unprocessed workpieces unloaded from the conveyor belt. The capacity of the cache is ; Finished product warehouses of different capacities are used to store processed finished products. The capacity of the finished product warehouse is At any time, customers corresponding to various finished products also arrive randomly in independent Poisson processes. The arrival rate of customers of this category is , and each customer takes a processed finished product from the finished product warehouse. If the finished product warehouse is empty, the customer will automatically be lost;

[0008] The conveyor belt material processing system optimization control method comprises the following steps:

[0009] Step 1: Define the system state, action, and corresponding strategy according to the system optimization goal, and let the state variable express The free space of cache libraries, , , express The status of the cache The corresponding state space; let the state variable Indicates The available stock of finished product warehouses, including , express Status of finished product warehouse Corresponding state space; define the system state It is composed of the free space of all cache libraries and finished product libraries, that is, ,in represents the state space, Indicates the state space status, is the total number of system states. The distance from the pick-up point to a certain position upstream on the conveyor belt is recorded as the foresight distance, and the maximum foresight distance is set to , since the conveyor belt moves at a constant speed, the foresight distance can be expressed in time; let the variable Representation and Status The relevant foresight distance, where , define the state The control action under the condition of forward sight distance , then the forward-looking control strategy of the system ,in The strategy set consisting of all feasible stable strategies;

[0010] Step 2: According to the working mechanism of the system, determine the decision-making time and process of the system; is the decision moment of the system, where , Indicates that over time The state process of evolution; Decision Moments , the system is in state , at this time according to the state The system will take different actions according to the specific situation.

[0011] Step 3: According to the random characteristics of the system and the decision-making process, a semi-Markov decision-making process SMDP model for the optimization control problem is established; let Representation strategy The SMDP model under control, Indicates time, Indicates the state process, is the state space, It is a collection of actions, is the semi-Markov kernel matrix, The cost function matrix is Indicates the state space status, Indicates the state space status;

[0012] Step 4: According to the semi-Markov kernel matrix And the cost function matrix , define the equivalent infinitesimal factor and equivalent performance vector , and the SMDP model Convert to a continuous-time Markov decision process model ,in is the discount factor, ;

[0013] Step 5: Based on the continuous time Markov decision process model , and use the policy iteration algorithm to obtain the optimal strategy.

[0014] This technical solution is further optimized, according to the state In specific situations, the system will take different actions as follows:

[0015] (1) When all caches are empty, regardless of the status of the finished product library, the processing subject will wait until a workpiece arrives and unload the first workpiece into the corresponding cache before entering the next decision moment. , define the corresponding waiting time as ,but , the action in this case is , here It only means waiting for a random time, not looking ahead to an infinite distance;

[0016] (2) When all caches are full and all finished product warehouses are full, the processing entity cannot perform any operation and can only wait for any customer to arrive and take a finished product before entering the next decision moment. , define the corresponding waiting time as ,but , the action in this case is ;

[0017] (3) When all caches are full and all finished product warehouses are not full, the processing subject takes out a workpiece from the cache for processing according to the set processing workpiece selection rules. After the processing is completed, the processed finished product will be placed in the corresponding finished product warehouse and then enter the next decision moment. , define the corresponding processing time as ,but , the action in this case is ;

[0018] (4) When all caches are neither completely empty nor completely full and all finished product warehouses corresponding to non-empty caches are full, the processing subject cannot perform processing operations and can only wait for the workpiece corresponding to the non-full cache to arrive or the customer corresponding to the non-empty finished product warehouse to arrive, and then enter the next decision moment. , define the corresponding waiting time as ,but , the action in this case is ;

[0019] (5) In other cases, according to the current control strategy , the system will take corresponding actions ,in is the maximum foresight distance. If there is at least one non-full cache corresponding to the workpiece within the foresight distance determined by this action, it will wait until the first such workpiece arrives at the pick-up point, then unload it and put it into the corresponding cache, and then enter the next decision moment. , define the corresponding waiting time as ,but If there is no non-full cache corresponding workpiece within the foresight distance determined by the action, a workpiece will be taken out from the cache for processing according to the set processing workpiece selection rules. During this period, any arriving workpiece will be lost. After the processing is completed, the processed finished product will be placed in the corresponding finished product warehouse, and then enter the next decision moment. , define the corresponding processing time as , in order to ensure the ineffectiveness of the decision-making process, .

[0020] This technical solution is further optimized, and the step 3 is specifically as follows:

[0021] Step 3.1: Based on step 1, define ,in express The status of the cache The corresponding state space, express The status of the cache The corresponding state space, express Status of finished product warehouse The corresponding state space, express Status of finished product warehouse The corresponding state space, Indicates the state space status, Indicates the total number of states;

[0022] Step 3.2: Based on step 2, define ;

[0023] Step 3.3: Define the strategy The system's sojourn time distribution function matrix is ,in Indicates in status Next Action Transfer to next state The sojourn time distribution function of the system is defined as ,in Indicates in status Take Action Transfer to next state The specific calculation method is as follows:

[0024] (1) All caches are empty. Assume that the type of the first arriving workpiece is , at this time there is

[0025]

[0026]

[0027] in, , represents the total workpiece arrival rate, is the multiplication symbol, Indicates passing Time The status of the finished product warehouse changes from Transfer to The probability of , the specific calculation formula is as follows:

[0028]

[0029] in, ;

[0030] (2) All caches are full and all finished product warehouses are also full. Assume that the first customer to arrive is of type , at this time there is

[0031]

[0032]

[0033] in, Represents the total customer arrival rate.

[0034] (3) All caches are full and all finished product libraries are not full, and the processing workpiece type is set to , at this time there is

[0035]

[0036]

[0037] in, and For the The parameters of the Erlang distribution that the processing time of the workpiece of the type obeys, is the number of workpiece types that meet the processing conditions. hour

[0038]

[0039] when hour, The calculation method of is the same as in (1);

[0040] (4) All caches are neither completely empty nor completely full, and all finished product libraries corresponding to non-empty caches are full.

[0041]

[0042] If the workpiece arrives first, assume that the first workpiece that meets the requirements is of type

[0043]

[0044] If the customer arrives first, assume that the first customer type that meets the requirements is

[0045]

[0046] in, It is the sum of the arrival rates of workpieces corresponding to all non-full buffer warehouses and the arrival rates of customers corresponding to non-empty finished product warehouses;

[0047] (5) In other cases, when there are workpieces corresponding to the non-full buffer within the current viewing distance, wait for the unloading process. is the type of workpiece corresponding to the first non-full buffer within the foresight distance.

[0048]

[0049]

[0050] in, is the sum of the arrival rates of all workpieces in non-full buffers. The calculation method of is the same as in (1);

[0051] When there is no workpiece corresponding to the non-full buffer within the current viewing distance, the workpiece processing process, For the workpiece type,

[0052]

[0053]

[0054] in, is the number of types that meet the processing conditions, ;

[0055] If the processing time ,when hour

[0056]

[0057] when hour, The calculation method of is the same as in (1);

[0058] If the processing time ,when hour

[0059]

[0060] when hour, The calculation method of is the same as in (1);

[0061] For in action Under the state pair that cannot actually evolve, let , ;

[0062] By using the residence time distribution function matrix and embedding chain transition probability matrix , we can define the semi-Markov kernel matrix ,in Indicates the multiplication of corresponding elements of the matrix;

[0063] Step 3.4, order is the system performance function matrix, where Indicates that the system is in state Take Action Transfer to next state The expected cost per unit time of the previous system; according to the specific optimization goal, consider the following costs:

[0064] : Cache Library The storage cost per unit time is ;

[0065] :Processing per unit time The price of the variety artifacts;

[0066] : Waiting cost per unit time;

[0067] :Finished product warehouse The storage cost per unit time;

[0068] :Complete one Instant rewards for variety artifacts.

[0069] This technical solution is further optimized, The specific calculation method is as follows:

[0070] (1) All buffers are empty.

[0071]

[0072] in, Indicates The capacity of the finished product warehouse;

[0073] (2) All buffer warehouses are full and all finished product warehouses are also full. Assume that the first arriving customer is , at this time there is

[0074]

[0075] in, Indicates The capacity of the cache;

[0076] (3) All buffers are full and all finished product bins are not full, and the workpiece type is set to , at this time there is

[0077]

[0078] in, Is the first The cost conversion function that amortizes the processing reward of the workpiece to the entire decision cycle, It is The processing time distribution function of the workpiece The definition is as follows

[0079]

[0080] (4) All buffers are neither completely empty nor completely full. All finished product warehouses corresponding to non-empty buffers are full.

[0081]

[0082] (5) In other cases, when there are workpieces corresponding to the non-full buffer within the current viewing distance, wait for the unloading process. is the type of workpiece corresponding to the first non-full buffer within the foresight distance.

[0083]

[0084] When there is no workpiece corresponding to the non-full buffer within the current viewing distance, the workpiece processing process, Processing workpiece type, in this case

[0085]

[0086] in, Is the first The conversion function that amortizes the processing cost of a workpiece to the entire decision cycle, is the conversion function of waiting cost amortized over the entire decision cycle, where the integral variable satisfy ;

[0087] The technical solution is further optimized, and the strategy iteration algorithm is performed according to the following steps:

[0088] Step 5.1: Initialize strategy ,set up , is a smaller constant;

[0089] Step 5.2: Solve the equation get ,in is the unit column vector, For equivalent Markov decision processes in strategy According to the Poisson equation get ,in is the identity matrix, For the vector Related constants:

[0090]

[0091]

[0092]

[0093] when hour, Indicated in strategy In state The evaluation duration of stay;

[0094] Step 5.3, pass Calculate the update strategy ;

[0095] Step 5.4: Update strategy Calculated ,judge Is it true? If the condition is met, the algorithm ends and we get is the optimal forward-looking control strategy; otherwise, let Go to step 5.2 to continue.

[0096] Different from the prior art, the above technical solution has the following beneficial effects:

[0097] The optimization control method proposed in the present invention is aimed at a multi-dimensional flexible CSPS system with controllable forward-looking distance, random arrival of multiple varieties of workpieces, and random changes in product demand. These forms will be the norm in future production models, which are not taken into account by the existing technology, and the proposed method can effectively improve the productivity of the system and better meet customer needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 It is a schematic diagram of a conveyor belt material processing system for mixed flow of multiple types of workpieces;

[0099] Figure 2 This is the workflow diagram of the conveyor belt material processing system for mixed flow of multiple varieties of workpieces. DETAILED DESCRIPTION

[0100] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.

[0101] See also Figure 1 As shown, it is a schematic diagram of a conveyor belt feeding processing system for mixed flow of multiple varieties of workpieces. A preferred embodiment of the present invention is a method for optimizing control of a conveyor belt feeding processing system for mixed flow of multiple varieties of workpieces, which is applied to Figure 1 The figure shows a uniformly moving conveyor belt, a photoelectric sensor, Different types of workpieces, processing subjects, In the production system, there are cache libraries of different capacities and corresponding finished product libraries of different capacities.

[0102] The processing body is located on one side of the conveyor belt, and a buffer store and a finished product store are respectively arranged on both sides of the processing body. The photoelectric sensor is located upstream of the fixed picking point of the conveyor belt; Different types of workpieces arrive at the processing station randomly along the conveyor belt in an independent Poisson process. The arrival rate of the workpiece type is ,in The processing subject can unload the workpiece that arrives at the picking point and put it into the corresponding cache, or take out a workpiece from the cache for processing. The finished product will be stored in the corresponding finished product warehouse. The processing time of each type of workpiece is subject to the parameter of The photoelectric sensor can detect and record the position and time information of the workpiece within a distance from the pick-up point to a certain position upstream on the conveyor belt; The buffers of different capacities are used to store the unprocessed workpieces unloaded from the conveyor belt. The capacity of the cache is ; Finished product warehouses of different capacities are used to store processed finished products. The capacity of the finished product warehouse is At any time, customers corresponding to various finished products also arrive randomly in independent Poisson processes. The arrival rate of customers of this category is , and each customer takes a processed finished product from the finished product warehouse. If the finished product warehouse is empty, the customer will automatically be lost.

[0103] In this embodiment, when selecting the workpiece to be processed in the processing stage, in order to meet customer needs as much as possible, among all the workpieces that meet the processing conditions (the cache warehouse is not empty and the corresponding finished product warehouse is not full), the workpiece corresponding to the finished product warehouse with the largest free space is preferentially selected for processing; if there are multiple types of workpieces whose corresponding finished product warehouses have the largest free space at the same time, the workpiece corresponding to the buffer warehouse with the smallest free space is selected from them for processing; if there are still multiple types of workpieces whose corresponding buffer warehouses have the smallest free space at the same time, one type of workpiece is randomly selected from them for processing.

[0104] The optimization control method is carried out in the following steps:

[0105] Step 1: Define the system state, action, and corresponding strategy according to the system optimization goal. express The free space of cache libraries, , , express The status of the cache The corresponding state space; let the state variable Indicates The available stock of finished product warehouses, including , express Status of finished product warehouse Corresponding state space; define the system state It is composed of the free space of all cache libraries and finished product libraries, that is, ,in represents the state space, Indicates the state space status, is the total number of system states. The distance from the pick-up point to a certain position upstream on the conveyor belt is recorded as the foresight distance, and the maximum foresight distance is set to , since the conveyor belt moves at a constant speed, the foresight distance can be expressed in time; let the variable Representation and Status The relevant foresight distance, where , define the state The control action under the condition of forward sight distance , then the forward-looking control strategy of the system ,in, The strategy set consisting of all feasible stable strategies;

[0106] Step 2: According to the working mechanism of the system, determine the decision-making time and process of the system; is the decision moment of the system, where , Indicates that over time The state process of evolution; Decision Moments , the system is in state ,like Figure 2 As shown, at this time, according to the status Depending on the specific situation, the system will take different actions:

[0107] (1) When all caches are empty, regardless of the status of the finished product library, the processing subject will wait until a workpiece arrives and unload the first workpiece into the corresponding cache before entering the next decision moment. , define the corresponding waiting time as ,but , the action in this case is , here It only means waiting for a random time, not looking ahead to an infinite distance;

[0108] (2) When all caches are full and all finished product warehouses are full, the processing entity cannot perform any operation and can only wait for any customer to arrive and take a finished product before entering the next decision moment. , define the corresponding waiting time as ,but , the action in this case is ;

[0109] (3) When all caches are full and all finished product warehouses are not full, the processing subject takes out a workpiece from the cache for processing according to the set processing workpiece selection rules. After the processing is completed, the processed finished product will be placed in the corresponding finished product warehouse and then enter the next decision moment. , define the corresponding processing time as ,but , the action in this case is ;

[0110] (4) When all caches are neither completely empty nor completely full and all finished product warehouses corresponding to non-empty caches are full, the processing subject cannot perform processing operations and can only wait for the workpiece corresponding to the non-full cache to arrive or the customer corresponding to the non-empty finished product warehouse to arrive, and then enter the next decision moment. , define the corresponding waiting time as ,but , the action in this case is ;

[0111] (5) In other cases, according to the current control strategy , the system will take corresponding actions ,in is the maximum foresight distance. If there is at least one non-full cache corresponding to the workpiece within the foresight distance determined by this action, it will wait until the first such workpiece arrives at the pick-up point, then unload it and put it into the corresponding cache, and then enter the next decision moment. , define the corresponding waiting time as ,but If there is no non-full cache corresponding workpiece within the foresight distance determined by the action, a workpiece will be taken out from the cache for processing according to the set processing workpiece selection rules. During this period, any arriving workpiece will be lost. After the processing is completed, the processed finished product will be placed in the corresponding finished product warehouse, and then enter the next decision moment. , define the corresponding processing time as , in order to ensure the ineffectiveness of the decision-making process, ;

[0112] Step 3: According to the random characteristics of the system and the decision-making process, a semi-Markov decision process (SMDP) model for the optimization control problem is established; let Representation strategy The SMDP model under control, Indicates time, Indicates the state process, is the state space, It is a collection of actions, is the semi-Markov kernel matrix, The cost function matrix is Indicates the state space status, Indicates the state space The specific steps are as follows:

[0113] Step 3.1: Based on step 1, define ,in express The status of the cache The corresponding state space, express The status of the cache The corresponding state space, express Status of finished product warehouse The corresponding state space, express Status of finished product warehouse The corresponding state space, Indicates the state space status, Indicates the total number of states;

[0114] Step 3.2: Based on step 2, define ;

[0115] Step 3.3: Define the strategy The system's sojourn time distribution function matrix is ,in Indicates in status Next Action Transfer to next state The sojourn time distribution function of the system is defined as ,in Indicates in status Take Action Transfer to next state The specific calculation method is as follows:

[0116] (1) All caches are empty. Assume that the type of the first arriving workpiece is , at this time there is

[0117]

[0118]

[0119] in, , represents the total workpiece arrival rate, is the multiplication symbol, Indicates passing Time The status of the finished product warehouse changes from Transfer to The probability of , the specific calculation formula is as follows:

[0120]

[0121] in, ;

[0122] (2) All caches are full and all finished product warehouses are also full. Assume that the first customer to arrive is of type , at this time there is

[0123]

[0124]

[0125] in, Represents the total customer arrival rate.

[0126] (3) All caches are full and all finished product libraries are not full, and the processing workpiece type is set to , at this time there is

[0127]

[0128]

[0129] in, and For the The parameters of the Erlang distribution that the processing time of the workpiece of the type obeys, is the number of workpiece types that meet the processing conditions. hour

[0130]

[0131] when hour, The calculation method of is the same as in (1);

[0132] (4) All caches are neither completely empty nor completely full, and all finished product libraries corresponding to non-empty caches are full.

[0133]

[0134] If the workpiece arrives first, assume that the first workpiece that meets the requirements is of type

[0135]

[0136] If the customer arrives first, assume that the first customer type that meets the requirements is

[0137]

[0138] in, It is the sum of the arrival rates of workpieces corresponding to all non-full buffer warehouses and the arrival rates of customers corresponding to non-empty finished product warehouses;

[0139] (5) In other cases, when there are workpieces corresponding to the non-full buffer within the current viewing distance, wait for the unloading process. is the type of workpiece corresponding to the first non-full buffer within the foresight distance.

[0140]

[0141]

[0142] in, is the sum of the arrival rates of all workpieces in non-full buffers. The calculation method of is the same as in (1);

[0143] When there is no workpiece corresponding to the non-full buffer within the current viewing distance, the workpiece processing process, For the workpiece type,

[0144]

[0145]

[0146] in, is the number of types that meet the processing conditions, ;

[0147] If the processing time ,when hour

[0148]

[0149] when hour, The calculation method of is the same as in (1);

[0150] If the processing time ,when hour

[0151]

[0152] when hour, The calculation method of is the same as in (1);

[0153] For in action Under the state pair that cannot actually evolve, let , ;

[0154] By using the residence time distribution function matrix and embedding chain transition probability matrix , we can define the semi-Markov kernel matrix ,in Indicates the multiplication of corresponding elements of the matrix;

[0155] Step 3.4, order is the system performance function matrix, where Indicates that the system is in state Take Action Transfer to next state The expected cost per unit time of the previous system; according to the specific optimization goal, consider the following costs:

[0156] : Cache Library The storage cost per unit time is ;

[0157] :Processing per unit time The price of the variety artifacts;

[0158] : Waiting cost per unit time;

[0159] :Finished product warehouse The storage cost per unit time;

[0160] :Complete one Immediate payment for variety artifacts;

[0161] The specific calculation method is as follows:

[0162] (1) All buffers are empty.

[0163]

[0164] in, Indicates The capacity of the finished product warehouse;

[0165] (2) All buffer warehouses are full and all finished product warehouses are also full. Assume that the first arriving customer is , at this time there is

[0166]

[0167] in, Indicates The capacity of the cache;

[0168] (3) All buffers are full and all finished product bins are not full, and the workpiece type is set to , at this time there is

[0169]

[0170] in, Is the first The cost conversion function that amortizes the processing reward of the workpiece to the entire decision cycle, It is The processing time distribution function of the workpiece The definition is as follows

[0171]

[0172] The variables here is the integration variable;

[0173] (4) All buffers are neither completely empty nor completely full. All finished product warehouses corresponding to non-empty buffers are full.

[0174]

[0175] (5) In other cases, when there are workpieces corresponding to the non-full buffer within the current viewing distance, wait for the unloading process. is the type of workpiece corresponding to the first non-full buffer within the foresight distance.

[0176]

[0177] When there is no workpiece corresponding to the non-full buffer within the current viewing distance, the workpiece processing process, Processing workpiece type, in this case

[0178]

[0179] in, Is the first The conversion function that amortizes the processing cost of a workpiece to the entire decision cycle, is the conversion function of waiting cost amortized over the entire decision cycle, where the integral variable satisfy ;

[0180] Step 4: According to the semi-Markov kernel matrix And the cost function matrix , define the equivalent infinitesimal factor and equivalent performance vector , and the SMDP model Convert to a continuous-time Markov decision process model ,in is the discount factor, ;

[0181] Step 5: Based on the continuous time Markov decision process model , use the policy iteration algorithm to get the optimal strategy; the policy iteration algorithm is carried out in the following steps:

[0182] Step 5.1: Initialize strategy ,set up , is a smaller constant;

[0183] Step 5.2: Solve the equation get ,in is the unit column vector, For equivalent Markov decision processes in strategy According to the Poisson equation get ,in is the identity matrix, For the vector Related constants:

[0184]

[0185]

[0186]

[0187] when hour, Indicated in strategy In state The evaluation duration of stay;

[0188] Step 5.3, pass Calculate the update strategy ;

[0189] Step 5.4: Update strategy Calculated ,judge Is it true? If the condition is met, the algorithm ends and we get is the optimal forward-looking control strategy; otherwise, let Go to step 5.2 to continue.

[0190] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "include..." or "comprise..." do not exclude the existence of other elements in the process, method, article or terminal device including the elements. In addition, in this article, "greater than", "less than", "exceed" and the like are understood to exclude the number itself; "above", "below", "within" and the like are understood to include the number itself.

[0191] Although the above embodiments have been described, once those skilled in the art know the basic creative concepts, they can make additional changes and modifications to these embodiments. Therefore, the above are only embodiments of the present invention and do not limit the patent protection scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An optimization control method for a conveyor belt material processing system with mixed flow of multiple types of workpieces, characterized in that: The conveyor belt material processing system for mixed flow of multiple types of workpieces includes: a conveyor belt, a photoelectric sensor, Different types of workpieces, processing bodies, Cache libraries of different capacities and corresponding finished product libraries of different capacities, The processing body is located on one side of the conveyor belt, a buffer storehouse and a finished product storehouse are respectively arranged on both sides of the processing body, and the photoelectric sensor is located upstream of the fixed picking point of the conveyor belt; Different types of workpieces arrive at the processing station randomly along the conveyor belt in an independent Poisson process. The arrival rate of the workpiece type is ,in The processing subject can unload the workpiece that has arrived at the picking point and put it into the corresponding cache, or take out a workpiece from the cache for processing. The finished product will be stored in the corresponding finished product warehouse. The processing time of each type of workpiece is subject to the parameter of The photoelectric sensor can detect and record the position and time information of the workpiece within a distance from the pick-up point to a certain position upstream on the conveyor belt. The buffers of different capacities are used to store the unprocessed workpieces unloaded from the conveyor belt. The capacity of the cache is ; Finished product warehouses of different capacities are used to store processed finished products. The capacity of the finished product warehouse is At any time, customers corresponding to various finished products also arrive randomly in independent Poisson processes. The arrival rate of customers of this category is , and each customer takes a processed finished product from the finished product warehouse. If the finished product warehouse is empty, the customer will automatically be lost; The conveyor belt material processing system optimization control method comprises the following steps: Step 1: Define the system state, action, and corresponding strategy according to the system optimization goal, and let the state variable express The free space of cache libraries, , , express The status of the cache The corresponding state space; let the state variable Indicates The available stock of finished product warehouses, including , express Status of finished product warehouse Corresponding state space; define the system state It is composed of the free space of all cache libraries and finished product libraries, that is, ,in represents the state space set, Indicates the state space status, is the total number of system states, and the distance from the pick-up point to a certain position upstream on the conveyor belt is recorded as the foresight distance. The maximum foresight distance is set to , since the conveyor belt moves at a constant speed, the foresight distance can be expressed in time; let the variable Representation and Status The relevant foresight distance, where , define the state The control action under the condition of forward sight distance , then the forward-looking control strategy of the system ,in, The strategy set consisting of all feasible stable strategies; Step 2: According to the working mechanism of the system, determine the decision-making time and process of the system; is the decision moment of the system, where , Indicates that over time The state process of evolution; Decision Moments , the system is in state , at this time according to the status The system will take different actions according to the specific situation. Step 3: According to the random characteristics of the system and the decision-making process, a semi-Markov decision-making process SMDP model for the optimization control problem is established; let Representation strategy The SMDP model under control, Indicates time, Indicates the state process, is the state space, It is a collection of actions, is the semi-Markov kernel matrix, The cost function matrix, where Indicates the state space status, Indicates the state space status; Step 4: According to the semi-Markov kernel matrix And the cost function matrix , define the equivalent infinitesimal factor and equivalent performance vector , and the SMDP model Convert to a continuous-time Markov decision process model ,in is the discount factor, ; Step 5: Based on the continuous time Markov decision process model , and use the policy iteration algorithm to obtain the optimal strategy.

2. The optimization control method for a conveyor belt material processing system with mixed flow of multiple types of workpieces as claimed in claim 1, characterized in that: According to the status In specific situations, the system will take different actions as follows: (1) When all caches are empty, regardless of the status of the finished product library, the processing subject will wait until a workpiece arrives and unload the first workpiece into the corresponding cache before entering the next decision moment. , define the corresponding waiting time as ,but , the action in this case is , here It just means waiting for a random time; (2) When all caches are full and all finished product warehouses are full, the processing entity cannot perform any operation and can only wait for any customer to arrive and take a finished product before entering the next decision moment. , define the corresponding waiting time as ,but , the action in this case is ; (3) When all caches are full and all finished product warehouses are not full, the processing subject takes out a workpiece from the cache for processing according to the set processing workpiece selection rules. After the processing is completed, the processed finished product will be placed in the corresponding finished product warehouse and then enter the next decision moment. , define the corresponding processing time as ,but , the action in this case is ; (4) When all caches are neither completely empty nor completely full and all finished product warehouses corresponding to non-empty caches are full, the processing subject cannot perform processing operations and can only wait for the workpiece corresponding to the non-full cache to arrive or the customer corresponding to the non-empty finished product warehouse to arrive, and then enter the next decision moment. , define the corresponding waiting time as ,but , the action in this case is ; (5) In other cases, according to the current control strategy , the system will take corresponding actions ,in is the maximum foresight distance. If there is at least one non-full cache corresponding to the workpiece within the foresight distance determined by this action, it will wait until the first such workpiece arrives at the pick-up point, then unload it and put it into the corresponding cache, and then enter the next decision moment. , define the corresponding waiting time as ,but If there is no non-full cache corresponding workpiece within the foresight distance determined by the action, a workpiece will be taken out from the cache for processing according to the set processing workpiece selection rules. During this period, any arriving workpiece will be lost. After the processing is completed, the processed finished product will be placed in the corresponding finished product warehouse, and then enter the next decision moment. , define the corresponding processing time as To ensure the decision-making process has no aftereffect, .

3. The optimization control method for a conveyor belt material processing system with mixed flow of multiple types of workpieces as claimed in claim 1, characterized in that: The step 3 is as follows: Step 3.1: Based on step 1, define ,in express The status of the cache The corresponding state space, express The status of the cache The corresponding state space, express Status of finished product warehouse The corresponding state space, express Status of finished product warehouse The corresponding state space, Indicates the state space status, Indicates the total number of states; Step 3.2: Based on step 2, define ; Step 3.3: Define the strategy The system’s sojourn time distribution function matrix is ,in Indicates in status Next Action Transfer to next state The residence time distribution function of The embedded chain transition probability matrix of the system is defined as ,in Indicates in status Take Action Transfer to next state The specific calculation method is as follows: (1) All caches are empty. Assume that the type of the first arriving workpiece is , at this time there is in, , represents the total workpiece arrival rate, is the multiplication symbol, Indicates passing Time The status of the finished product warehouse changes from Transfer to The probability of , The specific calculation formula is as follows: in, ; (2) All caches are full and all finished product warehouses are also full. Assume that the first customer to arrive is of type , at this time there is in, represents the total customer arrival rate; (3) All caches are full and all finished product libraries are not full, and the processing workpiece type is set to , at this time there is in, and For the The parameters of the Erlang distribution that the processing time of the workpiece of the type obeys, is the number of workpiece types that meet the processing conditions. hour when hour, The calculation method of is the same as in (1); (4) All caches are neither completely empty nor completely full, and all finished product libraries corresponding to non-empty caches are full. If the workpiece arrives first, assume that the first workpiece that meets the requirements is of type , If the customer arrives first, assume that the first customer type that meets the requirements is in, It is the sum of the arrival rates of workpieces corresponding to all non-full buffer warehouses and the arrival rates of customers corresponding to non-empty finished product warehouses; (5) In other cases, when there are workpieces corresponding to the non-full buffer within the current viewing distance, wait for the unloading process. is the type of workpiece corresponding to the first non-full buffer within the foresight distance. in, is the sum of the arrival rates of all workpieces in non-full buffers. The calculation method of is the same as in (1); When there is no workpiece corresponding to the non-full buffer within the current viewing distance, the workpiece processing process, For the workpiece type, in, is the number of types that meet the processing conditions, ; If the processing time ,when hour when hour, The calculation method of is the same as in (1); If the processing time ,when hour when hour, The calculation method of is the same as in (1); For in action Under the state pair that cannot actually evolve, let , ; By using the residence time distribution function matrix and the embedding chain transition probability matrix , we can define the semi-Markov kernel matrix ,in Indicates the multiplication of corresponding elements of the matrix; Step 3.4, order is the system performance function matrix, where Indicates that the system is in state Take Action Transfer to next state The expected cost per unit time of the previous system; according to the specific optimization goal, consider the following costs: : Cache Library The storage cost per unit time is ; :Processing per unit time The price of the variety artifacts; : Waiting cost per unit time; :Finished product warehouse The storage cost per unit time; :Complete one Instant rewards for variety artifacts.

4. The method for optimizing control of a conveyor belt material processing system for mixed flow of multiple types of workpieces as claimed in claim 3, characterized in that: Said The specific calculation method is as follows: (1) All buffers are empty, then there is in, Indicates The capacity of the finished product warehouse; (2) All buffer warehouses are full and all finished product warehouses are also full. Assume that the first arriving customer is , at this time there is in, Indicates The capacity of the cache; (3) All buffers are full and all finished product bins are not full, and the workpiece type is set to , at this time there is in, Is the first The cost conversion function that amortizes the processing reward of the workpiece to the entire decision cycle, It is The processing time distribution function of the workpiece The definition is as follows (4) All buffers are neither completely empty nor completely full. All finished product warehouses corresponding to non-empty buffers are full. (5) In other cases, when there are workpieces corresponding to the non-full buffer within the current viewing distance, wait for the unloading process. The type of workpiece corresponding to the first non-full buffer within the foresight distance. When there is no workpiece corresponding to the non-full buffer within the current viewing distance, the workpiece processing process, Processing workpiece type, in this case in, Is the first The conversion function that amortizes the processing cost of a workpiece to the entire decision cycle, is the conversion function of waiting cost amortized over the entire decision cycle, where the integral variable satisfy .

5. The method for optimizing control of a conveyor belt material processing system for mixed flow of multiple types of workpieces as claimed in claim 1, characterized in that: The strategy iteration algorithm is performed as follows: Step 5.1: Initialize strategy ,set up , is a smaller constant; Step 5.2: Solve the equation get ,in is the unit column vector, For the equivalent Markov decision process in strategy According to the Poisson equation get ,in is the identity matrix, For the vector Related constants: when hour, Indicated in strategy In state The evaluation duration of stay; Step 5.3, pass Calculate the update strategy ; Step 5.4: Update strategy Calculated ,judge Is it true? If the condition is met, the algorithm ends and we get is the optimal forward-looking control strategy; otherwise, let Go to step 5.2 to continue.

Images