A method for efficient algorithm selection for personalized customization production lines

Abstract

The present invention provides a method for selecting efficient algorithms for personalized customized production lines, relating to the field of smart factory technology. The method integrates various scheduling algorithms and designs a unique algorithm selection mechanism. It uses a constraint-based algorithm selector, then a hyperparameter optimizer to perform optimal parameter configuration, and finally sends it to a solver to solve the scheduling strategy. The present invention solves the deficiency that the flexible production line scheduling system cannot autonomously match algorithms. For production operation problems, it can significantly reduce the overall system energy consumption, improve manufacturing production efficiency, shorten the maximum completion time, and significantly reduce the waste of materials and time. In addition, since the products produced using the optimal algorithm are of high quality, the production process is smoother, and economic benefits are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of smart factories, and in particular to a method for selecting an efficient algorithm for a personalized customized production line. Background Art

[0002] With the powerful entry of artificial intelligence into the manufacturing industry, many manufacturing companies are seeking efficient, intelligent manufacturing products to meet growing public demand. The flexible production scheduling problem inherent in this is becoming a key research topic for modern intelligent manufacturing companies. The demand for customized products tailored to meet customer needs is growing. Consequently, the algorithm selection and solution systems for these scheduling problems are a hot topic for development teams.

[0003] Solving the Flexible Job Shop Scheduling Problem (FJSSP) is a crucial research topic in combinatorial optimization and production management. It is an extension of the classic Job Shop Scheduling Problem (JSSP) and is considered strongly NP-hard. In the FJSSP, multiple machines may be required to process the same process. The FJSSP consists of two subproblems: the first is to assign a set of optional machines to a given process (task), and the second is to calculate the completion time for a sequence of processes assigned to a given machine. While the FJSSP only adds the step of assigning a set of optional machines to a given process, solving the FJSSP requires significant improvements in chromosome encoding, decoding, crossover, and mutation, making it significantly more challenging.

[0004] GAUSS proposes a new design and implementation method for an algorithm recommendation system. Traditional recommendation systems summarize performance data offline and are limited in their coverage of application domains (e.g., using weak learning methods such as neural network training). However, GAUSS uses relational descriptions for domain modeling and dynamically interacts with the environment to collect the data needed to mine recommendations. Therefore, it is an online recommendation system. New information about problems and algorithms can be easily incorporated without having to retrain old data. In addition, end users of the recommendation system can query the system to obtain the basis for recommendations (relational rules that associate problem features with the impact of algorithm performance). However, due to the limitations of its system framework, the GUASS system is not compatible with metaheuristic algorithms, and therefore relatively effective metaheuristic algorithms cannot be used in flexible production scheduling problems.

[0005] Segmenting nontrivial images is one of the most important tasks in image processing. While easy for humans, it is extremely difficult for computers. Researchers studying the human segmentation process have proposed an algorithm selection model based on performance prediction to bridge the knowledge gap between images and segmentation algorithms. By learning from human experience, the optimal segmentation algorithm can be found for each image. However, this image segmentation algorithm selector is only applicable to image processing problems. Prediction-based algorithm selection models cannot effectively meet the performance requirements of scheduling policy problems.

[0006] Existing SCOS research primarily focuses on providing high-quality services to users based on Quality of Service (QoS). Little work has been done on providing high-quality services with low energy consumption. Therefore, some researchers have studied the SCOS problem based on QoS and QoS Energy Consumption (QoS-EnCon). First, a multi-objective service composition model was established; QoS and Energy Consumption Evaluation (ECON) and a dimensionless QoS objective function were developed. To effectively address the multi-objective SCOS problem, a new global optimization algorithm, the Group Leadership Algorithm (GLA), was proposed. In GLA, the influence of leaders in social groups is used as inspiration, and evolutionary techniques are designed into a group architecture. Next, the mapping from the SCOS problem to the GLA solution is studied. A new GLA-Pareto-based multi-objective optimization algorithm is proposed. A method for addressing the SCOS problem combining Pareto solutions with GLA concepts is proposed, and key operators are designed to implement the Pareto-GA. However, the scheduling selection mechanism of cloud manufacturing systems is not suitable for flexible production requirements. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned existing technologies and provide a method for selecting efficient algorithms for personalized customized production lines. The method intelligently and automatically matches corresponding algorithms and solvers according to the characteristics of customized order data, selects the optimal algorithm based on calculation time and calculation cost, and then uses this algorithm to solve the scheduling strategy, which solves the deficiency that the flexible production line scheduling system cannot autonomously match algorithms, and utilizes the hyperparameter optimization mechanism based on reinforcement learning to intelligently match the algorithm parameters used internally, and finally outputs the scheduling strategy for personalized customized orders as a production reference for manufacturing enterprises, thereby improving the production efficiency and economic benefits of the enterprises.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0009] A method for efficient algorithm selection for personalized customized production lines. First, the server is logged in using the key of each user, and the required product parameters and order requirements are entered on the order page. The order information will be sent to the manufacturer's client, and the manufacturer will perform primary algorithm filtering based on the order information characteristics. The customer's order enters the algorithm selector part. According to customer needs, the order information is transmitted to the data stream through the interface designed in the algorithm pool. The constraint-based algorithm filter in the local server is used to filter and remove inappropriate algorithm clusters; then the hyperparameter optimizer is entered to optimize the internal parameters of the selected algorithm suitable for the data information characteristics of this order; finally, the algorithm solver part is entered, the order data stream is input into the algorithm with configured parameters for solving, and the output scheduling results are displayed on the front-end interface.

[0010] Furthermore, after the order information is extracted, the algorithm selector classifies the algorithm, the hyperparameter optimizer implements the internal hyperparameter configuration of the algorithm, and the algorithm solver solves the scheduling result, corresponding logs will be sent to the back-end database for storage and recording.

[0011] Furthermore, the specific method of the algorithm selector part is:

[0012] Step 2.1: Obtain order information, including the process string, product quantity, optimization target quantity, four optimization target options, and whether flexible processing is required. The optimization target quantity is 0 for a single target and 1 for multiple targets. The four optimization target options are strings of 1 1 1 1, where 0 indicates the target is considered and 1 indicates it is not. Flexible processing is determined based on the process string.

[0013] The additional information that the algorithm needs to obtain includes:

[0014] (1) The processing power and fixed power of each machine, the fixed power of the workshop and the parts transmission power;

[0015] (2) The transfer time of the workpiece between different machines;

[0016] (3) Delivery date; if the target includes delay time, delivery date data is required;

[0017] (4) Algorithm type: Single-objective algorithms include classical algorithms, metaheuristic algorithms, and artificial intelligence algorithms; this may not be specified.

[0018] Step 2.2: Divide the optimization objective into single objective and multi-objective categories. If it is a single objective, proceed to step 2.3; if it is a multi-objective, proceed to step 2.5.

[0019] Step 2.3: Determine whether the algorithm type is specified; if so, determine whether it is flexible processing, and then classify it into the corresponding flexible or non-flexible algorithm pool for optimization, and jump to step 2.6; if not, execute step 2.4;

[0020] Step 2.4: Determine whether it is flexible processing, and then further classify it into four scales: small, medium, large, and extra-large. According to flexibility and scale, it is divided into corresponding algorithm pools for optimization, and then jump to step 2.6;

[0021] Step 2.5: Determine whether it is flexible, then classify it into the corresponding flexible or non-flexible multi-objective algorithm pool for optimization, and go to step 2.6;

[0022] Step 2.6: Select the best result from the corresponding algorithm pool and return it.

[0023] Furthermore, in the hyperparameter optimizer part, deep reinforcement learning is used to train the hyperparameter optimizer offline so that the results can be directly returned when applied. The process of the hyperparameter optimizer based on deep reinforcement learning is as follows:

[0024] The agent reads the hyperparameter table required by the algorithm;

[0025] The agent sequence outputs the distribution of hyperparameters to be optimized;

[0026] Sampling a normal distribution of hyperparameters;

[0027] Run on the validation dataset to get the reward function;

[0028] Use reinforcement learning methods to calculate the loss function and update the internal parameters of the agent;

[0029] Output the obtained approximate optimal strategy and save the plot in the log.

[0030] Furthermore, the hyperparameter optimizer uses the proximal policy optimization algorithm, namely the PPO algorithm, to handle the hyperparameter adaptive matching problem in the production line scheduling process. The specific process is as follows:

[0031] Step 3.1: Initialize network weights and set hyperparameters;

[0032] Step 3.2: According to the probability distribution π of the participant network from state to action output in the old policy θ′ (a t |s t ) Update the sample a at time t t ; Among them, s t is the state variable at time t;

[0033] Step 3.3: Receive the partial reward function r at time t tTransmitted to the state variable s at the next moment t+1 ;

[0034] Step 3.4: Store the trajectory vector (s) of the agent at time t t , a t , r t ) and the probability calculation of the old strategy π θ′ (a t |s t );

[0035] Step 3.5: Determine whether the single data set has been solved; if so, proceed to step 3.6; if not, return to step 3.1;

[0036] Step 3.6: Calculate the total reward value through the overall reward function

[0037] Step 3.7: Calculate the advantage estimate function

[0038] Step 3.8: Calculate the full loss function L(θ,φ);

[0039] Step 3.9: Use the backpropagation algorithm to update the agent's internal parameters θ and φ;

[0040] Step 3.10: Determine whether the number of updates is less than or equal to the threshold K; if so, return to step 3.7; if not, use the backpropagation algorithm to update the agent's internal parameters θ and φ again;

[0041] Step 3.11: Determine whether the number of data sets is less than or equal to the total number of data sets M; if so, return to step 3.1 and continue optimizing the next data set; if not, end the optimization process.

[0042] Furthermore, the algorithm solver part designs a unified algorithm interface to connect the algorithm library and the database. The interface realizes the interaction between the algorithm and the database through the function connect_mysql. The input of the function includes the host name hostname, username username, password pwd, and database name database_num. The interface function design is divided into two parts: data reading and returning scheduling results.

[0043] Data reading: In Python, connect to the MYSQL server by calling the connect method in the pymysql module; execute the SQL statement to return the order information stored in the specified table in the database, and convert the obtained data into the data format required by the optimization algorithm.

[0044] Return scheduling results: After the optimization algorithm is executed, the algorithm function will return the corresponding scheduling results. Call the pymysql module again and use the SQL statement to write the scheduling results into the database. The steps are as follows:

[0045] First, create a new table named time in the database to store scheduling information; if the table time already exists, clear the data in the table time;

[0046] Insert the scheduling results into the time table using the insert command;

[0047] After writing the scheduling results to the time table, disconnect from the MYSQL database.

[0048] The beneficial effects of adopting the above technical solution are: the method for efficient algorithm selection for personalized customized production lines provided by the present invention integrates various scheduling algorithms, designs a unique algorithm selection mechanism, uses a constraint-based algorithm selector, and then uses a hyperparameter optimizer to perform optimal parameter configuration, and finally sends it to the solver to solve the scheduling strategy. The present invention solves the deficiency that the flexible production line scheduling system cannot match the algorithm autonomously, and uses a hyperparameter optimization mechanism based on reinforcement learning to intelligently match the algorithm parameters used internally, and finally outputs a scheduling strategy for personalized customized orders to provide a production reference for manufacturing enterprises. In response to production operation problems, the present invention can significantly reduce the overall system energy consumption, improve manufacturing production efficiency, shorten the maximum completion time, and significantly reduce the waste of materials and time. Moreover, since the products produced using the most suitable algorithm are of high quality, the production process is smoother, and the economic benefits are effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of a method framework for efficient algorithm selection for a personalized customization production line provided by an embodiment of the present invention;

[0050] Figure 2 A flow chart of a method for selecting an efficient algorithm for a personalized customization production line provided by an embodiment of the present invention;

[0051] Figure 3 An execution flow chart of the algorithm selector provided in an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the design of the internal operation processing structure framework of the intelligent agent provided in an embodiment of the present invention;

[0053] Figure 5 Flowchart of the operation of the adaptive hyperparameter optimizer provided by an embodiment of the present invention;

[0054] Figure 6 Flowchart of the PPO algorithm provided by an embodiment of the present invention;

[0055] Figure 7 This is a diagram of the algorithm solver architecture provided by an embodiment of the present invention;

[0056] Figure 8 A schematic diagram of data stored in the table benchmark_p provided in an embodiment of the present invention;

[0057] Figure 9 A schematic diagram of data stored in the table benchmark_ma provided in an embodiment of the present invention;

[0058] Figure 10 A schematic diagram of data stored in the table benchmark_T provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0060] This embodiment provides a method for selecting an efficient algorithm for a personalized customized production line, such as Figure 1 As shown, it is the framework of the method of this embodiment. First, log in to the server using the key of each user, enter the order page to enter the product parameters and order requirements to be produced, such as the number of products, energy consumption requirements, maximum completion time constraints, extension time agreements, machine equipment load rates, single / multi-target selection, dynamic and static event triggering and other parameters; these order information will be sent to the manufacturer's client, and the manufacturer will perform preliminary algorithm filtering based on the order information characteristics. For example, if the customer requires the use of a static single-target algorithm to schedule the production strategy for the maximum completion time, the customer's order enters the algorithm selector part, and the order information is passed into the data stream through the interface designed in the algorithm pool according to customer needs. The constraint-based algorithm filter in the local server is used for screening, and inappropriate dynamic multi-target algorithm clusters are filtered out. Then, the hyperparameter optimizer is entered to optimize the internal parameters of these selected algorithms suitable for the data information characteristics of this order; finally, the algorithm solver part is entered, and the order data stream is input into the algorithm with configured parameters for solution, and the output results including the scheduling strategy Gantt chart and the scheduling result maximum completion time are displayed on the front-end interface. The flow chart is as follows Figure 2 shown.

[0061] This embodiment collects and transforms algorithms for solving single-objective, multi-objective, and dynamic and static production scheduling problems, including 33 meta-heuristic algorithms and deep Q-network learning algorithms. Meta-heuristic algorithms are methods for solving optimal or suboptimal solutions to complex optimization problems based on the mechanism of computational intelligence. They are sometimes also called intelligent optimization algorithms. Intelligent optimization reveals the design principles of optimization algorithms through the understanding of relevant behaviors, functions, experiences, rules, and mechanisms in systems or fields such as biology, physics, chemistry, society, and art. Under the guidance of specific problem characteristics, corresponding feature models are refined to design intelligent iterative search optimization algorithms. Meta-heuristic algorithms are improvements on heuristic algorithms and are the product of the combination of random algorithms and local search algorithms. Meta-heuristics is an iterative generation process that uses heuristic algorithms to explore and develop the search space through the intelligent combination of different concepts. In this process, learning strategies are used to acquire and master information to effectively discover approximate optimal solutions.

[0062] The function of the algorithm selector is to pre-process the order data submitted by the upstream customers. This embodiment proposes an order routing rule to preliminarily screen the algorithm pool that the order will enter, ensuring that the order can more accurately enter the algorithm pool that it is suitable for and proceed to the next step of the solution process. Figure 3 The specific process is as follows:

[0063] Step 2.1: Obtain order information, including the process string, product quantity, optimization target quantity, four optimization target options, and whether flexible processing is required. The optimization target quantity is 0 for a single target and 1 for multiple targets. The four optimization target options are strings of 1 1 1 1, where 0 indicates the target is considered and 1 indicates it is not. Flexible processing is determined based on the process string.

[0064] The additional information that the algorithm needs to obtain includes:

[0065] (1) The processing power and fixed power of each machine, the fixed power of the workshop and the parts transmission power;

[0066] (2) The transfer time of the workpiece between different machines;

[0067] (3) Delivery date; if the target includes delay time, delivery date data is required;

[0068] (4) Algorithm type: Single-objective algorithms include classical algorithms, metaheuristic algorithms, and artificial intelligence algorithms, which may not be specified. Multi-objective algorithms are mainly genetic algorithms and are not classified.

[0069] Step 2.2: Divide the optimization objective into single objective and multi-objective categories. If it is a single objective, proceed to step 2.3; if it is a multi-objective, proceed to step 2.5.

[0070] Step 2.3: Determine whether the algorithm type (classical algorithm, metaheuristic algorithm, artificial intelligence algorithm) is specified; if it is specified, determine whether it is flexible processing, and then classify it into the corresponding flexible or non-flexible algorithm pool for optimization, and jump to step 2.6; if it is not specified, execute step 2.4;

[0071] Step 2.4: Determine whether it is flexible processing, and then further classify it into four scales: small, medium, large, and extra-large. According to flexibility and scale, it is divided into corresponding algorithm pools for optimization, and then jump to step 2.6;

[0072] Step 2.5: Determine whether it is flexible, then classify it into the corresponding flexible or non-flexible multi-objective algorithm pool for optimization, and go to step 2.6;

[0073] Step 2.6: Select the best result from the corresponding algorithm pool and return it.

[0074] For orders sent from the system front end, an instruction document is attached to the order for initial screening of the algorithm pool of the algorithm selection solver. Orders with specific characteristics will be sent to the corresponding algorithm pool through the routing classifier, and the optimal solution will be calculated and returned.

[0075] After determining the algorithm cluster or algorithm pool, the next step is to solve the problem of determining the internal hyperparameters of the scheduling algorithm in the corresponding algorithm cluster / pool. This embodiment uses a deep reinforcement learning method - the proximal policy optimization (PPO) algorithm to solve the hyperparameter adaptive matching problem in the production line scheduling process.

[0076] The design principle of the intelligent agent in the PPO algorithm is as follows Figure 4 As shown in the figure, the agent consists of three main parts: the Input layer, the Core layer, and the Output layer. The Input layer is a fully connected network. This layer converts data into high-dimensional abstract features, enabling the subsequent Long Short-Term Memory (LSTM) network to more comprehensively perceive the state of the input data stream. The intermediate Core LSTM layer identifies and learns the intrinsic connections between the algorithm's hyperparameters, as well as the implicit associations in hyperparameter selection. The Output layer also uses a fully connected network. The output of this network is a normal distribution of hyperparameters, which requires sampling. To learn the explicit relationship between hyperparameters, the agent uses the output at time t as the input at time t+1. A standard normal distribution is also used as the initial state.

[0077] In this embodiment, the PPO algorithm is used to train the agent. This is a DRL algorithm based on the Actor-Critic framework, which has good solution space search capabilities and training stability. The reward function in the PPO algorithm is shown in formula (1):

[0078]

[0079] Among them, r can represent the target value in the single-objective algorithm solution, specifically r = -C max or r=-E t Or r=-T t Or r=-W t ; For multi-objective algorithms, r is a composite value, which is the sum of the weighted product values of each objective: if an algorithm needs to solve the hyperparameters of four objectives at the same time, then r = m*C1+n*C2+p*C3+q*C4.

[0080] First, the policy gradient algorithm is introduced, in which the agent, i.e., the participant, starts from a specific state s0 and continuously interacts with the environment to form a trajectory τ = {s0, a0, r0, s1, a1, r1, ..., s T The probability of each τ is shown in formula (2), where the probability of p is determined by the environment.

[0081]

[0082] Therefore, in the strategy π θ Under this condition, the agent's expected reward is similar to formula (4). The goal of RL is to make R θ As large as possible, and use the gradient method to update the policy network. Take the natural logarithm of formula (1) and combine it with formula (3) to obtain the calculation formula of gradient (4).

[0083]

[0084]

[0085] In formula (4), π θ (a t |s t ) is the probability distribution of the actor network output from state to action. Representing an advantage function is to update π θ (a t |s t ) weight, the calculation formula is shown in formula (5). The neural network fitting value function can be used It is the output of the Critic network and baseline.

[0086]

[0087] In the same policy environment, the policy agent and the policy update algorithm interact. Therefore, the trajectory can only be used once, resulting in a huge waste of data. This method is called on the policy. In PPO, the policy that interacts with the environment is the old policy π′ θ , the updated strategy is the new strategy π θ , which is called offpolicy, and its advantage is that the agent can interact with the environment multiple times to generate a large amount of data and then update the policy. The PPO algorithm updates the policy by calculating the gradient estimate through importance sampling, as shown in formula (6).

[0088]

[0089] In order to use importance sampling, it is assumed that the probability distributions of the two strategies are very close. When calculating the gradient through formula 6), the KL divergence constraint should be added. But this brings trouble to the calculation, π θ / π′ θ Equivalent to a proportional coefficient, which can be limited by the clip function to ensure that π θ and π′ θ The similarity is shown in the following formula (7):

[0090]

[0091]

[0092] Here, ε∈(0,1) is a hyperparameter that roughly reflects the similarity between the two strategies.

[0093] In addition to using L CLIP (θ) In addition to updating the Actor network with the loss function, the mean square error function L can also be used V (φ) Update the Critic network, as shown in formula (9). By deriving the above formula, the corresponding loss function can be calculated to update the agent network.

[0094]

[0095] The final loss function is shown in formula (10):

[0096] L(θ,φ)=c p L CLIP (θ)-c v L V (φ) (10)

[0097] Among them, c p is the strategy loss coefficient, c v is the loss coefficient of the value function, and the back-propagation algorithm is used to update θ and φ.

[0098] In order to solve the FJSP problem with DRL, it is necessary to model the problem as a Markov decision process and then establish a scheduling environment consisting of states, actions, and rewards.

[0099] (1) Description of the job shop scheduling environment status

[0100] The state consists of a process time matrix and a Boolean matrix of available machines, which can be used for feature extraction using a convolutional neural network (CNN). Additionally, there are three vector states: the completion percentage of each job, the total processing time for each workpiece, and the machine efficiency.

[0101] (2) Action definition

[0102] In FJSP, the operation is to select a suitable job at a decision point and dispatch it to a feasible machine. Therefore, an action consists of two parts: a job and a machine, with a size of |J| × |M|. To simplify the search space, a soft coding format is used, defining the action as a PDR. The information matrix is then used to decode the PDR and select the job and machine.

[0103] (3) Setting the reward function

[0104] In this embodiment, the objective of FJSP is to minimize the makespan. Therefore, the reward function should be designed to make the two consistent. After the agent performs the action, the environment changes from state s t Change to state s t+1 , give the agent a timely reward R(s t ,a t We don’t know the maximum completion time at this point, but we can use the maximum completion time on all machines in the previous state E(t) minus the current state E(t+1), as shown in Formula (11). This way we can evaluate the quality of the current operation.

[0105] R(s t ,a t )=E(t)-E(t+1) (11)

[0106] The above formula (11) shows that the maximum value of the immediate reward is a non-positive number, usually a negative number. This is very useful when training agents and can accelerate the convergence of the network. The adaptive hyperparameter optimizer operation flow chart is as follows Figure 5 Assume that a certain algorithm model to be optimized has n hyperparameters. Since the number and type of hyperparameters of different algorithms are different, Table 1 in this embodiment uses the hyperparameter table optimized by the PPO algorithm.

[0107] Table 1 Hyperparameters

[0108]

[0109]

[0110] The process of the hyperparameter optimizer based on deep reinforcement learning is as follows:

[0111] The agent reads the hyperparameter table required by the algorithm;

[0112] The agent sequence outputs the distribution of hyperparameters to be optimized;

[0113] Sampling a normal distribution of hyperparameters;

[0114] Run on the validation dataset to get the reward function;

[0115] Use reinforcement learning methods to calculate the loss function and update the internal parameters of the agent;

[0116] Output the obtained approximate optimal strategy and save the plot in the log.

[0117] The execution process of the hyperparameter optimizer in this embodiment is as follows Figure 6 As shown, the details are as follows:

[0118] Step 3.1: Initialize network weights and set hyperparameters;

[0119] Step 3.2: According to the probability distribution π of the old policy θ′ (a t |s t ) Update the sample a at time t t ; Among them, s t is the state variable at time t;

[0120] Step 3.3: Receive the partial reward function r at time t t Transmitted to the state variable s at the next moment t+1 ;

[0121] Step 3.4: Store the trajectory vector (s) of the agent at time t t , a t , r t ) and the probability calculation of the old strategy π θ′ (a t |s t );

[0122] Step 3.5: Determine whether the single dataset has been solved, that is, whether it meets the solution requirements; if so, proceed to step 3.6; if not, return to step 3.1: The dataset refers to the set of internal algorithm parameters that have been iterated through the neural network cycle;

[0123] Step 3.6: Calculate the total reward value through the overall reward function

[0124] Step 3.7: Calculate the advantage estimate function

[0125] Step 3.8: Calculate the full loss function L(θ,φ);

[0126] Step 3.9: Use the backpropagation algorithm to update the agent's internal parameters θ and φ;

[0127] Step 3.10: Determine whether the number of updates is less than or equal to the threshold K; if so, return to step 3.7; if not, use the backpropagation algorithm to update the agent's internal parameters θ and φ again;

[0128] Step 3.11: Determine whether the number of data sets is less than or equal to the total number of data sets M; if so, return to step 3.1 and continue optimizing the next data set; if not, end the optimization process.

[0129] In the algorithm solver part, a unified algorithm interface is designed to connect the algorithm library and the database. The interface realizes the interaction between the algorithm and the database through the function connect_mysql. The function input includes the host name hostname, username username, password pwd, and database name database_num. The interface function design is divided into two parts: data reading and returning scheduling results. The solver architecture is as follows Figure 7 shown.

[0130] Data reading: In Python, connect to the MYSQL server by calling the connect method in the pymysql module; execute the SQL statement to return the order information stored in the specified table in the database, and convert the obtained data into the data format required by the optimization algorithm.

[0131] Taking multi-objective optimization as an example, order information is stored in tables benchmark_1, benchmark_p, benchmark_t, and benchmark_ma in database db1. Hyperparameters used in the multi-objective optimization algorithm are stored in table parameter. The data stored in table benchmark_1 is shown in Table 2.

[0132] Table 2benchmark_1 table

[0133] id m n delay machine comp_time def_time equ_load ene_consu 1 5 3 550 6 1 1 0 1

[0134] Where m is the number of workpieces (int type); n is the number of processes (int type); machine is the total number of machines available for workpiece production (int type); delay is the delivery deadline (float type). The four parameters comp_time, def_time, def_time, and equ_time determine the number of objectives in the multi-objective algorithm. These parameters are bit type. Comp_time represents the maximum completion time, def_time represents the delay time, equ_time represents the total equipment load, and ene_consum represents the total energy consumption.

[0135] If the parameter is equal to 1, it means that the multi-objective algorithm considers the target, and if it is equal to 0, it means that the target is not considered. Figure 1 Taking the middle case as an example, it means that the multi-objective algorithm considers the maximum completion time, delay time and total energy consumption, but does not consider the total equipment load.

[0136] The data stored in the table benchmark_p is as follows Figure 8 As shown in the table benchmark_p , each row represents a workpiece. id is the workpiece number, and part is the number of machines that can process each step in the workpiece. The data type is char, and the machine numbers corresponding to each step are separated by spaces. For example, the first row indicates that workpiece 1 has three steps. Step 1 can be processed using three machines, step 2 can be processed using two machines, and step 3 can be processed using two machines.

[0137] The data stored in the table benchmark_ma is as follows Figure 9 As shown in the table, each row in the benchmark_ma table represents a workpiece, where id is the workpiece number, and ma is the machine number that can be used for each process of processing the workpiece. The data type is char, corresponding to table benchmark_p, and the rows are separated by spaces. For example, the first row indicates that the first process of workpiece 1 can be processed using machine 1, machine 2, or machine 3; the second process can be processed using machine 4 or machine 2; and the third process can be processed using machine 4 or machine 5.

[0138] The data stored in the table benchmark_T is as follows Figure 10As shown in the table benchmark_ma, each row represents a workpiece, where id is the workpiece number, and t is the time required for each process to process the workpiece using different machines. Each row is of type char and corresponds to the table benchmark_ma, separated by spaces. For example, the first row indicates that the time required to process the first process of workpiece 1 using machine 1, machine 2, or machine 3 is 147, 123, and 145, respectively. The time required to process the second process of workpiece 1 using machine 4 or machine 3 is 140 and 130, respectively. The time required to process the third process of workpiece 1 using machine 4 or machine 5 is 150 and 160, respectively.

[0139] Taking the multi-objective optimization algorithm NSGA3 as an example, the data stored in the parameter table is shown in Table 3.

[0140] Table 3 parameter table

[0141] id lter pop gen pop_f theta across mutation 1 1 200 250 100 5 0.8 0.5

[0142] Among them, gen is the maximum number of iterations, the data type is int; pop_f is the parent population size, the data type is int; theta is the crossover rate, the data type is float; mutation is the mutation rate, the data type is fl; Iter is the total number of executions of the multi-objective algorithm, the data type is int; pop is the population size, the data type is inoat.

[0143] Return scheduling results: After the optimization algorithm is executed, the algorithm function will return the corresponding scheduling results. Call the pymysql module again and use the SQL statement to write the scheduling results into the database. The steps are as follows:

[0144] First, create a new table named time in the database to store scheduling information; if the table time already exists, clear the data in the table time;

[0145] Insert the scheduling results into the time table through the insert command; the time table is shown in Table 4;

[0146] Table 4 time table

[0147]

[0148]

[0149] Here, "work" represents the workpiece number, "part" represents the process number for that workpiece, "machine" represents the machine number used to process that process, "start_time" represents the start time of that process, and "end_time" represents the end time of that process. For example, the first row of data in the table indicates that workpiece 1, process 1, was processed on machine 3, with a start time of 0 and an end time of 145.

[0150] After writing the scheduling results to the time table, disconnect from the MYSQL database.

[0151] This invention integrates various scheduling algorithms and designs a unique algorithm selection mechanism, addressing the inability of flexible production line scheduling systems to autonomously match algorithms. It also utilizes a hyperparameter optimization mechanism based on reinforcement learning to intelligently match internal algorithm parameters, ultimately outputting a scheduling strategy tailored to personalized orders as a production reference for manufacturing companies. This significantly reduces overall system energy consumption, improves manufacturing efficiency, shortens maximum completion time, and significantly reduces material and time waste. Furthermore, the use of the optimal algorithm results in high-quality products, a more streamlined production process, and effectively increases economic returns.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A method for selecting an efficient algorithm for a personalized customization production line, characterized by: The method first uses each user's key to log in to the server, enter the order page, and enter the product parameters and order requirements to be produced. The order information will be sent to the manufacturer's client, and the manufacturer will perform primary algorithm filtering based on the order information characteristics. The customer's order enters the algorithm selector part. According to customer needs, the order information is transmitted to the data stream through the interface designed in the algorithm pool. The constraint-based algorithm filter in the local server is filtered to filter out inappropriate algorithm clusters; then the hyperparameter optimizer is entered to optimize the internal parameters of the selected algorithm suitable for the characteristics of the order data information; finally, the algorithm solver part is entered, the order data stream is input into the algorithm with configured parameters for solution, and the output scheduling results are displayed on the front-end interface; The specific method of the algorithm selector part is: Step 2.1: Obtain order information, including the process string, product quantity, optimization target quantity, four optimization target options, and whether flexible processing is required. The optimization target quantity is 0 for a single target and 1 for multiple targets. The four optimization target options are strings of 1 1 1 1, where 0 indicates the target is considered and 1 indicates it is not. Flexible processing is determined based on the process string. The additional information that the algorithm needs to obtain includes: The processing power and fixed power of each machine, the fixed power of the workshop and the parts transmission power; The transfer time of the workpiece between different machines; Delivery period; Type of algorithm; Step 2.2: Divide the optimization objective into single objective and multi-objective categories. If it is a single objective, proceed to step 2.3; if it is a multi-objective, proceed to step 2.

5. Step 2.3: Determine whether the algorithm type is specified; if so, determine whether it is flexible processing, and then classify it into the corresponding flexible or non-flexible algorithm pool for optimization, and jump to step 2.6; if not, execute step 2.4; Step 2.4: Determine whether it is flexible processing, and then further classify it into four scales: small, medium, large, and extra-large. According to flexibility and scale, it is divided into corresponding algorithm pools for optimization, and then jump to step 2.6; Step 2.5: Determine whether it is flexible, then classify it into the corresponding flexible or non-flexible multi-objective algorithm pool for optimization, and go to step 2.6; Step 2.6: Select the best result from the corresponding algorithm pool and return it.

2. The method for selecting an efficient algorithm for a personalized customization production line according to claim 1, characterized in that: After the order information is extracted, the algorithm selector classifies the algorithm, the hyperparameter optimizer implements the internal hyperparameter configuration of the algorithm, and the algorithm solver solves the scheduling result, the corresponding log will be sent to the backend database for storage and recording.

3. The method for selecting an efficient algorithm for a personalized customization production line according to claim 1, characterized in that: The hyperparameter optimizer is trained offline through deep reinforcement learning so that the results can be directly returned when applied. The process of the hyperparameter optimizer based on deep reinforcement learning is as follows: The agent reads the hyperparameter table required by the algorithm; The agent sequence outputs the distribution of hyperparameters to be optimized; Sampling a normal distribution of hyperparameters; Run on the validation dataset to get the reward function; Use reinforcement learning methods to calculate the loss function and update the internal parameters of the agent; Output the obtained approximate optimal strategy and save the plot in the log.

4. The method for selecting an efficient algorithm for a personalized customization production line according to claim 3, characterized in that: The hyperparameter optimizer uses the proximal policy optimization algorithm, namely the PPO algorithm, to handle the hyperparameter adaptive matching problem in the production line scheduling process. The specific process is as follows: Step 3.1: Initialize network weights and set hyperparameters; Step 3.2: According to the probability distribution of the participant network from state to action output in the old strategy Update the sample at time t a t ; in, s t is the state variable at time t; Step 3.3: Receive the partial reward function at time t r t Transfer to the state variable of the next moment s t+1 ; Step 3.4: Store the trajectory vector of the agent at time t ( s t , a t , r t ) and the probability calculation of the old strategy ; Step 3.5: Determine whether the solution of a single data set has been completed; If completed, proceed to step 3.6; if not completed, return to step 3.1; Step 3.6: Calculate the total reward value through the overall reward function ; Step 3.7: Calculate the advantage estimate function ; Step 3.8: Calculate the full loss function ; Step 3.9: Update the agent's internal parameters using the backpropagation algorithm and ; Step 3.10: Determine whether the number of updates is less than or equal to the threshold K; if so, return to step 3.7; if not, use the back propagation algorithm to update the internal parameters of the agent again and ; Step 3.11: Determine whether the number of data sets is less than or equal to the total number of data sets M; if so, return to step 3.1 and continue optimizing the next data set; if not, end the optimization process.

5. The method for selecting an efficient algorithm for a personalized customization production line according to claim 1, characterized in that: In the algorithm solver part, a unified algorithm interface is designed to connect the algorithm library and the database. The interface realizes the interaction between the algorithm and the database through the connect_mysql function. The input of the function includes the host name hostname, username username, password pwd, and database name database_num. The interface function design is divided into two parts: data reading and returning scheduling results; Data reading: Connect to the MYSQL server in Python by calling the connect method in the pymysql module; Execute SQL statements to return order information stored in the specified table in the database, and convert the obtained data into the data format required by the optimization algorithm; Return scheduling results: After the optimization algorithm is executed, the algorithm function will return the corresponding scheduling results. Call the pymysql module again and use the SQL statement to write the scheduling results into the database. The steps are as follows: First, create a new table named time in the database to store scheduling information; if the table time already exists, clear the data in the table time; Insert the scheduling results into the time table using the insert command; After writing the scheduling results to the time table, disconnect from the MYSQL database.

Images