Reinforcement dual-learning driven evolutionary optimization method and system for aircraft assembly scheduling

Abstract

The present application relates to the technical field of assembly scheduling, and more particularly to a reinforced dual-learning driven evolutionary optimization method and system for aircraft assembly scheduling, wherein the method comprises: constructing an initial hierarchical encoding strategy and an initial decoding strategy in an evolutionary strategy based on the process sequence constraints and worker station capacity constraints of the target aircraft assembly; optimizing the statistical learning performance and evolutionary learning performance of the initial hierarchical encoding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy; generating a new evolutionary strategy based on the optimized hierarchical encoding strategy and decoding strategy and executing it to schedule the target aircraft assembly. The present application can convert the complex process constraints of aircraft assembly into hierarchical encoding and decode the hierarchical encoding using a queue data structure, thereby quickly and adaptively generating different aircraft assembly scheduling schemes while considering complex characteristics such as assembly process grouping, process serial and parallel process relationship coupling, and station capacity constraints.

Description

Technical Field

[0001] The present application relates to the technical field of assembly scheduling, and in particular to a reinforcement dual-learning driven evolutionary optimization method and system for aircraft assembly scheduling. Background Art

[0002] The assembly shop is a crucial production link in the design and manufacture of aviation products. However, an aircraft typically contains millions of parts, making the assembly process complex. Aircraft manufacturing demands extremely high quality, with a factory accident rate of less than one in a million. Therefore, assembly process route planning is crucial. However, frequent material shortages during the assembly process require engineers to frequently re-arrange process route plans.

[0003] In the prior art, rescheduling aircraft assembly process routing plans primarily relies on manual adjustments by process operators. Each workstation has 400 to 1,000 assembly steps, and manual rescheduling takes days. This forces production lines to wait for assembly process plans or independently assemble according to on-site conditions, increasing assembly time while failing to guarantee production capacity. Furthermore, due to the large number of steps at each workstation, process operators must integrate multiple groups of steps, reducing the number to a few dozen to quickly rescheduling. However, such solutions lack readability and interpretability, leading to some research using evolutionary algorithms to solve this problem.

[0004] However, the evolutionary algorithms in related technologies have difficulty constructing feasible solutions when faced with the complex sequential constraints of the working procedures in aircraft assembly process route planning. In addition, when faced with the capacity constraints of worker positions, the number and scale differences of working procedures in actual scenarios, the solution speed is prone to be slow and it is difficult to adaptively adjust its own strategies to solve different problems. This urgently needs to be solved. Summary of the Invention

[0005] The present application provides an evolutionary optimization method and system driven by reinforced dual learning for aircraft assembly scheduling, so as to address the problems that evolutionary algorithms in related technologies have difficulty in constructing feasible solutions when faced with the complex sequential constraints of the working procedures in aircraft assembly process route planning. In addition, when faced with the capacity constraints of worker positions, the number and scale differences of working procedures in actual scenarios, the solution speed is slow and it is difficult to adaptively adjust their own strategies to solve different problems.

[0006] The first aspect of the present application provides an evolutionary optimization method driven by enhanced dual learning for aircraft assembly scheduling, comprising the following steps: constructing an initial hierarchical encoding strategy and an initial decoding strategy in an evolutionary strategy based on the process sequence constraints of the target aircraft assembly and the capacity constraints of the worker stations; optimizing the statistical learning performance and evolutionary learning performance of the initial hierarchical encoding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy to obtain an optimized hierarchical encoding strategy and an optimized decoding strategy; generating a new evolutionary strategy based on the optimized hierarchical encoding strategy and the optimized decoding strategy to execute the new evolutionary strategy to schedule the target aircraft assembly.

[0007] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial layered encoding strategy and the initial decoding strategy through the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, it also includes: reading the process relationship of the target aircraft assembly scheduling, and layering the processes according to the process relationship to obtain different process layers; based on the different process layers, generating an initial sampling population using the initial layered encoding strategy, and arranging the initial sampling population according to the objective function to obtain an arranged initial sampling population, and selecting an elite solution based on the arranged initial sampling population; statistically analyzing the distribution information of the elite solution in the different process layers, and establishing a probability statistical matrix based on the distribution information and initializing the probability statistical matrix to obtain an initialized statistical matrix; constructing a new population based on the probability distribution information in the initialized statistical matrix and the initial sampling population, so as to update the probability distribution information of the initialized statistical matrix according to the elite solution in the new population, and obtain the updated probability distribution information and its corresponding updated statistical matrix, so as to determine the preset search strategy based on the updated statistical matrix.

[0008] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial layered encoding strategy and the initial decoding strategy through the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, it also includes: constructing an operator distribution matrix based on the different process layers and initializing the operator distribution matrix to obtain an initialized operator distribution matrix; through the initialized operator matrix, evolving the layered search performance of learning and reinforcing the feedback adjustment performance of learning to obtain the layered search strategy after evolutionary learning and the feedback adjustment strategy after reinforcement learning; combining the layered search strategy after evolutionary learning and the feedback adjustment strategy after reinforcement learning to analyze the optimal search operators of the different process layers to generate the preset multi-operator collaborative feedback search strategy.

[0009] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial layered coding strategy and the initial decoding strategy through the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, it also includes: defining the state set, action set, reward, and state transition of the initial layered coding strategy and the initial decoding strategy; based on the state set, the action set, the reward and the state transition, determining the parameter selection range of the preset search strategy to determine the preset parameter selection strategy.

[0010] Optionally, in one embodiment of the present application, a new evolutionary strategy is generated based on the optimized hierarchical coding strategy and the optimized decoding strategy to execute the new evolutionary strategy to schedule the target aircraft assembly, including: based on the different process layers, using the optimized hierarchical coding strategy to obtain a complete scheduling solution; based on the scheduling solution, using the optimized decoding strategy to calculate the time constraints of the target aircraft assembly to obtain a final scheduling plan for the target aircraft assembly, and scheduling the target aircraft assembly according to the final scheduling plan.

[0011] A second aspect of the present application provides a computer program system, including a computer program, characterized in that when the computer program is executed, it is used to implement the above-mentioned reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling.

[0012] The embodiments of the present application can design and optimize a specific hierarchical encoding and decoding strategy for the complex process sequence constraints and worker station capacity constraints in the aircraft assembly problem, thereby generating an assembly scheduling plan for the target aircraft using the optimized hierarchical encoding and decoding strategies. This achieves the goal of converting complex process constraints into hierarchical encoding using the hierarchical encoding strategy, and decoding the hierarchical encoding using the queue data structure in the decoding strategy to obtain the final scheduling plan. Furthermore, the statistical learning-based search strategy used in the optimization process can provide a high-quality initial solution for subsequent evolutionary learning. The reinforcement learning-based multi-operator collaborative feedback search strategy can analyze the optimal search operator at each layer, and the reinforcement learning-based parameter selection strategy can adjust the optimal parameters for problems of different scales, thereby modeling the production scenario of the aircraft assembly workshop. The aircraft assembly scheduling plan is rapidly generated while considering complex characteristics such as assembly process grouping, serial and parallel process relationship coupling, and station capacity constraints, and is adaptively adjusted based on the actual aircraft assembly scheduling problem. This solves the problem that evolutionary algorithms in related technologies find it difficult to construct feasible solutions when faced with the complex sequential constraints of the working procedures in aircraft assembly process route planning. In addition, when faced with the capacity constraints of worker positions, the number and scale differences of working procedures in actual scenarios, they are prone to slow solution speeds and find it difficult to adaptively adjust their own strategies to solve different problems.

[0013] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0015] Figure 1 This is a schematic diagram of process constraints and workstation constraints for an aircraft assembly scheduling problem according to one embodiment of the present application;

[0016] Figure 2 This is a flowchart of a reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling provided according to an embodiment of the present application;

[0017] Figure 3 This is a flowchart of an evolutionary optimization method driven by enhanced dual learning according to one embodiment of the present application;

[0018] Figure 4 This is a schematic diagram of a global search strategy framework based on statistical learning according to an embodiment of the present application;

[0019] Figure 5 This is a schematic diagram of using a crossover operator to exchange the assembly order of processes within each layer according to one embodiment of the present application;

[0020] Figure 6 A schematic diagram of a framework for feedback regulation based on reinforcement learning according to an embodiment of the present application

[0021] Figure 7 This is a schematic diagram of the modeling process of an embodiment of the present application;

[0022] Figure 8 This is a coding and decoding diagram of an embodiment of the present application; DETAILED DESCRIPTION

[0023] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0024] The following describes, with reference to the accompanying drawings, an evolutionary optimization method and system for aircraft assembly scheduling driven by a reinforcement dual learning approach according to an embodiment of the present application. The evolutionary algorithms in the related art mentioned in the background art above have difficulty constructing feasible solutions when faced with the complex sequence constraints of the processes in aircraft assembly process route planning. Furthermore, when faced with the capacity constraints of worker stations, the number of processes, and scale differences in actual scenarios, they are prone to slow solution speeds and difficulty adaptively adjusting their own strategies to solve different problems. The present application provides an evolutionary optimization method for aircraft assembly scheduling driven by a reinforcement dual learning approach. In this method, a certain hierarchical encoding strategy and decoding strategy can be designed and optimized for the complex process sequence constraints and worker station capacity constraints in the aircraft assembly problem, thereby generating an assembly scheduling plan for the target aircraft using the optimized hierarchical encoding strategy and decoding strategy. This approach achieves the goal of converting complex process constraints into layered codes using a layered encoding strategy, and decoding the layered codes using the queue data structure in the decoding strategy to obtain the final scheduling solution. Furthermore, the statistical learning-based search strategy employed in the optimization process provides high-quality initial solutions for subsequent evolutionary learning. A multi-operator collaborative feedback search strategy based on reinforcement learning analyzes the optimal search operator at each layer, and a parameter selection strategy based on reinforcement learning adjusts the optimal parameters for problems of varying scales. This modeling of the production scenario of an aircraft assembly plant allows for rapid generation of aircraft assembly scheduling solutions while taking into account complex characteristics such as assembly process grouping, serial and parallel process coupling, and workstation capacity constraints. Furthermore, this solution adaptively adjusts to the actual aircraft assembly scheduling problem. This approach addresses the challenges faced by prior evolutionary algorithms in the art, which often struggle to construct feasible solutions when faced with complex sequential constraints on processes in aircraft assembly process route planning. Furthermore, these algorithms often suffer from slow solution speeds and difficulty adaptively adjusting their strategies to solve diverse problems, particularly in the face of real-world constraints such as worker station capacity constraints, the number of processes, and differences in scale.

[0025] Before explaining the reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling in the embodiment of the present application, the aircraft assembly scheduling problem involved in the implementation of the present application is first explained.

[0026] The problem is described as follows:

[0027] An assembly outline has a total of N process groups for assembly, each group contains N i Each process O ij There is P ij The preceding process constraint is T. ij , assembly station is M k , the number of people required is W ij The entire assembly station can be divided into K small stations, and the number of people in each station is constrained to C kWhen the current number of people at a workstation does not reach the threshold, the subsequent processing steps need to compare whether the remaining positions are sufficient to accommodate O ij If there is no processing, you need to wait. If there is, you need to consider the constraints of the previous process and put it in the appropriate processing position. The optimization goal is to minimize the maximum completion time C max , which is calculated as:

[0028]

[0029] Among them, F ij Indicates O ij The completion time can be calculated. Figure 1 As shown, Figure 1 A schematic diagram of process constraints and workstation constraints for an aircraft assembly scheduling problem according to one embodiment of the present application.

[0030] Specifically, Figure 2 This is a flowchart of a reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling provided in an embodiment of the present application.

[0031] like Figure 2 As shown in FIG, the reinforcement dual learning driven evolutionary optimization method for aircraft assembly scheduling includes the following steps:

[0032] In step S201 , based on the process sequence constraints of the target aircraft assembly and the capacity constraints of the worker stations, an initial hierarchical encoding strategy and an initial decoding strategy in the evolutionary strategy are constructed.

[0033] It is understood that the target aircraft here refers to the aircraft that requires assembly scheduling. The initial hierarchical encoding strategy and initial decoding strategy in the evolutionary strategy here refer to the hierarchical encoding strategy and decoding strategy in the evolutionary strategy (evolutionary algorithm) constructed to solve the assembly scheduling problem of the target aircraft under process sequence constraints and worker station capacity constraints.

[0034] In some embodiments, the actual scenario of aircraft assembly scheduling involves complex process sequence relationships. During actual scheduling, a scheduling plan must be designed based on this process sequence relationship. Therefore, process sequence constraints, i.e., process sequence constraints, must be considered when scheduling aircraft assembly processes. Furthermore, each aircraft assembly station has certain worker station constraints. The total number of worker stations, how many stations are currently occupied, and how many workers can be accommodated all need to be considered when scheduling aircraft assembly.

[0035] Based on this, the present invention can design a hierarchical encoding strategy to address the complex process sequence constraints in the aircraft assembly problem, thereby converting the complex process sequence constraints into hierarchical codes. To address the capacity constraints of worker stations during assembly, the present invention can design a queue-based decoding strategy to utilize the queue data structure to decode the hierarchical codes. Combining the hierarchical encoding and decoding strategies can effectively solve the target aircraft assembly scheduling problem.

[0036] Step S202 , optimizing the statistical learning performance and evolutionary learning performance of the initial layered coding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy, to obtain an optimized layered coding strategy and an optimized decoding strategy.

[0037] During the actual execution process, taking into account the divergence problems, solution speed problems and some other adaptive problems that may exist in the initial layered coding strategy and the initial decoding strategy in the evolutionary strategy, the embodiment of the present application can also optimize the initial layered coding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy and a preset parameter selection strategy, thereby achieving optimization of the evolutionary strategy.

[0038] The preset search strategy here refers to a search strategy based on statistical learning that is pre-built to accelerate the convergence of the algorithm, which can provide high-quality initial solutions for subsequent evolutionary learning.

[0039] The preset multi-operator collaborative feedback search strategy here refers to a multi-operator collaborative feedback search strategy based on reinforcement learning that is pre-built to further accelerate the convergence of the algorithm, and can analyze the optimal search operator at each process layer.

[0040] The preset parameter selection strategy here refers to a reinforcement learning-based parameter selection strategy that is pre-built to enable the statistical learning-based search strategy to intelligently adjust the optimal parameters for problems of different scales.

[0041] Through a certain statistical learning-based search strategy, a reinforcement learning-based multi-operator collaborative feedback search strategy, and a reinforcement learning-based parameter selection strategy, the initial hierarchical encoding strategy and the initial decoding strategy can be optimized, and a reinforcement dual-learning-driven evolutionary optimization method for aircraft assembly scheduling can be obtained, which can effectively solve the aircraft assembly scheduling problem.

[0042] Figure 3 This is a flow chart of an evolutionary optimization method driven by enhanced dual learning according to an embodiment of the present application. Figure 3 As shown:

[0043] Step 1: Read the process relationship and divide the processes into layers;

[0044] Step 2: Use the layered coding method to generate an initial sampling population P1 of size ps, select the elite solutions of population P1 as statistical data, and then establish a probability statistical matrix for different layers based on the distribution of these solutions at different process layers, and initialize the probability statistical matrix;

[0045] Step 3: decode population P1 and calculate the objective function value;

[0046] Step 4: Count the process order distribution of the top λ% elite individuals in each layer;

[0047] Step 5, generate the offspring population C1 according to the probability distribution sampling in the probability statistics matrix, and evaluate the offspring objective function;

[0048] Step 6: Merge P1 and C1, sort them in descending order according to the objective function, and retain the first ps solutions as the new population P1';

[0049] Step 7: Update the probability distribution in the probability statistics matrix. If stage 1 is not completed, return to step 5; otherwise, go to step 8.

[0050] Step 8, initialize the operator distribution statistics matrix;

[0051] Step 9, sample the crossover operator for each solution in the population;

[0052] Step 10, using tournament selection to generate the mating pool population M;

[0053] Step 11: Each individual in the population randomly performs the selected crossover operator with an elite in the mating pool to produce offspring;

[0054] Step 12: Evaluate the offspring objective function to determine whether the parent can be updated, and count the success and failure times of the current operator.

[0055] Step 13: Update the operator distribution statistics matrix based on the success and failure statistics, merge the subpopulation and the parent population, and retain the first ps solutions as the new population P1';

[0056] Step 14: If stage 2 is not completed, return to step 9; otherwise, the algorithm ends and outputs the global optimal solution.

[0057] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial hierarchical encoding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy, it also includes: reading the process relationship of the target aircraft assembly scheduling, and hierarchizing the processes according to the process relationship to obtain different process layers; based on different process layers, generating an initial sampling population using the initial hierarchical encoding strategy, and arranging the initial sampling population according to the objective function to obtain an arranged initial sampling population, and selecting an elite solution based on the arranged initial sampling population; statistically calculating the distribution information of the elite solutions at different process layers, and establishing a probability statistical matrix based on the distribution information and initializing the probability statistical matrix to obtain an initialized statistical matrix; constructing a new population based on the probability distribution information in the initialized statistical matrix and the initial sampling population, so as to update the probability distribution information of the initialized statistical matrix according to the elite solution in the new population, and obtain the updated probability distribution information and its corresponding updated statistical matrix, so as to determine the preset search strategy based on the updated statistical matrix.

[0058] In some embodiments, in order to provide high-quality initial solutions for the evolutionary learning phase of the reinforced dual-learning driven evolutionary optimization method for aircraft assembly scheduling in this application and enhance the convergence of the population, this application can use certain search strategies to locate the position of high-quality initial solutions in the solution space.

[0059] Figure 4 This is a schematic diagram of a global search strategy framework based on statistical learning according to an embodiment of the present application. Figure 4 As shown, the global search strategy based on statistical learning in the embodiment of the present application can be generally divided into, but not limited to, three parts: first, an initial sampling population is constructed based on different process layers, and a sample set is constructed using the elite solutions in the initial sampling population; then, the process distribution of each group in the sample is statistically analyzed to construct a probability statistical matrix; finally, based on the probability statistical matrix, new solutions are sampled and generated. The specific steps can be expressed as follows:

[0060] Sample collection: Based on different process layers, a certain hierarchical coding strategy is used to generate the initial sampling population, and all solutions in the population are arranged in ascending order according to the objective function. The sample set is constructed based on the top λ% individuals in the arranged initial sampling population as elite solutions.

[0061] Constructing statistical matrix: Statistical elite solves the distribution information of different process layers, and builds probability statistical matrix based on the distribution information and initializes the probability statistical matrix to obtain the initialized statistical matrix. Among them, the element P in the initialized statistical matrix ij represents the probability that process i is at position j in the current set. Under the initial conditions, P ij (1) L, indicating that all positions have the same probability of appearing.

[0062] Statistical matrix update: After the second iteration, the probability distribution information in the initialized statistical matrix is ​​statistically analyzed, and a new population is constructed in combination with the initial sampled population to update the probability distribution information of the initialized statistical matrix according to the position of the elite solution in the new population. The initialized statistical matrix is ​​then updated according to the updated probability distribution information. The specific process can be expressed in the form of a formula, but is not limited to, as follows:

[0063]

[0064] If process i appears at position j in sample k, then is 1.

[0065] Sampling solution generation: Scan the updated statistical matrix by column, select the process of each column, sample the roulette selection strategy, normalize the probability of the jth column, and generate a random number x. When x is greater than the probability P i-1,j , less than P ij , then process i is selected. After selecting process i, all probabilities in row i of the matrix are set to 0 to prevent process i from being selected repeatedly. Repeat the above steps until all processes are selected, thus completing the design of a global search strategy based on statistical learning.

[0066] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial hierarchical encoding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy, it also includes: constructing an operator distribution matrix and initializing the operator distribution matrix based on different process layers to obtain an initialized operator distribution matrix; by initializing the operator matrix, evolving the hierarchical search performance of learning and reinforcing the feedback adjustment performance of learning, to obtain the hierarchical search strategy after evolutionary learning and the feedback adjustment strategy after reinforcement learning; combining the hierarchical search strategy after evolutionary learning and the feedback adjustment strategy after reinforcement learning to analyze the optimal search operators of different process layers to generate a preset multi-operator collaborative feedback search strategy.

[0067] In other embodiments, the global search strategy based on statistical learning in the first stage can quickly search the solution space, but due to the limited number of samples, the final probability distribution will converge to a steady-state distribution, and the algorithm may lose its search capability at this time. Therefore, the embodiment of the present application can design a certain multi-operator collaborative feedback search strategy based on reinforcement learning in the second stage. For example, based on different process layers, an operator distribution matrix is ​​constructed and initialized, and the hierarchical search performance of the initialized operator matrix is ​​evolved and the feedback adjustment performance of the reinforcement learning is enhanced, so as to analyze the optimal search operators of different process layers, obtain a preset multi-operator collaborative feedback search strategy, and realize the collaborative evolution of reinforcement learning. Among them, the multi-operator collaborative feedback search strategy includes but is not limited to two parts: hierarchical search based on evolutionary learning and feedback adjustment based on reinforcement learning.

[0068] Hierarchical search based on evolutionary learning (multi-operator collaborative hierarchical evolution): First, a two-player tournament algorithm is used to select a mating pool from the population. Secondly, each individual in the population randomly selects a parent from the mating pool. Next, considering that the hierarchical coding is used to represent the assembly scheme in the embodiment of the present application, the crossover operator can be executed independently between each layer. Specifically, the crossover operator can be, but is not limited to, randomly selecting a crossover method from the process-based crossover (Precedence Operation Crossover, POX), partial matching crossover (Partial Matched Crossover, PMX), and order crossover (Order Crossover, OX) to exchange the assembly order of the processes in each layer, Figure 5 This is a schematic diagram of using a cross operator to exchange the assembly order of processes within each layer in one embodiment of the present application, as shown in FIG. Figure 5 As shown, the detailed process of exchanging different crossover operators is demonstrated.

[0069] Feedback adjustment based on reinforcement learning: Since the optimal crossover operators for solving different problems at different layers are different, in order to enable the algorithm to have adaptive adjustment capabilities, the embodiments of the present application can, but are not limited to, adopt a feedback strategy based on reinforcement learning to enhance the selection probability of the dominant crossover operator to enhance the convergence of the algorithm. Figure 6 This is a schematic diagram of a framework for feedback regulation based on reinforcement learning according to an embodiment of the present application. Figure 6As shown, the specific process can be, but is not limited to, expressed as follows: first, the data set is empty in the initial stage, so the embodiment of the present application can allow the algorithm to randomly select a crossover operator, and after iterating h generations, count the number of successes and failures of each operator to form an initial data set; then, the number of successes and failures of each generation of operators is added to the data set. In order to ensure that the algorithm is not affected by useless search data in the early stage, when the number of data in the data set exceeds the threshold E, an equal amount of data is deleted starting from the first row; then, the selection probability of each operator in the next generation is calculated based on the data set, and the calculation formula can be, but is not limited to: p = number of successes / (number of successes + number of failures); then, all selection probabilities are normalized as the selection probability of the next generation evolution operator; finally, if the algorithm has iterated h generations, then starting from generation h+1, according to the selection probability of the operator, a roulette strategy is sampled to select a crossover operator for each solution in the population, and the same crossover operator is used for all layers in the solution. The above steps are repeated until the end of the algorithm.

[0070] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial layered coding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy, it also includes: defining the state set, action set, reward, and state transition of the initial layered coding strategy and the initial decoding strategy; based on the state set, action set, reward, and state transition, determining the parameter selection range of the preset search strategy to determine the preset parameter selection strategy.

[0071] In certain embodiments, the number of samples used in the first stage of statistical learning can affect model learning efficiency. For example, too few samples can lead to rapid distribution convergence, resulting in poor initial population positioning. Too many samples can cause the population to spend too much time evaluating the quality of the sampled solutions, thus reducing learning efficiency. Therefore, the present application can design a specific parameter selection strategy based on reinforcement learning.

[0072] Specifically, Figure 7 This is a schematic diagram of the modeling process of an embodiment of the present application. The embodiment of the present application can be, but is not limited to, limiting the parameter selection range such as P = [10, 15, 20, 30, 40, 60], and modeling the algorithm parameter selection as follows Figure 7 The Markov decision process shown in Figure 1 is a simple one. In this process, the application first needs to define the state set, action set, reward, and state transition in the algorithm.

[0073] State Set: To accurately describe state transitions, the algorithm can be divided into six states using the current parameter selection as the state. If the number of samples currently selected is 10, then state 1 is used, and states for other values ​​are 2-6.

[0074] Action Set: When the algorithm detects that the probability distribution matrix of the statistical learning model in the current state has converged and the covariance with the previous matrix is ​​greater than 0.99, the algorithm is considered to have converged and other parameters need to be reselected. Therefore, the action set is also a parameter set. For example, if the current state is P = 10, and the parameter is changed to P = 30, the current selected action is 4.

[0075] Reward: After selecting the current action and undergoing another round of statistical learning, the probability distribution of the statistical matrix is ​​recalculated based on the new sample set, and the similarity with the previous generation matrix is ​​calculated. If the similarity is less than 0.99, that is, the current parameters have updated the distribution of the population, broadened the sampling range of the model, can generate new solutions, and enhanced the diversity of the initial population, then Rt = 10, otherwise Rt = 0.

[0076] State transition: After the current action receives a reward, due to the selection of other parameters, the next state St+1 becomes the selected action At.

[0077] Model update: When a state transition occurs, the model collects a set of strategies, namely (St, At, Rt, St+1), and the Q table update formula can be expressed as: Q(St, At) = Q(St, At) + α*(γ*max(Q(St+1, At))-Q(St, At)).

[0078] In summary, the specific steps of the parameter selection strategy based on reinforcement learning in the embodiment of the present application can be expressed as follows:

[0079] Step 1: Initialize parameter selection range;

[0080] Step 2: Initialize the Q table, Q(6,6);

[0081] Step 3: Initialize the current state St=2, At=2;

[0082] Step 4: Execute the statistical learning module;

[0083] Step 5: Calculate the statistical matrix covariance value C. If C > 0.99, reselect the action. If the random number β > 0.95, randomly select an action. Otherwise, select the action with the largest Q value. If C < 0.99, give a reward Rt = 10, maintain the current state, and update the Q table.

[0084] Step 6: If the action changes, that is, the number of samples P changes, if P becomes larger, the random initialization strategy is used to replenish the remaining samples; if P becomes smaller, the samples are arranged in ascending order according to the objective function and the redundant samples are deleted;

[0085] Step 7: If the end condition is met, the statistical learning phase ends, otherwise, execute step 4.

[0086] Step S203 : generating a new evolutionary strategy according to the optimized layered encoding strategy and the optimized decoding strategy, and executing the new evolutionary strategy to schedule the assembly of the target aircraft.

[0087] After optimizing the initial hierarchical coding strategy and the initial decoding strategy through certain search strategies, multi-operator collaborative feedback search strategies, and parameter selection strategies, the embodiment of the present application realizes the optimization of the evolutionary strategy, and can execute a new evolutionary strategy, and use the optimized hierarchical coding strategy and decoding strategy in the new evolutionary strategy to generate the final scheduling plan for the target aircraft assembly scheduling problem, thereby scheduling the assembly of the target aircraft.

[0088] Optionally, in one embodiment of the present application, a new evolutionary strategy is generated based on the optimized hierarchical coding strategy and the optimized decoding strategy to execute the new evolutionary strategy to schedule the target aircraft assembly, including: based on different process layers, using the optimized hierarchical coding strategy to obtain a complete scheduling solution; based on the scheduling solution, using the optimized decoding strategy to calculate the time constraints of the target aircraft assembly to obtain the final scheduling plan for the target aircraft assembly, and scheduling the target aircraft assembly according to the final scheduling plan.

[0089] In the actual execution process, the final scheduling plan for the target aircraft assembly is generated according to the optimized hierarchical coding strategy and the optimized decoding strategy. The main step is to hierarchically divide the processes in the target aircraft assembly scheduling into different process layers. Based on the different process layers, the hierarchical coding method is used to generate the initial population to obtain a complete scheduling solution. Finally, the queue decoding method is used to calculate the objective function value of the target aircraft assembly time constraint to obtain the final scheduling plan for the target aircraft.

[0090] Among them, the process stratification is mainly based on the sequential relationship of the process groups, indexing the child nodes of each process group, adding one to the number of layers of the child nodes, and then recursively obtaining the stratification of all process groups, where the number of initialization layers within each process group is equal to the number of layers of the process group; then, the number of layers of the processes in each group is counted, and the number of layers is also calculated recursively; finally, the layer index of all processes is obtained, and the processes at the same layer are summarized into a set to ensure the sequential relationship between each layer and the parallel relationship between the same layers, thereby ensuring the feasibility of the solution.

[0091] Layered encoding: Based on the set of processes at each layer, different permutation sequences are randomly generated, and the fragments of each group are spliced ​​together to form a complete scheduling solution.

[0092] Queue decoding: First, establish a process queue Q for each workstation k ; Then, sequentially take out the assembly process Q from the scheduling solution ij ; According to the P of the previous process ijCompletion time calculation process Q ij The constrained start time S ij =max(F(P ij )); Then, scan the selected station M k The process queue, find the start time less than or equal to S ij , completion time is greater than S ij The process set H is sorted in ascending order according to the completion time, and the process Q′ in H is removed in sequence ij , determine whether the remaining capacity can accommodate process Q ij :If it can accommodate, then the start time is the constrained start time S ij Otherwise, S ij =max(S ij ,F′ ij ); Then, calculate the process Q ij Completion time F ij =S ij +T ij Finally, the completion time of all processes is calculated, and the maximum completion time C is obtained. max .

[0093] Thus, the decoding calculation of the target aircraft assembly time constraint is completed, and the final scheduling plan is generated according to the completion time and maximum completion time of all processes.

[0094] Figure 8 This is a coding and decoding diagram of an embodiment of the present application. Figure 8 As shown, the embodiment of the present application can convert two layers of complex process constraints into layered codes, use the queue data structure to decode the layered codes, and obtain the final scheduling solution.

[0095] Additionally, the embodiment of the present application also randomly generated 20 test problems based on actual production data for testing to verify the generalization and convergence of the enhanced dual-learning driven evolutionary optimization method for aircraft assembly scheduling in the embodiment of the present application on aircraft assembly scheduling problems of different scales.

[0096] The parameters used in the test are as follows: number of process groups N = {20, 25, 30, 35}, number of processes within a group Ni = {20, 25, 30, 35, 40}, workstations are divided into six categories based on aircraft fill-in, the workstation number threshold is 10, processing time is randomly selected from U to [1, 10], the required number of workers is U to [2, 4], and workstations in the process group are randomly selected from the six workstations. All strategies and corresponding algorithms included in the method were independently run 20 times on 20 test problems to verify the generalizability of the proposed method using statistical results. Furthermore, the proposed method was validated using test problems from three actual aircraft manufacturing workstations.

[0097] The algorithm in the embodiment of the present application is programmed in Matlab, the simulation environment is Intel (R) core (TM) i9-13900k CPU @ 3.4GHz / 16GB RAM, GPU NVIDIA4070 12GB, and the operating system is Windows 10. The termination criterion of the algorithm is that the algorithm stops when the maximum running time MaxT reaches 0.025*N*Ni. In the experiment, the relative percentage deviation (RPD) is used to measure the distance between the algorithm and the lower bound. The lower bound is the minimum objective function value found by all algorithms. The RPD calculation formula is RPD = Cmax / (Cmax-LB).

[0098] To verify the effectiveness of the REL (Reinforcement Learning-Based Co-evolution) module, five variant algorithms were designed in the experiment: EL_POX, EL_PMX, and EL_OX. These all represent evolutionary learning using various crossover operators alone. REL represents the reinforcement learning-based co-evolutionary algorithm. RSL represents RSLEA (Reinforcement Dual Learning-Driven Evolutionary Optimization for Aircraft Assembly Scheduling) with the second stage removed. REL represents RSLEA with the first stage removed. Tables 1 and 2 show the statistical comparison of the best and average RPD values ​​of REL and its variant algorithms on different test problems, which can be expressed as follows:

[0099] Table 1

[0100]

[0101]

[0102] Table 2

[0103]

[0104]

[0105] As shown in Tables 1 and 2, it can be seen that REL in the embodiment of the present application achieves better optimal and average indicators than EL_OX, EL_PMX, and EL_POX, verifying the effectiveness of the reinforcement learning operator selection module. Furthermore, by comparing the results of RSLEA with those of RSL and REL, it can be seen that after removing any of the different stages of RSLEA, the algorithm performance decreases significantly, thus verifying the effectiveness of the REL module in the embodiment of the present application.

[0106] Furthermore, to accurately verify the effectiveness of the RSL module, the present invention also designed seven variant algorithms in the experiment: SLREL with six different sampling parameters, and REL without SL (supervised learning) assisted evolution. Tables 3 and 4 are statistical tables comparing the best RPD values ​​and average RPD values ​​of RSL and variant algorithms on different test problems, which can be expressed as follows:

[0107] Table 3

[0108]

[0109]

[0110] Table 4

[0111]

[0112] As shown in Tables 3 and 4, the average RPD of REL decreases when compared to SLRELs with different parameters, verifying that selecting different sampling ranges for SL improves REL's convergence and confirming the effectiveness of the SL module. Comparing RSLEA with various SLRELs shows superior performance in both optimal and average RPD, demonstrating that the RSL module can comprehensively improve REL's performance and verifying that the reinforcement learning parameter adaptation module can enhance the algorithm's adaptability and intelligence in parameter selection while consuming minimal computational resources.

[0113] Table 5 is a ranking table of Friedeman rank sum test results of RSLEA and other variant algorithms, which can be expressed as follows:

[0114] Table 5

[0115]

[0116] As shown in Table 5, Table 5 shows the rank sum test results of all variant algorithms. RSLEA achieved the highest ranking compared with other variant algorithms, indicating that the algorithm performance will decrease if any improved module is deleted, verifying the effect of the proposed modules and corresponding algorithms on improving the convergence of RSLEA in the embodiment of the present application.

[0117] To further verify the advantages of the RSLEA method in the examples of this application, the examples of this application also selected three latest algorithms, namely HGA, QIGA and EGHA. In order to compare the generalization of the algorithms, all methods were independently run 20 times with the same time on 20 test problems. Table 6 is a statistical table of the RPD index results of RSLEA and the comparison algorithms on the test set, which can be expressed as follows:

[0118] Table 6

[0119]

[0120]

[0121] As shown in Table 6, it can be seen that the RSLEA method in the embodiment of the present application achieved the best RPD index and RPD mean index on all 20 test problems. It shows the significance of the statistical algorithm's 20 runs on each test problem, where '-' indicates that the algorithm is significantly worse than RSLEA. The results show that all comparison algorithms are significantly worse than RSLEA on all 20 test problems, which verifies that the RSLEA method has good effectiveness and generalization on test problems of different scales.

[0122] In addition, this application example also selected three real-world cases to demonstrate the excellent application capabilities of the RSLEA method in actual aircraft assembly scheduling scenarios. These three cases are the assembly outlines of the aircraft wing docking station, the fuselage hole drilling station, and the horizontal tail assembly station. Each algorithm was independently run 30 times, with a runtime of 10 seconds. Tables 7 and 8 are statistical tables of the application results of the RPD indicators of RSLEA and the comparison algorithms in actual aircraft assembly scheduling problems, which can be expressed as follows:

[0123] Table 7

[0124]

[0125] Table 8

[0126]

[0127] As shown in Tables 7 and 8, the results show that RSLEA achieved the best optimal RPD and average RPD indicators, further verifying the effectiveness of the enhanced dual-learning driven evolutionary optimization method for aircraft assembly scheduling in the embodiment of the present application.

[0128] According to the reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling proposed in the embodiments of the present application, a specific hierarchical encoding strategy and decoding strategy can be designed and optimized for the complex process sequence constraints and worker station capacity constraints in the aircraft assembly problem, thereby generating an assembly scheduling plan for the target aircraft using the optimized hierarchical encoding strategy and decoding strategy. This method utilizes the hierarchical encoding strategy to convert complex process constraints into hierarchical codes, and decodes the hierarchical codes using the queue data structure in the decoding strategy to obtain the final scheduling plan. Furthermore, the statistical learning-based search strategy utilized in the optimization process can provide a high-quality initial solution for subsequent evolutionary learning. The reinforcement learning-based multi-operator collaborative feedback search strategy can analyze the optimal search operator at each layer, and the reinforcement learning-based parameter selection strategy can adjust the optimal parameters for problems of different scales. This allows for modeling the production scenario of an aircraft assembly workshop, rapidly generating an aircraft assembly scheduling plan while considering complex characteristics such as assembly process grouping, serial and parallel process coupling, and station capacity constraints, and adaptively adjusting the plan based on the actual aircraft assembly scheduling problem. This solves the problem that evolutionary algorithms in related technologies find it difficult to construct feasible solutions when faced with the complex sequential constraints of the working procedures in aircraft assembly process route planning. In addition, when faced with the capacity constraints of worker positions, the number and scale differences of working procedures in actual scenarios, they are prone to slow solution speeds and find it difficult to adaptively adjust their own strategies to solve different problems.

[0129] An embodiment of the present application also provides a computer program system, including a computer program, which can run computer instructions. When the computer instructions are executed by a processor, the reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling provided in the embodiment of the present application is implemented.

[0130] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0131] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0132] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

Claims

1. A reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling, characterized by: The following steps are involved: Based on the process sequence constraints of the target aircraft assembly and the capacity constraints of the worker stations, the initial hierarchical encoding strategy and initial decoding strategy in the evolutionary strategy are constructed; Optimizing the statistical learning performance and evolutionary learning performance of the initial layered coding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy to obtain an optimized layered coding strategy and an optimized decoding strategy; generating a new evolutionary strategy according to the optimized layered encoding strategy and the optimized decoding strategy, and executing the new evolutionary strategy to schedule the assembly of the target aircraft; Wherein, before optimizing the statistical learning performance and evolutionary learning performance of the initial hierarchical coding strategy and the initial decoding strategy through the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, it also includes: reading the process relationship of the target aircraft assembly scheduling, and hierarchizing the processes according to the process relationship to obtain different process layers; based on the different process layers, generating an initial sampling population using the initial hierarchical coding strategy, and arranging the initial sampling population according to the objective function to obtain an arranged initial sampling population, and selecting an elite solution based on the arranged initial sampling population; statistically analyzing the distribution information of the elite solution in the different process layers, and establishing a probability statistical matrix based on the distribution information and initializing the probability statistical matrix to obtain an initialized statistical matrix; constructing a new population based on the probability distribution information in the initialized statistical matrix and the initial sampling population, so as to update the probability distribution information of the initialized statistical matrix according to the elite solution in the new population, obtain updated probability distribution information and its corresponding updated statistical matrix, and determine the preset search strategy based on the updated statistical matrix; Among them, before optimizing the statistical learning performance and evolutionary learning performance of the initial layered encoding strategy and the initial decoding strategy through the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, it also includes: constructing an operator distribution matrix based on the different process layers and initializing the operator distribution matrix to obtain an initialized operator distribution matrix; through the initialized operator distribution matrix, evolving the layered search performance of learning and reinforcing the feedback adjustment performance of learning, to obtain the layered search strategy after evolutionary learning and the feedback adjustment strategy after reinforcement learning; combining the layered search strategy after evolutionary learning and the feedback adjustment strategy after reinforcement learning to analyze the optimal search operators of the different process layers to generate the preset multi-operator collaborative feedback search strategy; Among them, before optimizing the statistical learning performance and evolutionary learning performance of the initial layered coding strategy and the initial decoding strategy through the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, it also includes: defining the state set, action set, reward, and state transition of the initial layered coding strategy and the initial decoding strategy; based on the state set, the action set, the reward and the state transition, determining the parameter selection range of the preset search strategy to determine the preset parameter selection strategy.

2. The method according to claim 1, characterized in that Generating a new evolutionary strategy according to the optimized layered encoding strategy and the optimized decoding strategy, and executing the new evolutionary strategy to schedule the target aircraft assembly, includes: Based on the different process layers, the optimized layered coding strategy is used to obtain a complete scheduling solution; Based on the scheduling solution, the time constraint of the target aircraft assembly is calculated using the optimized decoding strategy to obtain a final scheduling solution for the target aircraft assembly, and the target aircraft assembly is scheduled according to the final scheduling solution.

3. A computer program system comprising a computer program, characterized in that: When the computer program is executed, it is used to implement the reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling as described in any one of claims 1-2.

Images