Enhanced dual-learning-driven evolutionary optimization method and system for aircraft assembly scheduling

Abstract

The invention relates to the technical field of assembly scheduling, in particular to an aircraft assembly scheduling enhanced dual-learning-driven evolutionary optimization method and system, and the method comprises the steps: constructing an initial hierarchical coding strategy and an initial decoding strategy in an evolutionary strategy based on the process sequence constraint and the worker station capacity constraint of target aircraft assembly; optimizing statistical learning performance and evolutionary learning performance of the initial hierarchical coding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy and a preset parameter selection strategy; and generating a new evolutionary strategy according to the optimized hierarchical coding strategy and decoding strategy, and executing the new evolutionary strategy to schedule the target aircraft assembly. According to the method, the complex process constraint of aircraft assembly can be converted into layered coding, the layered coding is decoded by using the queue data structure, and different aircraft assembly scheduling schemes are quickly and adaptively generated under the condition of considering the complex characteristics of assembly process grouping, process serial and parallel process relation coupling, station capacity constraint and the like.

Description

Technical Field

[0001] The present application relates to the technical field of assembly scheduling, and in particular to an evolutionary optimization method and system driven by enhanced dual learning for aircraft assembly scheduling. Background Art

[0002] The assembly workshop is an important production link in the design and manufacture of aviation products. However, an aircraft generally contains millions of parts, and the assembly process is complicated. However, aircraft manufacturing has extremely high quality requirements, requiring the factory accident rate to be one in a million. Therefore, assembly process route planning is extremely important. However, material shortages occur frequently during the assembly process, causing technicians to frequently rearrange process route planning plans.

[0003] In related technologies, the assembly process route planning scheme for rearranging aircraft mainly relies on manual adjustments by process operators. Each workstation has 400 to 1,000 assembly procedures, and manual rearrangement takes several days, which means that the production line can only wait for the assembly process plan or assemble independently according to the on-site conditions, increasing the assembly time while failing to guarantee production capacity. In addition, due to the large number of workstation procedures, process operators need to integrate multiple groups of procedures and reduce the number of groups to dozens in order to quickly rearrange them, but such a scheme lacks readability and explainability. As a result, some studies have used evolutionary algorithms to solve such problems.

[0004] However, the evolutionary algorithms in related technologies find it difficult to construct feasible solutions when faced with the complex sequential constraints of the working procedures in aircraft assembly process route planning. In addition, when faced with the capacity constraints of worker positions, the number and scale differences of working procedures in actual scenarios, the solution speed is prone to be slow and it is difficult to adaptively adjust its own strategies to solve different problems, which needs to be solved urgently. Summary of the invention

[0005] The present application provides an evolutionary optimization method and system driven by reinforced dual learning for aircraft assembly scheduling, in order to solve the problems that evolutionary algorithms in related technologies find it difficult to construct feasible solutions when faced with complex sequential constraints on working procedures in aircraft assembly process route planning, and that they are prone to slow solution speeds and difficulty in adaptively adjusting their own strategies to solve different problems when faced with capacity constraints on worker positions, number of working procedures, and scale differences in actual scenarios.

[0006] The first aspect of the present application provides an evolutionary optimization method driven by enhanced dual learning for aircraft assembly scheduling, comprising the following steps: constructing an initial hierarchical coding strategy and an initial decoding strategy in an evolutionary strategy based on process sequence constraints of target aircraft assembly and capacity constraints of worker positions; optimizing the statistical learning performance and evolutionary learning performance of the initial hierarchical coding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy to obtain an optimized hierarchical coding strategy and an optimized decoding strategy; generating a new evolutionary strategy based on the optimized hierarchical coding strategy and the optimized decoding strategy to execute the new evolutionary strategy to schedule the target aircraft assembly.

[0007] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial hierarchical encoding strategy and the initial decoding strategy through the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, it also includes: reading the process relationship of the target aircraft assembly scheduling, and stratifying the processes according to the process relationship to obtain different process layers; based on the different process layers, generating an initial sampling population using the initial hierarchical encoding strategy, and arranging the initial sampling population according to the objective function to obtain the arranged initial sampling population, and selecting an elite solution based on the arranged initial sampling population; statistically analyzing the distribution information of the elite solution at the different process layers, and establishing a probability statistical matrix based on the distribution information and initializing the probability statistical matrix to obtain an initialized statistical matrix; constructing a new population based on the probability distribution information in the initialized statistical matrix and the initial sampling population, so as to update the probability distribution information of the initialized statistical matrix according to the elite solution in the new population, and obtain the updated probability distribution information and its corresponding updated statistical matrix, so as to determine the preset search strategy according to the updated statistical matrix.

[0008] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial layered encoding strategy and the initial decoding strategy through the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, it also includes: based on the different process layers, constructing an operator distribution matrix and initializing the operator distribution matrix to obtain an initialized operator distribution matrix; through the initialized operator matrix, evolving the layered search performance of evolutionary learning and reinforcing the feedback adjustment performance of learning to obtain the layered search strategy after evolutionary learning and the feedback adjustment strategy after reinforcement learning; analyzing the optimal search operators of the different process layers in combination with the layered search strategy after evolutionary learning and the feedback adjustment strategy after reinforcement learning to generate the preset multi-operator collaborative feedback search strategy.

[0009] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial layered coding strategy and the initial decoding strategy through the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, it also includes: defining the state set, action set, reward, and state transition of the initial layered coding strategy and the initial decoding strategy; based on the state set, the action set, the reward, and the state transition, determining the parameter selection range of the preset search strategy to determine the preset parameter selection strategy.

[0010] Optionally, in one embodiment of the present application, a new evolutionary strategy is generated based on the optimized hierarchical coding strategy and the optimized decoding strategy to execute the new evolutionary strategy to schedule the target aircraft assembly, including: based on the different process layers, using the optimized hierarchical coding strategy to obtain a complete scheduling solution; based on the scheduling solution, using the optimized decoding strategy to calculate the time constraints of the target aircraft assembly to obtain a final scheduling plan for the target aircraft assembly, so as to schedule the target aircraft assembly according to the final scheduling plan.

[0011] A second aspect of the present application provides a computer program system, including a computer program, characterized in that when the computer program is executed, it is used to implement the above-mentioned reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling.

[0012] The embodiment of the present application can design and optimize a certain hierarchical coding strategy and decoding strategy for the complex process sequence constraints and worker station capacity constraints in the aircraft assembly problem, so as to generate the assembly scheduling plan of the target aircraft using the optimized hierarchical coding strategy and decoding strategy. Thus, it is realized that the complex process constraints are converted into hierarchical coding using the hierarchical coding strategy, and the hierarchical coding is decoded using the queue data structure in the decoding strategy to obtain the final scheduling plan, and the search strategy based on statistical learning used in the optimization process can provide a high-quality initial solution for subsequent evolutionary learning, the multi-operator collaborative feedback search strategy based on reinforcement learning can analyze the optimal search operator of each layer, and the parameter selection strategy based on reinforcement learning can adjust the optimal parameters for problems of different scales, thereby realizing the modeling of the production scene of the aircraft assembly workshop, and quickly generating the aircraft assembly scheduling plan under the consideration of complex characteristics such as assembly process grouping, process serial parallel process relationship coupling, and station capacity constraints, and adaptively adjusting according to the actual aircraft assembly scheduling problem. This solves the problem that evolutionary algorithms in related technologies find it difficult to construct feasible solutions when faced with complex sequential constraints in the process planning of aircraft assembly process routes. In addition, when faced with capacity constraints on worker positions, number of processes, and scale differences in actual scenarios, they are prone to slow solution speeds and find it difficult to adaptively adjust their own strategies to solve different problems.

[0013] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0015] Figure 1 A schematic diagram of process constraints and workstation constraints for an aircraft assembly scheduling problem according to an embodiment of the present application;

[0016] Figure 2 A flowchart of a reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling provided according to an embodiment of the present application;

[0017] Figure 3 A flowchart of an evolutionary optimization method driven by enhanced dual learning according to an embodiment of the present application;

[0018] Figure 4 A schematic diagram of a global search strategy framework based on statistical learning according to an embodiment of the present application;

[0019] Figure 5 A schematic diagram of using a crossover operator to exchange the assembly order of processes within each layer according to an embodiment of the present application;

[0020] Figure 6 A schematic diagram of a framework for feedback regulation based on reinforcement learning according to an embodiment of the present application

[0021] Figure 7 A schematic diagram of a modeling process for an embodiment of the present application;

[0022] Figure 8 This is a coding and decoding schematic diagram of an embodiment of the present application; DETAILED DESCRIPTION

[0023] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0024] The following describes the enhanced dual-learning driven evolutionary optimization method and system for aircraft assembly scheduling according to the embodiment of the present application with reference to the accompanying drawings. In view of the fact that the evolutionary algorithm in the related art mentioned in the above background technology is difficult to construct a feasible solution when faced with the complex sequence constraint relationship between the processes in the aircraft assembly process route planning, and when faced with the capacity constraints of the worker positions, the number of processes and the scale differences in the actual scene, it is easy to have a slow solution speed and it is difficult to adaptively adjust its own strategy to solve different problems, the present application provides an enhanced dual-learning driven evolutionary optimization method for aircraft assembly scheduling, in which a certain hierarchical encoding strategy and decoding strategy can be designed and optimized for the complex process sequence constraints and the capacity constraints of the worker positions in the aircraft assembly problem, so as to generate the assembly scheduling plan of the target aircraft using the optimized hierarchical encoding strategy and decoding strategy. Thus, the hierarchical coding strategy is used to convert complex process constraints into hierarchical coding, and the queue data structure in the decoding strategy is used to decode the hierarchical coding to obtain the final scheduling plan. The search strategy based on statistical learning used in the optimization process can provide high-quality initial solutions for subsequent evolutionary learning. The multi-operator collaborative feedback search strategy based on reinforcement learning can analyze the optimal search operator of each layer. The parameter selection strategy based on reinforcement learning can adjust the optimal parameters for different scale problems, thereby realizing the production scene modeling of the aircraft assembly workshop, and quickly generating the aircraft assembly scheduling plan under the consideration of complex characteristics such as assembly process grouping, process serial and parallel process relationship coupling, and station capacity constraints, and adaptively adjusting according to the actual aircraft assembly scheduling problem. Thus, the evolutionary algorithm in the related technology is difficult to construct a feasible solution when facing the complex sequence constraint relationship between processes in the aircraft assembly process route planning, and it is easy to have a slow solution speed and difficult to adaptively adjust its own strategy to solve different problems when facing the capacity constraints of the worker's station in the actual scene, the number of processes and the scale difference.

[0025] Before explaining the reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling in the embodiment of the present application, the aircraft assembly scheduling problem involved in the implementation of the present application is first explained.

[0026] The problem is described as follows:

[0027] An assembly outline has a total of N process groups for assembly, each group contains N i Each process O ij Yes ij The preceding process constraint is T. ij , assembly station is M k , the number of people required is W ij , the entire assembly station can be divided into K small stations, and the number of people in each station is constrained to C kWhen the current number of people at a workstation does not reach the threshold, the subsequent processing steps need to compare whether the remaining positions are sufficient to accommodate O ij If there is no processing, you need to wait. If there is, you need to consider the constraints of the previous process and put it in the appropriate processing position. The optimization goal is to minimize the maximum completion time C max , which is calculated as:

[0028]

[0029] Among them, F ij Indicates O ij The completion time can be obtained by calculation. Figure 1 As shown, Figure 1 A schematic diagram of process constraints and workstation constraints for an aircraft assembly scheduling problem according to one embodiment of the present application.

[0030] Specifically, Figure 2 A flowchart of a reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling provided in an embodiment of the present application.

[0031] like Figure 2 As shown, the evolutionary optimization method driven by reinforcement dual learning for aircraft assembly scheduling includes the following steps:

[0032] In step S201, based on the process sequence constraints of the target aircraft assembly and the capacity constraints of the worker stations, an initial hierarchical encoding strategy and an initial decoding strategy in the evolutionary strategy are constructed.

[0033] It can be understood that the target aircraft here refers to the aircraft that needs to be assembled and scheduled. The initial hierarchical encoding strategy and initial decoding strategy in the evolutionary strategy here refer to the hierarchical encoding strategy and decoding strategy in the evolutionary strategy (evolutionary algorithm) constructed to solve the assembly scheduling problem of the target aircraft under the constraints of process sequence and worker station capacity.

[0034] In some embodiments, the actual scenario of aircraft assembly scheduling contains a complex process sequence relationship. When performing actual scheduling, it is necessary to design a scheduling plan according to the process sequence relationship. Therefore, when scheduling the aircraft assembly process, it is necessary to consider the process sequence constraint relationship, that is, the process sequence constraint. In addition, each assembly station of the aircraft has a certain worker station constraint. How many worker stations are there, how many worker stations have workers, and how many workers can be accommodated should all be considered when scheduling aircraft assembly.

[0035] Based on this, for the complex process sequence constraints in the aircraft assembly problem, the embodiment of the present application can design a certain hierarchical encoding strategy, so as to convert the complex process sequence constraints into hierarchical encoding; for the capacity constraints of the worker positions in the assembly, the embodiment of the present application can design a certain queue-based decoding strategy to use the queue data structure to decode the hierarchical encoding. The combination of hierarchical encoding strategy and decoding strategy can effectively solve the assembly scheduling problem of the target aircraft.

[0036] Step S202, optimizing the statistical learning performance and evolutionary learning performance of the initial layered coding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy and a preset parameter selection strategy to obtain an optimized layered coding strategy and an optimized decoding strategy.

[0037] During the actual implementation process, taking into account the divergence problems, solution speed problems and some other adaptive problems that may exist in the initial hierarchical coding strategy and the initial decoding strategy in the evolutionary strategy, the embodiment of the present application can also optimize the initial hierarchical coding strategy and the initial decoding strategy through preset search strategies, preset multi-operator collaborative feedback search strategies and preset parameter selection strategies, thereby achieving the optimization of the evolutionary strategy.

[0038] The preset search strategy here refers to a search strategy based on statistical learning that is pre-built to accelerate the convergence of the algorithm, which can provide high-quality initial solutions for subsequent evolutionary learning.

[0039] The preset multi-operator collaborative feedback search strategy here refers to a multi-operator collaborative feedback search strategy based on reinforcement learning that is pre-built in order to further accelerate the convergence of the algorithm, which can analyze the optimal search operator at each process layer.

[0040] The preset parameter selection strategy here refers to a reinforcement learning-based parameter selection strategy that is pre-built to enable the statistical learning-based search strategy to intelligently adjust the optimal parameters for problems of different scales.

[0041] Through certain statistical learning-based search strategies, reinforcement learning-based multi-operator collaborative feedback search strategies and reinforcement learning-based parameter selection strategies, the initial hierarchical encoding strategy and the initial decoding strategy can be optimized, and a reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling can be obtained, which can effectively solve the aircraft assembly scheduling problem.

[0042] Figure 3 This is a flow chart of an evolutionary optimization method driven by enhanced dual learning according to an embodiment of the present application. Figure 3 As shown:

[0043] Step 1: Read the process relationship and divide the processes into layers;

[0044] Step 2: Use the hierarchical coding method to generate an initial sampling population P1 of size ps, select the elite solutions of population P1 as statistical data, and then establish a probability statistical matrix for different layers according to the distribution of these solutions in different process layers, and initialize the probability statistical matrix;

[0045] Step 3, decode population P1 and calculate the objective function value;

[0046] Step 4: Count the process order distribution of the top λ% elite individuals in each layer;

[0047] Step 5, generate the offspring population C1 according to the probability distribution sampling in the probability statistics matrix, and evaluate the offspring objective function;

[0048] Step 6, merge P1 and C1, sort them in descending order according to the objective function, and retain the first ps solutions as the new population P1';

[0049] Step 7, update the probability distribution in the probability statistics matrix, if stage 1 is not finished, return to step 5, otherwise go to step 8;

[0050] Step 8, initializing the operator distribution statistics matrix;

[0051] Step 9, sampling the crossover operator for each solution in the population;

[0052] Step 10, using tournament selection to generate a mating pool population M;

[0053] Step 11, each individual in the population randomly performs the selected crossover operator with an elite in the mating pool to produce offspring;

[0054] Step 12, evaluate the offspring objective function to determine whether the parent generation can be updated, and count the number of successes and failures of the current operator;

[0055] Step 13, based on the success and failure statistics, update the operator distribution statistics matrix, merge the sub-population and the parent population, and retain the first ps solutions as the new population P1';

[0056] Step 14: If stage 2 is not completed, return to step 9; otherwise, the algorithm ends and the global optimal solution is output.

[0057] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial hierarchical encoding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy, it also includes: reading the process relationship of the target aircraft assembly scheduling, and stratifying the processes according to the process relationship to obtain different process layers; based on different process layers, generating an initial sampling population using the initial hierarchical encoding strategy, and arranging the initial sampling population according to the objective function to obtain the arranged initial sampling population, and selecting an elite solution based on the arranged initial sampling population; statistically calculating the distribution information of the elite solutions at different process layers, and establishing a probability statistical matrix based on the distribution information and initializing the probability statistical matrix to obtain the initialized statistical matrix; constructing a new population based on the probability distribution information in the initialized statistical matrix and the initial sampling population, so as to update the probability distribution information of the initialized statistical matrix according to the elite solution in the new population, to obtain the updated probability distribution information and its corresponding updated statistical matrix, so as to determine the preset search strategy based on the updated statistical matrix.

[0058] In some embodiments, in order to provide high-quality initial solutions for the evolutionary learning phase of the reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling in the present application and enhance the convergence of the population, the present application may utilize certain search strategies to locate the position of high-quality initial solutions in the solution space.

[0059] Figure 4 This is a schematic diagram of a global search strategy framework based on statistical learning according to an embodiment of the present application. Figure 4 As shown, the global search strategy based on statistical learning in the embodiment of the present application can be generally divided into three parts, but not limited to: first, the initial sampling population is constructed based on different process layers, and the sample set is constructed through the elite solution in the initial sampling population; then the process distribution of each group in the sample is counted to construct a probability statistical matrix; finally, according to the probability statistical matrix, new solutions are generated by sampling. The specific steps can be expressed as follows:

[0060] Sample collection: Based on different process layers, a certain hierarchical coding strategy is used to generate the initial sampling population, and all solutions in the population are arranged in ascending order according to the objective function. The sample set is constructed based on the top λ% individuals in the arranged initial sampling population as elite solutions.

[0061] Constructing statistical matrix: The statistical elite solves the distribution information at different process levels, and builds a probability statistical matrix based on the distribution information and initializes the probability statistical matrix to obtain an initialized statistical matrix. Among them, the element P in the initialized statistical matrix ij represents the probability of process i being at position j in the current set. Under the initial condition, P ij (1) L, indicating that all positions have the same probability of appearing.

[0062] Statistical matrix update: After the second iteration, the probability distribution information in the initialized statistical matrix is ​​statistically analyzed, and a new population is constructed in combination with the initial sampling population to update the probability distribution information of the initialized statistical matrix according to the position of the elite solution in the new population, and the initialized statistical matrix is ​​updated according to the updated probability distribution information. The specific process can be expressed in the form of a formula, but is not limited to, as follows:

[0063]

[0064] If process i appears at position j in sample k, then is 1.

[0065] Sampling solution generation: Scan the updated statistical matrix column by column, select the process of each column, sample the roulette selection strategy, normalize the probability of the jth column, and generate a random number x. When x is greater than the probability P i-1,j , less than P ij , then select process i. After selecting process i, set all probabilities of the i-th row in the matrix to 0 to prevent process i from being selected repeatedly. Repeat the above steps until all processes are selected, thus completing the design of the global search strategy based on statistical learning.

[0066] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial hierarchical encoding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy, it also includes: constructing an operator distribution matrix and initializing the operator distribution matrix based on different process layers to obtain an initialized operator distribution matrix; by initializing the operator matrix, evolving the hierarchical search performance of evolutionary learning and reinforcing the feedback adjustment performance of learning to obtain a hierarchical search strategy after evolutionary learning and a feedback adjustment strategy after reinforcement learning; analyzing the optimal search operators of different process layers in combination with the hierarchical search strategy after evolutionary learning and the feedback adjustment strategy after reinforcement learning to generate a preset multi-operator collaborative feedback search strategy.

[0067] In other embodiments, the global search strategy based on statistical learning in the first stage can quickly search the solution space, but due to the limited number of samples, the final probability distribution will converge to a steady-state distribution, and the algorithm may lose its search capability at this time. Therefore, the embodiment of the present application can design a certain multi-operator collaborative feedback search strategy based on reinforcement learning in the second stage. For example, based on different process layers, an operator distribution matrix is ​​constructed and initialized, and the hierarchical search performance of evolutionary learning and the feedback adjustment performance of reinforcement learning are evolved through the initialized operator matrix, so as to analyze the optimal search operators of different process layers, obtain a preset multi-operator collaborative feedback search strategy, and realize the collaborative evolution of reinforcement learning. Among them, the multi-operator collaborative feedback search strategy includes but is not limited to two parts: hierarchical search based on evolutionary learning and feedback adjustment based on reinforcement learning.

[0068] Hierarchical search based on evolutionary learning (multi-operator collaborative hierarchical evolution): First, a two-player tournament algorithm is used to select a mating pool from the population. Secondly, each individual in the population randomly selects a parent from the mating pool. Next, considering that the hierarchical coding is used to represent the assembly scheme in the embodiment of the present application, the crossover operator can be executed independently between each layer. Specifically, the crossover operator can be, but is not limited to, randomly selecting a crossover method from the precedence operation crossover (POX), partial matching crossover (PMX), and order crossover (OX) to exchange the assembly order of the processes in each layer. Figure 5 This is a schematic diagram of using a crossover operator to exchange the assembly order of processes in each layer in one embodiment of the present application, such as Figure 5 As shown, the detailed process of exchanging different crossover operators is demonstrated.

[0069] Feedback adjustment based on reinforcement learning: Since the optimal crossover operators for solving different problems at different layers are different, in order to enable the algorithm to have adaptive adjustment capabilities, the embodiments of the present application may, but are not limited to, adopt a feedback strategy based on reinforcement learning to enhance the selection probability of the dominant crossover operator to enhance the convergence of the algorithm. Figure 6 This is a schematic diagram of a framework of feedback regulation based on reinforcement learning according to an embodiment of the present application. Figure 6As shown, the specific process can be but not limited to the following: first, the data set is empty in the initial stage, so the embodiment of the present application can allow the algorithm to randomly select a crossover operator, and after iterating h generations, the number of successes and failures of each operator is counted to form an initial data set; then, the number of successes and failures of each generation of the operator is added to the data set. In order to ensure that the algorithm is not affected by useless search data in the early stage, when the number of data in the data set exceeds the threshold E, an equal amount of data is deleted starting from the first row; then, the selection probability of each operator of the next generation is calculated according to the data set, and the calculation formula can be but not limited to: p = number of successes / (number of successes + number of failures); then, all selection probabilities are normalized as the selection probability of the next generation evolution operator; finally, if the algorithm has iterated for h generations, then starting from the h+1 generation, according to the selection probability of the operator, a roulette strategy is sampled to select a crossover operator for each solution in the population, and the same crossover operator is used for all layers in the solution, and the above steps are repeated until the algorithm ends.

[0070] Optionally, in one embodiment of the present application, before optimizing the statistical learning performance and evolutionary learning performance of the initial layered coding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy, it also includes: defining the state set, action set, reward, and state transition of the initial layered coding strategy and the initial decoding strategy; based on the state set, action set, reward, and state transition, determining the parameter selection range of the preset search strategy to determine the preset parameter selection strategy.

[0071] In some embodiments, the number of samples in the first stage of statistical learning may affect the learning efficiency of the model. For example, too few samples may lead to rapid convergence of the distribution, which may result in poor initial population positioning. Too many samples may cause the population to spend too much time evaluating the quality of the sampled solution, thereby reducing the learning efficiency. Therefore, the present application may design a certain parameter selection strategy based on reinforcement learning.

[0072] Specifically, Figure 7 This is a schematic diagram of the modeling process of an embodiment of the present application. The embodiment of the present application can be, but is not limited to, limiting the parameter selection range such as P = [10, 15, 20, 30, 40, 60], and modeling the algorithm parameter selection as follows Figure 7 The Markov decision process shown in Figure 1 is a process in which the application first needs to define the state set, action set, reward, and state transition in the algorithm.

[0073] State set: In order to accurately describe the state transition change, the embodiment of the present application can use the current parameter selection as the state and divide the algorithm into 6 states. If the number of samples currently selected is 10, the state is 1, and the other value states are 2-6 respectively.

[0074] Action set: When the algorithm detects that the probability distribution matrix of the statistical learning model in the current state converges and the covariance with the previous matrix is ​​greater than 0.99, the algorithm is considered to have converged and other parameters need to be reselected. Therefore, the action set is also a parameter set. For example, if the current state is P = 10, then the parameter is selected to be modified to P = 30, and the current selected action is 4.

[0075] Reward: After selecting the current action, after another round of statistical learning, the probability distribution of the statistical matrix is ​​recalculated according to the new sample set, and the similarity with the previous generation matrix is ​​calculated. If the similarity is less than 0.99, that is, the current parameters have updated the distribution of the population, broadened the sampling range of the model, can generate new solutions, and enhanced the diversity of the initial population, then Rt = 10, otherwise Rt = 0.

[0076] State transition: After the current action receives a reward, due to the selection of other parameters, the next state St+1 becomes the selected action At.

[0077] Model update: When a state transition occurs, the model collects a set of strategies, namely (St, At, Rt, St+1), and the Q table update formula can be expressed as: Q(St, At) = Q(St, At) + α*(γ*max(Q(St+1, At))-Q(St, At)).

[0078] In summary, the specific steps of the parameter selection strategy based on reinforcement learning in the embodiment of the present application can be expressed as follows:

[0079] Step 1: Initialize parameter selection range;

[0080] Step 2: Initialize the Q table, Q(6,6);

[0081] Step 3: Initialize the current state St=2, At=2;

[0082] Step 4: Execute the statistical learning module;

[0083] Step 5: Calculate the statistical matrix covariance value C. If C>0.99, reselect the action. If the random number β>0.95, randomly select an action. Otherwise, select the action with the largest Q value. If C<0.99, give a reward Rt=10, maintain the current state, and update the Q table.

[0084] Step 6: If the action changes, that is, the number of samples P changes, if P becomes larger, the remaining samples are supplemented with a random initialization strategy; if P becomes smaller, the samples are arranged in ascending order according to the objective function and the redundant samples are deleted;

[0085] Step 7: If the end condition is met, the statistical learning phase ends, otherwise, execute step 4.

[0086] Step S203, generating a new evolutionary strategy according to the optimized layered encoding strategy and the optimized decoding strategy, so as to execute the new evolutionary strategy to schedule the target aircraft assembly.

[0087] After optimizing the initial hierarchical coding strategy and the initial decoding strategy through certain search strategies, multi-operator collaborative feedback search strategies and parameter selection strategies, the embodiment of the present application realizes the optimization of the evolutionary strategy, and can execute a new evolutionary strategy, and use the optimized hierarchical coding strategy and decoding strategy in the new evolutionary strategy to generate the final scheduling plan for the target aircraft assembly scheduling problem, thereby scheduling the assembly of the target aircraft.

[0088] Optionally, in one embodiment of the present application, a new evolutionary strategy is generated based on the optimized hierarchical coding strategy and the optimized decoding strategy to execute the new evolutionary strategy to schedule the target aircraft assembly, including: based on different process layers, using the optimized hierarchical coding strategy to obtain a complete scheduling solution; based on the scheduling solution, using the optimized decoding strategy to calculate the time constraints of the target aircraft assembly to obtain the final scheduling plan for the target aircraft assembly, so as to schedule the target aircraft assembly according to the final scheduling plan.

[0089] In the actual execution process, the final scheduling plan for the target aircraft assembly is generated according to the optimized hierarchical coding strategy and the optimized decoding strategy. The main thing is to hierarchically divide the processes in the target aircraft assembly scheduling into different process layers, and then use the hierarchical coding method to generate the initial population based on different process layers to obtain a complete scheduling solution. Finally, the queue decoding method is used to calculate the objective function value of the target aircraft assembly time constraint to obtain the final scheduling plan for the target aircraft.

[0090] Among them, the process stratification is mainly based on the sequential relationship of the process groups, indexing the child nodes of each process group, adding one to the number of layers of the child nodes, and then recursively obtaining the stratification of all process groups, wherein the number of initialization layers within each process group is equal to the number of layers of the process group; then, the number of layers of the processes in each group is counted, and the number of layers is also calculated recursively; finally, the layer index of all processes is obtained, and the processes at the same layer are summarized into a set to ensure the sequential relationship between each layer and the parallel relationship between the same layers, thereby ensuring the feasibility of the solution.

[0091] Layered encoding: Based on the set of processes at each layer, different permutation sequences are randomly generated, and the fragments of each group are spliced ​​together to form a complete scheduling solution.

[0092] Queue decoding: First, establish a process queue Q for each workstation k ; Then, sequentially take out the assembly process Q from the scheduling solution ij ; According to the P of the previous process ijCompletion time calculation process Q ij The constraint start time S ij =max(F(P ij )); Then, scan the selected station M k The process queue is found to have a start time less than or equal to S ij , the completion time is greater than S ij The process set H is sorted in ascending order according to the completion time, and the process Q′ in H is removed in sequence ij , determine whether the remaining capacity can accommodate process Q ij :If it can be accommodated, then the start time is the constrained start time S ij Otherwise, S ij =max(S ij ,F′ ij );Then, calculate the process Q ij The completion time F ij =S ij +T ij ; Finally, the completion time of all processes is calculated, and the maximum completion time C can be obtained max .

[0093] Thus, the decoding calculation of the assembly time constraint of the target aircraft is completed, and the final scheduling plan is generated according to the completion time and maximum completion time of all processes.

[0094] Figure 8 This is a schematic diagram of encoding and decoding according to an embodiment of the present application. Figure 8 As shown, the embodiment of the present application can convert two layers of complex process constraints into hierarchical codes, decode the hierarchical codes using a queue data structure, and obtain a final scheduling solution.

[0095] Additionally, the embodiment of the present application also randomly generated 20 test problems based on actual production data for testing to verify the generalization and convergence of the reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling of the embodiment of the present application on aircraft assembly scheduling problems of different scales.

[0096] The parameter settings during the test can be expressed as follows: the number of process groups N = {20, 25, 30, 35}, the number of processes within the group Ni = {20, 25, 30, 35, 40}, divided into 6 types of workstations according to the aircraft filling position, the threshold of the number of workstations is 10, the processing time is randomly selected from U to [1, 10], the number of workers required is U to [2, 4], and the workstations of the process group are randomly selected from 6 workstations. All strategies and corresponding algorithms contained in the method are independently run 20 times on 20 test problems to verify the generalization of this application using statistical results. Furthermore, this application implements the test problems of 3 actual aircraft manufacturing workstations to verify the effectiveness of the method proposed in this application.

[0097] Among them, the algorithm in the embodiment of the present application is programmed with Matlab, the simulation environment is Intel (R) core (TM) i9-13900k CPU @ 3.4GHz / 16GB RAM, GPU NVIDIA4070 12GB, and the operating system is Windows 10. The termination criterion of the algorithm is that the algorithm stops when the maximum running time MaxT of the algorithm reaches 0.025*N*Ni. In the experiment, the relative percentage deviation (Relative Percent Difference, RPD) is used to measure the gap between the algorithm and the lower bound. The lower bound is the minimum objective function value found by all algorithms. The RPD calculation formula is RPD = Cmax / (Cmax-LB).

[0098] In order to verify the effectiveness of the REL (co-evolution based on reinforcement learning) module, five variant algorithms were designed in the experiment: EL_POX, EL_PMX, EL_OX, all of which represent evolutionary learning using various crossover operators alone, REL represents the co-evolution algorithm based on reinforcement learning, RSL represents RSLEA (reinforcement dual learning driven evolutionary optimization method for aircraft assembly scheduling) with the second stage deleted, and REL represents RSLEA with the first stage deleted. Tables 1 and 2 are statistical tables of the comparison results of the best RPD value and the average RPD value of REL and variant algorithms on different test problems, which can be expressed as follows:

[0099] Table 1

[0100]

[0101]

[0102] Table 2

[0103]

[0104]

[0105] As shown in Table 1 and Table 2, it can be seen that REL in the embodiment of the present application has achieved better optimal indicators and average indicators than EL_OX, EL_PMX, and EL_POX, verifying the effectiveness of the reinforcement learning operator selection module. In addition, by comparing the results of RSLEA, RSL, and REL, it can be seen that after RSLEA deletes any different stage, the algorithm performance is significantly reduced, which can verify the effectiveness of the REL module in the embodiment of the present application.

[0106] Furthermore, in order to accurately verify the effectiveness of the RSL module, the embodiment of the present application also designed 7 variant algorithms in the experiment, namely: SLREL with 6 different sampling parameters, and REL without SL (supervised learning) assisted evolution. Tables 3 and 4 are statistical tables of the comparison results of the best RPD value and the average RPD value of RSL and variant algorithms on different test problems, which can be expressed as follows:

[0107] Table 3

[0108]

[0109]

[0110] Table 4

[0111]

[0112] As shown in Table 3 and Table 4, it can be seen that the average RPD index of REL has decreased compared with SLREL with different parameters, which verifies that SL can improve the convergence of REL by selecting different sampling ranges, and verifies the effectiveness of the SL module. Comparing RSLEA with various SLRELs, better indicators have been achieved in both optimal RPD and average RPD, indicating that the RSL module can make the performance of REL comprehensively improved, and verifies that the reinforcement learning parameter adaptation module can spend a small amount of computing resources to enhance the algorithm's adaptability and intelligence in parameter selection.

[0113] Table 5 is a ranking table of the Friedeman rank sum test results of RSLEA and other variant algorithms, which can be expressed as follows:

[0114] Table 5

[0115]

[0116] As shown in Table 5, Table 5 shows the rank sum test results of all variant algorithms. RSLEA achieves the highest ranking compared with other variant algorithms, indicating that the algorithm performance will decrease if any improved module is deleted, verifying the effect of the proposed modules and corresponding algorithms on improving the convergence of RSLEA in the embodiment of the present application.

[0117] In order to further verify the advantages of the method RSLEA in the embodiment of the present application, the embodiment of the present application also selected three latest algorithms, namely HGA, QIGA and EGHA. In order to compare the generalization of the algorithms, all methods were independently run 20 times on 20 test problems at the same time. Table 6 is a statistical table of the results of the RPD index of RSLEA and the comparison algorithm in the test set, which can be expressed as follows:

[0118] Table 6

[0119]

[0120]

[0121] As shown in Table 6, it can be seen that the RSLEA method in the embodiment of the present application has achieved the best RPD index and RPD mean index on the 20 test problems. It shows the significance of the 20 running results of the statistical algorithm on each test problem, where '-' means that the algorithm is significantly worse than RSLEA. The results show that all the comparison algorithms are significantly worse than RSLEA on the 20 test problems, which verifies that the RSLEA method has good effectiveness and generalization on test problems of different scales.

[0122] In addition, the embodiment of the present application also selected three actual cases to verify the excellent application ability of the RSLEA method in the actual aircraft assembly scheduling scenario. These three cases are the assembly outlines of the aircraft's wing docking station, fuselage hole making station, and horizontal tail assembly station. Each algorithm runs independently 30 times, and the running time is 10 seconds. Tables 7 and 8 are both statistical tables of the application results of the RPD indicators of RSLEA and the comparison algorithm in the actual aircraft assembly scheduling problem, which can be expressed as follows:

[0123] Table 7

[0124]

[0125] Table 8

[0126]

[0127] As shown in Tables 7 and 8, the results show that RSLEA achieved the best optimal RPD and average RPD indicators, further verifying the effectiveness of the enhanced dual-learning driven evolutionary optimization method for aircraft assembly scheduling in the embodiment of the present application.

[0128] According to the evolutionary optimization method driven by enhanced dual learning for aircraft assembly scheduling proposed in the embodiment of the present application, a certain hierarchical coding strategy and decoding strategy can be designed and optimized for the complex process sequence constraints and worker station capacity constraints in the aircraft assembly problem, so as to generate the assembly scheduling plan of the target aircraft using the optimized hierarchical coding strategy and decoding strategy. Thus, it is realized that the complex process constraints are converted into hierarchical coding using the hierarchical coding strategy, and the hierarchical coding is decoded using the queue data structure in the decoding strategy to obtain the final scheduling plan, and the search strategy based on statistical learning used in the optimization process can provide a high-quality initial solution for subsequent evolutionary learning, the multi-operator collaborative feedback search strategy based on reinforcement learning can analyze the optimal search operator of each layer, and the parameter selection strategy based on reinforcement learning can adjust the optimal parameters for problems of different scales, thereby realizing the modeling of the production scene of the aircraft assembly workshop, and quickly generating the aircraft assembly scheduling plan under the consideration of complex characteristics such as assembly process grouping, process serial and parallel process relationship coupling, and station capacity constraints, and adaptively adjusting according to the actual aircraft assembly scheduling problem. This solves the problem that evolutionary algorithms in related technologies find it difficult to construct feasible solutions when faced with complex sequential constraints in the process planning of aircraft assembly process routes. In addition, when faced with capacity constraints on worker positions, number of processes, and scale differences in actual scenarios, they are prone to slow solution speeds and find it difficult to adaptively adjust their own strategies to solve different problems.

[0129] An embodiment of the present application also provides a computer program system, including a computer program, which can run computer instructions. When the computer instructions are executed by a processor, the reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling provided in the embodiment of the present application is implemented.

[0130] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0131] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0132] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

Claims

1. A reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling, characterized in that: The following steps are involved: Based on the process sequence constraints of the target aircraft assembly and the capacity constraints of the worker positions, the initial hierarchical encoding strategy and initial decoding strategy in the evolutionary strategy are constructed; Optimizing the statistical learning performance and evolutionary learning performance of the initial hierarchical coding strategy and the initial decoding strategy through a preset search strategy, a preset multi-operator collaborative feedback search strategy, and a preset parameter selection strategy to obtain an optimized hierarchical coding strategy and an optimized decoding strategy; A new evolutionary strategy is generated according to the optimized hierarchical encoding strategy and the optimized decoding strategy, so as to execute the new evolutionary strategy to schedule the assembly of the target aircraft.

2. The method according to claim 1, characterized in that Before optimizing the statistical learning performance and the evolutionary learning performance of the initial layered coding strategy and the initial decoding strategy by using the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, the method further includes: Reading the process relationship of the target aircraft assembly scheduling, and stratifying the processes according to the process relationship to obtain different process layers; Based on the different process layers, an initial sampling population is generated using the initial hierarchical coding strategy, and the initial sampling population is arranged according to the objective function to obtain an arranged initial sampling population, and an elite solution is selected according to the arranged initial sampling population; Counting the distribution information of the elite solutions at the different process layers, and establishing a probability statistical matrix according to the distribution information and initializing the probability statistical matrix to obtain an initialized statistical matrix; A new population is constructed based on the probability distribution information in the initialized statistical matrix and the initial sampling population, so as to update the probability distribution information of the initialized statistical matrix according to the elite solution in the new population, and obtain the updated probability distribution information and its corresponding updated statistical matrix, so as to determine the preset search strategy according to the updated statistical matrix.

3. The method according to claim 2, characterized in that Before optimizing the statistical learning performance and the evolutionary learning performance of the initial layered coding strategy and the initial decoding strategy by using the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, the method further includes: Based on the different process layers, construct an operator distribution matrix and initialize the operator distribution matrix to obtain an initialized operator distribution matrix; By initializing the operator matrix, the hierarchical search performance of evolutionary learning and the feedback adjustment performance of reinforcement learning are evolved, so as to obtain the hierarchical search strategy after evolutionary learning and the feedback adjustment strategy after reinforcement learning; The optimal search operators of the different process layers are analyzed in combination with the hierarchical search strategy after evolutionary learning and the feedback adjustment strategy after reinforcement learning to generate the preset multi-operator collaborative feedback search strategy.

4. The method according to claim 1, characterized in that: Before optimizing the statistical learning performance and evolutionary learning performance of the initial layered coding strategy and the initial decoding strategy through the preset search strategy, the preset multi-operator collaborative feedback search strategy and the preset parameter selection strategy, the method further includes: Defining a state set, an action set, a reward, and a state transition of the initial layered encoding strategy and the initial decoding strategy; Based on the state set, the action set, the reward and the state transition, a parameter selection range of the preset search strategy is determined to determine the preset parameter selection strategy.

5. The method according to claim 2, characterized in that: The step of generating a new evolutionary strategy according to the optimized hierarchical coding strategy and the optimized decoding strategy to execute the new evolutionary strategy to schedule the target aircraft assembly includes: Based on the different process layers, the optimized hierarchical coding strategy is used to obtain a complete scheduling solution; Based on the scheduling solution, the time constraint of the target aircraft assembly is calculated using the optimized decoding strategy to obtain a final scheduling solution for the target aircraft assembly, so as to schedule the target aircraft assembly according to the final scheduling solution.

6. A computer program system, comprising a computer program, characterized in that: When the computer program is executed, it is used to implement the reinforcement dual-learning driven evolutionary optimization method for aircraft assembly scheduling as described in any one of claims 1-5.

Images