Multi-target scheduling method and system for automobile engine manufacturing workshop

Abstract

The invention provides a multi-target scheduling method and system for an automobile engine manufacturing workshop, and relates to the technical field of the automobile industry, and the method comprises the steps: obtaining the technological parameters and technical specifications of the automobile engine manufacturing workshop; constructing a multi-target scheduling target function; initializing a population and actor-commentator network; constructing state vectors of individuals contained in the population; performing neighborhood search on the individuals to form trajectory experience of the individuals; judging whether the age of the individual is greater than a preset age; selecting dominant solutions in the population to form dominant sub-populations; preferentially determining a target solution from the current external optimal file, and updating the dominant sub-population by taking each solution in the dominant sub-population as an initial solution; comparing the target value of the updated solution with the target value of the solution in the Pareto optimal file; uniformly and randomly sampling multiple track experiences from the experience pool, and updating network parameters of the actor-commentator network; judging whether the execution time reaches preset execution time or not;

Description

Technical Field

[0001] This invention relates to the field of automotive industry technology, and in particular to a multi-objective scheduling method and system for automotive engine manufacturing workshops. Background Technology

[0002] The automotive industry is a pillar industry of the national economy, characterized by a long industrial chain, high correlation with other industries, wide employment opportunities, and significant consumer spending. As the "heart" of a car, the engine's performance, quality, and manufacturing cost directly determine the core competitiveness of the entire vehicle. Engine manufacturing is an extremely complex and precise engineering process, involving the machining and final assembly of various core components such as the cylinder block, cylinder head, crankshaft, and connecting rods. It involves a long process route and high precision requirements.

[0003] Existing technologies have very few applications for EEDAPSP-SDST (Energy-saving scheduling problem of distributed assembly line workshop considering sequence-dependent preparation time). Only Yu et al. proposed the Jaya algorithm driven by dual learning strategies. On the one hand, the algorithm guides the non-dominated solution to select a suitable machine speed adjustment strategy through the Q-learning state-action strategy (QL1). On the other hand, it constructs 8 neighborhood structures and proposes 3 new Q-learning strategies (QL2-QL4) to dynamically select neighborhood structures to optimize the search direction and improve convergence. By integrating Q-learning and neighborhood search, the algorithm improves convergence and solution quality in multi-objective optimization.

[0004] However, existing automotive engine manufacturing workshops have not fully developed the EEDAPSP-SDST solution space features, resulting in insufficient search efficiency; at the same time, traditional Q-learning methods are prone to getting stuck in local plateaus due to the excessive greediness of the ε-greedy strategy. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a multi-objective scheduling method for automobile engine manufacturing workshops, which can solve the problems of insufficient search efficiency caused by the failure to fully develop the EEDAPSP-SDST solution space features in the prior art; at the same time, the traditional Q-learning method is prone to getting stuck in local plateaus due to the excessive greediness of the ε-greedy strategy.

[0006] A first aspect of this invention provides a multi-objective scheduling method for an automotive engine manufacturing workshop, comprising: S1: Obtain the process parameters and technical specifications of the automotive engine manufacturing workshop.

[0007] S2: Construct a multi-objective scheduling objective function based on process parameters and technical specifications.

[0008] S3: Initialize the population and actor-critic network.

[0009] S4: Construct the state vectors of the individuals in the population based on the multi-objective scheduling objective function.

[0010] S5: Through the actor network, based on the state vector, a neighborhood search is performed on the individual to form the individual's trajectory experience.

[0011] S6: Store the individual's trajectory experience into the experience pool.

[0012] S7: Determine if the individual's age is greater than the preset age. If yes, proceed to the next step. Otherwise, return to step S5.

[0013] S8: Select the dominant solution from the population to form a dominant subpopulation.

[0014] S9: Select the target solution from the current best external files, take each solution in the dominant subpopulation as the starting solution, and gradually move the starting solution toward the target solution through swapping or insertion operations to update the dominant subpopulation.

[0015] S10: For the solutions in the updated dominant subpopulation, implement a Q-learning-based energy-saving strategy.

[0016] S11: Compare the objective values ​​of the solutions in the updated dominant subpopulation with the objective values ​​of the solutions in the Pareto optimal file, store the new non-dominated solutions in the updated dominant subpopulation into the optimal Pareto file, and remove the dominated solutions from the Pareto file.

[0017] S12: A reward backtracking mechanism is adopted to add backtracking rewards to the solution sequence that makes a positive contribution in the Pareto non-dominated solution.

[0018] S13: Randomly sample multiple trajectory experiences from the experience pool to update the network parameters of the actor-critic network.

[0019] S14: Determine if the execution time has reached the preset execution time. If yes, output the final Pareto optimal file. Otherwise, return to step S4.

[0020] A second aspect of this invention provides a multi-objective scheduling system for an automotive engine manufacturing workshop, comprising: a processor and a memory; The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the multi-objective scheduling method for an automotive engine manufacturing workshop as described in the first aspect.

[0021] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, initializing the population reduces ineffective searches; performing neighborhood searches on individuals using the actor network and updating the actor-critic network enables targeted search paths, significantly improving search efficiency; simultaneously, updating the dominant subpopulation fully utilizes the solution space features of EEDAPSP-SDST. Furthermore, using the actor network instead of the traditional Q-learning method avoids convergence bias caused by excessive greed; and employing a reward backtracking mechanism prevents the algorithm from getting stuck in local plateaus, enhancing global search capabilities. Attached Figure Description

[0022] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0023] Figure 1 This is a flowchart illustrating a multi-objective scheduling method for an automobile engine manufacturing workshop provided in an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of the EEDAPSP-SDST structure of a multi-objective scheduling method for an automobile engine manufacturing workshop provided in an embodiment of the present invention.

[0025] Figure 3 This is an actor network structure diagram of a multi-objective scheduling method for an automobile engine manufacturing workshop provided in an embodiment of the present invention.

[0026] Figure 4 This is a commentator network structure diagram of a multi-objective scheduling method for an automobile engine manufacturing workshop provided in an embodiment of the present invention.

[0027] Figure 5 This is a schematic diagram of the structure of a multi-objective scheduling system for an automobile engine manufacturing workshop provided in an embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] The multi-objective scheduling method for automobile engine manufacturing workshops provided by the present invention will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0030] Reference manual attached Figure 1 The diagram shows a flowchart of a multi-objective scheduling method for an automobile engine manufacturing workshop provided by an embodiment of the present invention.

[0031] Reference manual attached Figure 2 The diagram shows a schematic of the EEDAPSP-SDST structure of a multi-objective scheduling method for an automobile engine manufacturing workshop provided by an embodiment of the present invention.

[0032] Reference manual attached Figure 3 The diagram illustrates the actor network structure of a multi-objective scheduling method for an automobile engine manufacturing workshop provided by an embodiment of the present invention.

[0033] Reference manual attached Figure 4 The diagram shows a commentator network structure diagram of a multi-objective scheduling method for an automobile engine manufacturing workshop provided by an embodiment of the present invention.

[0034] This invention provides a multi-objective scheduling method for an automotive engine manufacturing workshop, which may include the following steps: S1: Obtain the process parameters and technical specifications of the automotive engine manufacturing workshop.

[0035] The specific process parameters and technical specifications for the automotive engine manufacturing workshop include: the number of workpieces. N Product quantity H Number of machines Q Number of factories R Number of machine-adjustable discrete speeds D Subordinate relationship between workpiece and product Oh h Workpiece preparation time matrix S and product preparation time matrix S A The processing time of the workpiece on each machine at the reference speed P ij and initial processing speed matrix Z Energy consumption per unit time of the machine during processing Energy consumption per unit time in idle state And the energy consumption of each machine shutdown and restart. ,in, v ij The first in the velocity matrix i Line number j The speed corresponding to each column element.

[0036] S2: Construct a multi-objective scheduling objective function based on process parameters and technical specifications.

[0037] In one possible implementation, S2 specifically includes sub-steps S201 to S207: S201: Set up a scheduling plan based on process parameters and technical specifications.

[0038] Among them, the scheduling scheme C Includes: allocation to all R In the factories R workpiece sequence Product assembly sequence and a size of N * Q Velocity selection matrix .

[0039] S202: Determine the workpiece completion time according to the scheduling plan: ; in, Indicates the first f The first factory i The number of each processed workpiece. express In the machine M j The completion time is shown above, and max indicates the maximum value. express In the machine M j-1 The completion time on the website Indicates the first f The first factory i -1 workpiece number, express In the machine M j The completion time on the website Indicates in the machine M j Above, at back, Preparation time beforehand Indicates in the machine M j superior Processing time.

[0040] S203: Based on the workpiece completion time, determine the processing completion time of the last workpiece on the last machine: ; in, β h Indicates the firsth The serial number of each assembled product. express β h The time when the last workpiece is completed on the last machine. Oh h Indicates the first h The collection of all workpieces to which an assembled product belongs.

[0041] S204: Determine the product assembly completion time based on the processing completion time: ; in, express β h Assembly completion time, β h-1 Indicates the first h -1 assembly product number, express β h-1 Assembly completion time, Indicates in assembly machine M A Above, at back, β h Preparation time beforehand Indicates in assembly machine M A superior β h Processing time.

[0042] S205: Calculate the maximum completion time of the scheduling plan based on the assembly completion time: ; in, C Indicate the scheduling scheme, C max ( C ) indicates the maximum completion time of the scheduling scheme.

[0043] S206: Calculate the total energy consumption of the scheduling scheme: ; in, TEC ( C The total energy consumption of the scheduling scheme is represented by ) E P ( C The ) represents the total processing energy consumption of the scheduling scheme. E I ( C The ) represents the total idle energy consumption of the scheduling scheme. E S ( C ) represents the total energy consumption for starting and stopping the scheduling scheme.

[0044] in, , E ij For machines M j With speed v z Machining workpieces J i Energy consumption per unit time p ijz For machines M j With speed v z Machining workpieces J i The actual processing time at that time v z The first in the discrete velocity set z Speed ​​levels.

[0045] in, , E idle This represents the energy consumption per unit time when the machine is idle. For the factory F f China Machinery M j Free time.

[0046] in, , C f For the factory F f The completion time of the last workpiece in the process. For the factory F f China Machinery M j The shutdown time.

[0047] in, , SD T This represents the total number of times the machine has been shut down. E off Power consumption for machine word shutdown and restart.

[0048] S207: Construct a multi-objective scheduling objective function with the goals of maximizing completion time and total energy consumption: ; Where min represents taking the minimum value.

[0049] Specifically, considering minimizing the maximum completion time and total energy consumption, a solution model for the energy-saving scheduling objective function of a distributed assembly line workshop that takes into account the sequence preparation time is established for automobile engine manufacturing workshops.

[0050] In this embodiment of the invention, when constructing the multi-objective scheduling objective function, a scheduling scheme is first set up in conjunction with process parameters and technical specifications to lay the foundation for subsequent calculations. Then, the completion time of the workpiece is determined according to the scheduling scheme, and the processing completion time of the last workpiece on the last machine is calculated. Based on this, the assembly completion time of the product is determined, and the maximum completion time of the scheduling scheme is obtained accordingly. At the same time, the total energy consumption of the scheduling scheme is accurately calculated. Finally, the objective function is constructed with "minimizing the maximum completion time" and "minimizing the total energy consumption" as the core. The entire process follows the actual process logic of engine manufacturing, decomposing the core elements of scheduling layer by layer. This ensures the quantitative accuracy of the two major objectives of time and energy consumption, and allows the objective function to accurately adapt to the actual scheduling requirements of EEDAPSP-SDST.

[0051] S3: Initialize the population and actor-critic network.

[0052] The population refers to a set of multiple "solutions", each of which corresponds to a complete scheduling scheme (including key scheduling information such as product processing sequence, workpiece processing order, and machine allocation).

[0053] The Actor-Critic Network is a classic model architecture in reinforcement learning. Its core consists of two sub-networks, "Actor" and "Critic," working collaboratively to solve the problem of decision-making and value assessment in dynamic environments. The "Actor" is responsible for generating specific action strategies; for example, a game AI decides its next move or a robot chooses its trajectory. It continuously adjusts its strategy through trial and error, aiming to maximize long-term gains. The "Critic," on the other hand, focuses on evaluating the value of the "Actor's" actions. Based on the current environmental state and the actor's actions, it calculates the total potential reward of the action, providing feedback to the actor on whether the action is good or bad. The two interact in real time: the actor optimizes its strategy based on the critic's evaluation, while the critic updates its value judgment by combining environmental feedback and the actor's new actions, ultimately helping the agent gradually learn to make better decisions in complex scenarios.

[0054] In one possible implementation, S3 specifically includes sub-steps S301 to S303: S301: Initialize the population.

[0055] Specifically, construction H One by P h The product series with the first productβ ( h ),in h =1,2,…, H Following the greedy principle, the remaining products are added to each product sequence in turn. At each step, the product with the shortest preparation time compared to the last product in the current sequence is selected from the unsorted products and appended to the end of the sequence, until a complete product sequence is constructed, generating a sequence of size equal to the number of products. H initial population u (0).

[0056] S302: Initialize the network parameters of the actor-critic network.

[0057] The network parameters of the actor-critic network specifically include: sampling batch size. BA =128, the learning rate of the actor and critic network. α =0.4, future return discount factor c =0.8, the number of segments in the local search algorithm SP =8, state vector Action space a ,award r Each network weight and bias term, and the number of iterations. g =0.

[0058] The actor-critic network specifically includes: a 12-dimensional input layer, a hidden layer consisting of 64 neurons, and a ReLU activation function. f ( x )=max(0, x ),in x The input values ​​of the neurons are represented by the output layer dimension of the actor network, which is set to the size of the action space. A =7. Value estimation of the output state of the output layer of the critic network V ( s Actor network parameters and critic network parameters; Actor network parameters i π Including hidden layer weights and bias Output layer weights and bias The input layer to hidden layer weight matrix is ​​in the interval Uniform random sampling within the inner layer, weights from hidden layer to output layer within the range Internal random sampling, all bias terms initialized to zero; commentator network parameters i v Including hidden layer weights and bias Output layer weights and bias The input layer to hidden layer weight matrix is ​​in the interval Uniform random sampling within the inner layer, weights from hidden layer to output layer within the range Internal random sampling, with all bias terms initialized to zero.

[0059] S303: Copy the network parameters of the actors and critics respectively, construct the corresponding target networks, set the soft update coefficient, and update the target network parameters: ; in, This represents the target network parameters of the actor network. t Indicates the soft update coefficient. , i π This represents the current network parameters of the actor network. This represents the target network parameters of the critic network. i v This represents the current network parameters of the critic network.

[0060] In this embodiment of the invention, initializing the population, actor-critic network parameters, and constructing the target network lays a solid foundation for the entire scheduling optimization process. Specifically, initializing the population generates an initial solution set, providing a high-quality starting point for subsequent solution space exploration and reducing ineffective searches; initializing the parameters of the actor-critic network (including the weights and biases of the hidden and output layers) enables the network to have initial policy generation and state value assessment capabilities, ensuring the rationality of decision-making and evaluation in the initial stage of reinforcement learning; and replicating the network parameters to construct the target network, and updating the target network parameters by setting a soft update coefficient, avoids drastic fluctuations during network parameter updates, ensuring a stable policy optimization direction and providing stable support for subsequent accurate exploration of the EEDAPSP-SDST solution space and improvement of search efficiency.

[0061] S4: Construct the state vectors of the individuals in the population based on the multi-objective scheduling objective function.

[0062] Among them, individuals β ( h ) 12-dimensional state vector Specifically, this includes: individual characteristics, sequence characteristics, and optimization progress.

[0063] Individual characteristics include: the normalized values ​​of the two target scheduling objective functions. and The dominant state of the current solution , respectively representing the age of the dominated, non-dominated, dominant, and normalized individuals. age norm .

[0064] The sequence features include: total preparation time percentage, average preparation time percentage, preparation time percentage at the end of the sequence, preparation time volatility, product type information entropy, and repeating product type rate.

[0065] The optimization progress includes: the number of steps the individual has taken in the evolutionary process. steps ratio and neighborhood operation acceptance rate accept ratio .

[0066] In this embodiment of the invention, constructing state vectors for individuals in the population based on a multi-objective scheduling objective function provides accurate "solution feature basis" for subsequent search optimization. On the one hand, relying on the quantification results of the objective function (focusing on maximum completion time and total energy consumption), the state vector's characterization of individuals (such as the normalized value of the objective function, dominant state, etc.) aligns with the core of optimization, avoiding the disconnect between features and objectives. On the other hand, the state vector encompasses multi-dimensional information such as individual characteristics, sequence characteristics (such as the proportion of total preparation time, product type information entropy), and optimization progress, comprehensively capturing the position and potential of individuals in the solution space. This provides clear guidance for subsequent actor networks to select neighborhood operations and identify high-potential solution regions based on state vectors, reducing blind exploration and fundamentally helping to improve the exploration efficiency of the EEDAPSP-SDST solution space.

[0067] S5: Through the actor network, based on the state vector, a neighborhood search is performed on the individual to form the individual's trajectory experience.

[0068] In one possible implementation, S5 specifically includes sub-steps S501 to S509: S501: Normalize the state vector.

[0069] S502: Input the normalized state vector into the input layer of the actor network to obtain the normalized state vector.

[0070] S503: The normalized state vector is linearly transformed through the hidden layer of the actor network to obtain the hidden layer output vector.

[0071] It should be noted that the normalized state vector is mapped through a hidden layer containing 64 neurons, applying the ReLU activation function. Through the weight matrix and bias terms Implement feature space transformation.

[0072] S504: By performing a linear transformation on the output vector of the hidden layer through the output layer of the actor network, the unnormalized score vector of each action is obtained.

[0073] It should be noted that the hidden layer output vector The data is input to the output layer and passed through the weight matrix. With bias term Obtain the unnormalized score vector for each action. s .

[0074] S505: Based on the unnormalized score vector, the action probability distribution is obtained through the Softmax activation function: ; in, Indicates the policy network in its current state. s gen Select action a gen The probability, s agen This indicates that the action is calculated by the actor network output layer. a gen The normalized score, This represents the index variable in the denominator used to iterate through all possible actions. This indicates the current state of the corresponding action. s gen The normalized score is as follows.

[0075] S506: Select the action with the highest probability in the action probability distribution and record the action number.

[0076] S507: Perform a neighborhood search on the individual corresponding to the action number to obtain a new solution.

[0077] The neighborhood search method is as follows: when a gen When =1, randomly swap two products. P h and Its position in the product sequence, where, h , These are all product numbers.

[0078] when a gen When =2, select two products in the product sequence. P h and ,if Then P h Insert into Before.

[0079] when a gen When =3, select two products in the product sequence. P h and ,if Then P h Insert into after.

[0080] when a gen When =4, select two products in the product sequence. P h and Then switch The order.

[0081] when a gen When the product value is 5, randomly swap two adjacent products. P h and Its position in the product series.

[0082] when a gen When the value is 6, two workpieces are randomly selected from a product. J i and And swap their positions.

[0083] when a gen When =7, randomly select a product from the product sequence. P h , put the sequence into P h The processing speed of all workpieces is uniformly adjusted to different levels.

[0084] when a gen If the value is not equal to 1-7, then the current individual will not be affected. β ( h To perform the operation.

[0085] S508: Compare the multi-objective scheduling objective function values ​​of the old and new solutions to update the position of the individual.

[0086] Specifically, if and Furthermore, at least one strict inequality holds, i.e., the new solution dominates the old solution, then the new solution is accepted; otherwise, the old solution is retained.

[0087] in, This indicates the maximum completion time of the new solution. This indicates the maximum completion time of the old solution. This represents the total energy consumption of the new solution. This represents the total energy consumption of the old solution.

[0088] Furthermore, based on the dominance relationship between the new and old solutions, the following reward is given: if the new solution dominates the old solution, then a reward is given. If the new solution and the old solution are not mutually exclusive, then a reward is given. If the new solution is dominated, then a reward is given. .

[0089] S509: Based on the results of neighborhood search, form the trajectory experience of an individual.

[0090] In this embodiment of the invention, by combining the actor network with state vectors to conduct neighborhood search and form trajectory experience, the accuracy and effectiveness of EEDAPSP-SDST solution space exploration can be improved from multiple dimensions: First, the state vectors are normalized to eliminate data bias and ensure input stability. Then, a feature space transformation is achieved through a hidden layer with 64 neurons (with ReLU activation function and specific weight matrix and bias term). Subsequently, a reasonable action probability distribution is generated through the output layer and Softmax activation function. The action with the highest probability is selected to perform the corresponding neighborhood operation (covering multiple dimensions such as product exchange, insertion, workpiece position adjustment, and processing speed adjustment). This can accurately focus on high-potential areas in the solution space and avoid blind exploration. At the same time, by comparing the maximum completion time and total energy consumption of new and old solutions to update individuals (only retaining cases where the new solution dominates or does not dominate each other), and giving differentiated rewards based on the dominance relationship, the quality of the solution can be ensured. Finally, the trajectory experience formed based on the search results provides high-quality data support for subsequent experience pool storage and actor-critic network updates. Overall, this not only improves the search targeting and efficiency, but also lays a reliable foundation for the continuous iteration of the optimization process.

[0091] S6: Store the individual's trajectory experience into the experience pool.

[0092] In one possible implementation, S6 specifically includes sub-steps S601 to S603: S601: Record the individual's age, encapsulate the trajectory experience into an experience object, and store it in a temporary buffer.

[0093] S602: Every fixed step SAVE_INTERVAL, the experience in the temporary buffer is stored into the global trajectory experience pool.

[0094] Furthermore, if all rewards in the temporary buffer are close to 0 or negative, an experience is randomly sampled and stored in the trajectory; otherwise, a probability distribution is constructed based on the reward size, and weighted sampling is performed to store high-reward experiences in the trajectory.

[0095] S603: After the experience is stored, clear the temporary buffer.

[0096] In this embodiment of the invention, a standardized trajectory experience storage process provides high-quality, high-value data support for subsequent algorithm iterations, with significant advantages: First, the individual's age is recorded, and the trajectory experience is encapsulated as an experience object and stored in a temporary buffer. This fully preserves key information such as the individual's state, actions, rewards, and subsequent states during the search process, ensuring the integrity of the experience. At fixed steps, the experience from the temporary buffer is stored in the global trajectory experience pool, avoiding efficiency losses caused by frequent reads and writes to the global pool and ensuring the stability and orderliness of experience storage. Simultaneously, the sampling strategy is flexibly adjusted based on rewards—if the temporary buffer contains only negative rewards or rewards close to zero, a single experience is randomly sampled to avoid missing high-quality experiences; otherwise, a probability distribution is constructed based on reward size to weight high-reward experiences, effectively filtering high-quality experiences and reducing resource consumption by low-quality experiences, ensuring the data quality of the global experience pool. Finally, the temporary buffer is cleared, releasing storage space promptly and maintaining the efficiency of data processing. Overall, this lays a reliable data foundation for subsequent sampling and updating of the actor-critic network from the experience pool, contributing to more accurate and efficient network optimization.

[0097] S7: Determine if the individual's age is greater than the preset age. If yes, proceed to the next step. Otherwise, return to step S5.

[0098] Those skilled in the art can set the preset age according to actual needs, and this invention does not limit it.

[0099] S8: Select the dominant solution from the population to form a dominant subpopulation.

[0100] In one possible implementation, S8 specifically includes sub-steps S801 to S807: S801: Copy the solution represented by each individual in the population to a temporary array.

[0101] S802: Perform non-dominated sorting on the solutions in the temporary array.

[0102] It should be noted that the sorting rule for non-dominated sorting is as follows: sort by non-dominated rank in ascending order: solutions with smaller rank indicate higher non-dominated rank and are retained first.

[0103] S803: Based on the results of the non-dominated sorting, the solution is divided into a first preset number of non-dominated layers: ; in, u temp ( g ) represents a temporary array. Ft l Indicates the first l The non-dominated solution set of a non-dominated front. , LThis represents the total number of non-dominated layers. Indicates the first l The first non-dominated solution set of the non-dominated front i One solution. , Indicates the first l The total number of solutions contained in the non-dominated solution set of a non-dominated front. ,untie and They do not control each other.

[0104] Those skilled in the art can set the size of the first preset quantity according to actual needs, and the present invention does not limit it.

[0105] S804: For solutions in the same non-dominated layer, calculate the normalized congestion distance of each solution relative to the maximum completion time target and the total energy consumption target: ; in, express α l ( i The normalized congestion distance relative to the maximum completion time target. C ml Indicates the first l The maximum completion time for all solutions of a non-dominated front. α l ( i +1) indicates the first l The first non-dominated solution set of the non-dominated front i +1 solution α l ( i -1) indicates the first l The first non-dominated solution set of the non-dominated front i -1 solutions, where max represents the maximum value and min represents the minimum value.

[0106] in, Sort in ascending order; Sort in ascending order.

[0107] Normalized crowding distance is a key metric in multi-objective optimization algorithms used to measure the "crowding" of individuals within a population. Its core function is to assess the "sparseness" of an individual's position in the objective space after eliminating differences in the dimensions of different objective dimensions. It first calculates the distance between an individual and its neighbors in each objective dimension (the raw value of the crowding distance). However, since the numerical ranges of different objective dimensions can vary greatly, direct calculation leads to the dominant distance result for the objective with the largest numerical range. Therefore, "normalization" is needed to scale the distance values ​​of each objective dimension to the same range (usually 0-1), ensuring a fair weighting of each objective's contribution to crowding. Ultimately, the larger the normalized distance value, the sparser the individual is in the objective space. The algorithm will prioritize retaining such individuals to maintain population diversity, avoid the optimization results being concentrated in local areas, and ensure a more comprehensive multi-objective optimal solution is found.

[0108] ; in, express α l ( i The normalized congestion distance relative to the total energy consumption target. C tl Indicates the first l The total energy consumption of all solutions for a non-dominated front.

[0109] S805: Sum the normalized congestion distances relative to the maximum completion time target and the total energy consumption target to obtain the total congestion distance for calculating each solution: ; in, CD ( α l ( i ))express α l ( i Total congestion distance.

[0110] S806: Sort the solutions in descending order based on the total crowding distance value.

[0111] S807: Based on the descending sorting results, select the second preset number of dominant solutions to obtain the dominant subpopulation.

[0112] Those skilled in the art can set the size of the second preset quantity according to actual needs, and the present invention does not limit this.

[0113] In this embodiment of the invention, a systematic process for selecting superior solutions forms a dominant subpopulation, which provides a core set of solutions with both high quality and diversity for subsequent scheduling optimization, with significant value: First, each solution in the population is copied to a temporary array. Then, by sorting in ascending order of non-dominated level, higher-level high-quality solutions are prioritized. Next, the hierarchical relationship of the non-dominated layers is clearly defined based on the sorting results. For solutions in the same non-dominated layer, their normalized congestion distance relative to the maximum completion time and total energy consumption targets is calculated and summed to obtain the total congestion distance. This effectively balances the spatial distribution diversity of solutions under the two optimization objectives, avoiding homogenization of solutions. Finally, a preset number of solutions are selected in descending order of total congestion distance to form a dominant subpopulation. This ensures that the subpopulation focuses on high-quality non-dominated solutions, guarantees high solution quality, and ensures solution diversity (covering high-quality areas with different target inclinations) through congestion distance. This provides an efficient starting point for subsequent in-depth optimization by using these solutions as initial solutions to approach the target solution, significantly reducing subsequent invalid searches and effectively improving the exploration efficiency and overall optimization effect of the EEDAPSP-SDST solution space.

[0114] S9: Select the target solution from the current best external files, take each solution in the dominant subpopulation as the starting solution, and gradually move the starting solution toward the target solution through swapping or insertion operations to update the dominant subpopulation.

[0115] In one possible implementation, S9 specifically includes sub-steps S901 to S904: S901: Select the target solution from the current best external archives, and construct the path between the target solution and each solution in the dominant subpopulation as the starting solution.

[0116] S902: Perform a swap or insertion operation on the path.

[0117] S903: In a swap or insertion operation, if the solution obtained by the operation has a smaller maximum completion time, then the solution is used as the new starting point, and the search continues on the updated path.

[0118] S904: Divide the energy-intensive workpiece subsequence into multiple workpiece segments. Within each segment, attempt to adjust the workpiece order by insertion and calculate the total energy consumption after insertion. If the energy consumption decreases, update the current solution and continue searching. Otherwise, proceed to the next segment. After completing the local optimization of all segments, obtain the final optimized workpiece order and update the dominant subpopulation.

[0119] In this embodiment of the invention, precise target setting and hierarchical optimization can significantly improve the quality of the dominant subpopulation and deeply explore the characteristics of the solution space. The target solution is selected from the current best external archive, ensuring the quality of the optimization direction. Then, a path is constructed using the high-quality solutions of the dominant subpopulation as the starting solution, avoiding blind exploration. Through swapping or insertion operations, the starting solution gradually moves closer to the target solution. If a solution with a smaller maximum completion time is obtained during the operation, the starting point is updated, allowing for precise focus on the more efficient search direction. Simultaneously, the energy-intensive workpiece subsequence is divided into multiple workpiece segments. Within each segment, the order is adjusted and the total energy consumption is calculated. Only updates that reduce energy consumption are retained, enabling targeted optimization of energy consumption targets and achieving synergy between efficiency and energy saving. Finally, the dominant subpopulation is updated, further improving the quality of the subpopulation solutions and deeply developing the characteristics of efficiency and energy consumption synergistic optimization in the EEDAPSP-SDST solution space, laying a better solution foundation for subsequent steps and effectively improving the overall optimization effect.

[0120] S10: For the solutions in the updated dominant subpopulation, implement a Q-learning-based energy-saving strategy.

[0121] Q-learning is a classic model-free temporal difference algorithm in the field of reinforcement learning. Its core is to allow the agent to learn a key model called the "state-action value function" through "trial and error interaction" with the environment. This function measures the value of an action in a given state. s Take action below a Then, the total cumulative reward that the intelligent agent may receive in the future.

[0122] Specifically, for the solution execution energy-saving strategy in the updated dominant subpopulation, the Q-learning framework is introduced to unify the workpiece scheduling postponement, machine speed adjustment and machine shutdown operations into a comprehensive energy-saving strategy.

[0123] In one possible implementation, S10 specifically includes sub-steps S1001 to S1003: S1001: Construct the state vector of the agent. The state vector of the agent includes the latest end time, the earliest assembly time of the product to which it belongs, the current speed of the machine, the idle duration, and the number of start and stop times.

[0124] S1002: According to e - Greedy strategy, choose one of the following actions: postpone processing, speed up processing, slow down processing, shut down the machine, or keep it idle.

[0125] in, e Greedy strategies are a classic approach in reinforcement learning that balances exploration and exploitation: among which... e It is a decimal number between 0 and 1 (e.g., 0.1). The policy specifies that when the agent selects an action, it uses a number between 1 and 1. eThe probability of "utilizing" known information—that is, choosing the action with the highest current estimated value (such as Q value) to ensure stable short-term returns; at the same time, e The agent uses probability to "explore" unknown possibilities—randomly selecting any action, even if it doesn't currently appear optimal, to avoid missing potentially better strategies by sticking to known information. This mechanism ensures the effective use of existing experience while leaving room for discovering new and better actions, helping the agent gradually approach the globally optimal strategy through long-term learning.

[0126] S1003: After performing the action, a reward is obtained and the Q table is updated. The reward is as follows: if the total energy consumption is reduced after the operation, a positive reward of 5 is given; otherwise, a negative reward of -5 is given.

[0127] In this embodiment of the invention, a comprehensive energy-saving strategy based on Q-learning is applied to the updated dominant subpopulation solution, which can effectively make up for the shortcomings of traditional single energy-saving methods and significantly improve the flexibility and accuracy of energy-saving optimization. By constructing a 5-dimensional state vector containing the latest end time, the earliest assembly time of the product, the current machine speed, the idle duration, and the number of start-stops, the characteristics of the actual production scenario can be fully captured, providing a realistic and accurate basis for energy-saving decisions. The workpiece scheduling postponement, machine speed adjustment, and shutdown operations are uniformly incorporated into the strategy, breaking the limitation of relying solely on machine speed adjustment, and can adapt to the working conditions of different equipment loads and order delivery dates in the engine manufacturing workshop. By exploring and utilizing the ε-greedy strategy to balance actions, more potential energy-saving solutions can be discovered. Furthermore, the Q-table is updated with a mechanism of "a positive reward of 5 for reducing total energy consumption and a negative reward of -5 for not reducing it", ensuring that the strategy is continuously optimized in the direction of energy saving. Finally, under the premise of ensuring scheduling efficiency (such as controllable maximum completion time), the total energy consumption is deeply reduced, accurately matching the multi-objective optimization requirements of EEDAPSP-SDST.

[0128] S11: Compare the objective values ​​of the solutions in the updated dominant subpopulation with the objective values ​​of the solutions in the Pareto optimal file, store the new non-dominated solutions in the updated dominant subpopulation into the optimal Pareto file, and remove the dominated solutions from the Pareto file.

[0129] The Pareto optimal archive is the core output of a multi-objective optimization algorithm. Essentially, it's a collection storing all Pareto optimal solutions (also known as non-dominated solutions), which together form the Pareto optimal front in the objective space. Each solution satisfies the "non-dominated property": no other solution is superior to it in all optimization objectives (such as cost, efficiency, and reliability). For example, when optimizing "low cost" and "high reliability," solutions A (cost 2, reliability 90) and B (cost 3, reliability 95) are non-dominated; therefore, A is more cost-efficient, and B is more reliable, and both will be included in the archive. Simultaneously, the archive uses indicators such as crowding distance to eliminate overly dense redundant solutions, ensuring a uniform distribution of solutions in the objective space. Ultimately, it doesn't provide a single "best solution," but rather a set of optimal solutions covering different objective trade-offs, allowing decision-makers to choose the appropriate solution based on their actual needs (such as prioritizing cost or reliability).

[0130] In this embodiment of the invention, by comparing the target values ​​of the updated dominant subpopulation solution with the Pareto optimal file solution, the high quality and effectiveness of the Pareto optimal file can be continuously optimized and maintained: storing new non-dominated solutions into the file can continuously supplement the high-quality new solutions generated during the optimization process, ensuring that the file always contains the current best multi-objective solution (taking into account both maximum completion time and total energy consumption), and avoiding the omission of potential high-quality solutions; at the same time, removing dominated solutions from the file can promptly eliminate inefficient and redundant solutions, reduce the file storage burden, and ensure the "conciseness" and "efficiency" of the file.

[0131] S12: A reward backtracking mechanism is adopted to add backtracking rewards to the solution sequence that makes a positive contribution in the Pareto non-dominated solution.

[0132] Among them, the reward backtracking mechanism is the core logic of reinforcement learning that allows agents to learn from "delayed rewards". It refers to the mechanism that when an agent receives a reward (or punishment) after a series of consecutive actions, it "backtracks" the result to the previous relevant state and action according to certain rules, so that early decisions can also receive valuable feedback.

[0133] Optionally, the backtracking reward is as follows: ; in, R extra Indicates retrospective reward. r final This indicates a reward for positive contributions. c Indicates the discount factor. Indicates the time interval step.

[0134] It should be noted that if the solution sequence makes a positive contribution, then r final =20.

[0135] S13: Randomly sample multiple trajectory experiences from the experience pool to update the network parameters of the actor-critic network.

[0136] In one possible implementation, S13 specifically includes sub-steps S1301 to S1303: S1301: Uniformly and randomly sample multiple trajectory experiences from the experience pool, and calculate the temporal difference objective of the critic network: ; in, y g Indicates the first g The temporal difference target value of the next iteration. r g Indicates the first g In the next iteration, the state vector s g Next, execute the action number. a g The reward received c Indicates the discount factor. s g+1 Indicates the first g The state vector obtained after +1 iterations V target ( s g+1 ) indicates with s g+1 The critic network calculates the state value as input.

[0137] S1302: Based on the temporal difference objective, update the network parameters of the critic network by minimizing the mean squared error loss function.

[0138] The temporal difference objective is the core reference standard used in temporal difference (TD) learning to update the value function. Essentially, it combines the immediate reward received with an estimate of the future reward in the next state. Specifically, when evaluating the value of a state, it's the current reward plus a coefficient (discount factor) balancing immediate and future rewards multiplied by the estimated value of the next state. When evaluating the value of a specific action in a state, it's typically the current reward plus this coefficient multiplied by the estimated value of the optimal action in the next state. It allows the agent to adjust its judgment of the value of states or actions in real time using a combination of current feedback and expected next steps, without waiting for the entire decision sequence to end (e.g., completing a game or task). Through this temporal value difference, it gradually optimizes its decision-making and cognition.

[0139] The mean squared error (MSE) loss function is a commonly used metric in machine learning to measure the predictive performance of a model, primarily used for regression problems (predicting continuous values ​​such as house prices and temperature). Its core logic is as follows: first, calculate the difference (error) between the model's predicted value and the actual value for each sample; then, square these errors individually (to avoid positive and negative errors canceling each other out and amplifying larger errors); finally, calculate the average of all squared errors. This average is the MSE; the smaller the value, the closer the model's prediction is to the actual situation. The goal of model training is to continuously reduce this value by adjusting parameters, thus making the prediction more accurate.

[0140] S1303: Update the network parameters of the actor network by minimizing the mean squared error loss function: ; in, i π This represents the current network parameters of the actor network. express i π Update volume α π Indicates the learning rate. The loss function is expressed as follows: i π gradient, Represents a given state vector s g Take action number below a g The action probability vector, A ( s g , a g ) represents the state vector s g Select action number below a g The returns that come with x Represents the entropy regularization coefficient. H ( π θ ( s g )) represents a given state vector s g Below, by policy network π θ The probability distribution of the output actions.

[0141] In this embodiment of the invention, updating the actor-critic network parameters by uniformly and randomly sampling trajectory experiences from the experience pool can significantly improve the network's learning performance and policy optimization capabilities. Specifically, the advantages are as follows: uniform random sampling breaks the temporal correlation between experiences, preventing network learning from being affected by data bias and ensuring training stability; calculating the temporal difference objective of the critic network (integrating immediate reward and future state value) allows for more accurate estimation of state value, and updating parameters by minimizing the mean squared error loss allows the critic network to more reliably evaluate action value, providing accurate optimization basis for the actor network; when updating the actor network, the policy gradient term guides the network to adjust towards high-reward actions to optimize the policy, while the entropy regularization term encourages diversity in action probability distributions, avoiding premature convergence to local optima and balancing exploration and utilization; the overall mechanism enables the actor-critic network to continuously iterate and upgrade based on historical experience, improving its ability to capture EEDAPSP-SDST solution space features and the quality of policy generation, providing more accurate directional guidance for subsequent neighborhood searches, and fundamentally enhancing the efficiency and effectiveness of scheduling optimization.

[0142] S14: Determine if the execution time has reached the preset execution time. If yes, output the final Pareto optimal file. Otherwise, return to step S4.

[0143] It should be noted that the determination is made based on whether the current algorithm's execution time is greater than the algorithm's maximum execution time. Q × N ×60 ms If the condition is met, the iteration ends, and the final Pareto optimal file is output; otherwise, the iteration ends. g = g +1, jump to step S4.

[0144] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, initializing the population reduces ineffective searches; performing neighborhood searches on individuals using the actor network and updating the actor-critic network enables targeted search paths, significantly improving search efficiency; simultaneously, updating the dominant subpopulation fully utilizes the solution space features of EEDAPSP-SDST. Furthermore, using the actor network instead of the traditional Q-learning method avoids convergence bias caused by excessive greed; and employing a reward backtracking mechanism prevents the algorithm from getting stuck in local plateaus, enhancing global search capabilities.

[0145] Reference manual attached Figure 5 The diagram shows a structural schematic of a multi-objective scheduling system for an automobile engine manufacturing workshop provided by an embodiment of the present invention.

[0146] This invention provides a multi-objective scheduling system 20 for an automobile engine manufacturing workshop, comprising: a processor 201 and a memory 202; The memory 202 stores programs or instructions that can run on the processor 201. When the program or instructions are executed by the processor 201, they implement the steps of the above-described multi-objective scheduling method for automobile engine manufacturing workshop and achieve the same technical effect. To avoid repetition, the present invention will not elaborate further.

[0147] It should be understood that the processor 201 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0148] It should also be understood that the memory 202 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM).

[0149] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0150] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0151] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0152] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0153] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0154] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0156] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] This invention provides a readable storage medium comprising: storing a program or instructions on the readable storage medium, wherein when the program or instructions are executed by a processor, the program or instructions implement the steps of the above-described multi-objective scheduling method for an automotive engine manufacturing workshop and achieve the same technical effect. To avoid repetition, this invention will not elaborate further.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-objective scheduling method for an automobile engine manufacturing workshop, characterized in that, The method comprises the following steps: S1: obtaining process parameters and technical specifications of an automobile engine manufacturing workshop; S2: constructing a multi-objective scheduling objective function according to the process parameters and technical specifications; S3: initializing a population and an actor-critic network; S4: constructing a state vector of an individual contained in the population according to the multi-objective scheduling objective function; S5: performing neighborhood search on the individual according to the state vector through the actor network to form trajectory experience of the individual; S6: storing the trajectory experience of the individual into an experience pool; S7: judging whether the age of the individual is greater than a preset age; if yes, proceeding to the next step; otherwise, returning to step S5; S8: selecting a dominant solution in the population to form a dominant subpopulation; S9: determining a target solution from a current external optimal archive, taking each solution in the dominant subpopulation as a starting solution, gradually moving the starting solution to the target solution through swapping or insertion operations, and updating the dominant subpopulation; S10: executing an energy-saving strategy based on Q learning on the solutions in the updated dominant subpopulation; S11: comparing the target values of the solutions in the updated dominant subpopulation with the target values of the solutions in a Pareto optimal archive, storing new non-dominated solutions in the updated dominant subpopulation into the optimal Pareto archive, and removing dominated solutions in the Pareto archive; S12: using a reward backtracking mechanism to add backtracking rewards to a solution sequence that makes a positive contribution to the Pareto non-dominated solutions; S13: uniformly and randomly sampling multiple pieces of trajectory experience from the experience pool to update network parameters of the actor-critic network; S14: judging whether the execution time reaches a preset execution time; if yes, outputting a final Pareto optimal archive; otherwise, returning to step S4.

2. The multi-objective scheduling method for an automobile engine manufacturing plant according to claim 1, wherein, The S2 specifically comprises: S201: setting a scheduling scheme according to the process parameters and technical specifications; S202: determining workpiece completion times according to the scheduling scheme: ； in, Indicates the first f The first factory i The number of each processed workpiece. express In the machine M j The completion time is shown above, and max indicates the maximum value. express In the machine M j-1 The completion time on the website Indicates the first f The first factory i -1 workpiece number, express In the machine M j The completion time on the website Indicates in the machine M j Above, at back, Preparation time beforehand Indicates in the machine M j superior Processing time; S203: determining processing completion times of the last workpiece on the last machine according to the workpiece completion times: ； wherein, β h denotes the number of the h assembled product, denotes β h the completion time of the last workpiece of the Ω h denotes the set of all workpieces to which the h assembled product belongs; S204: determining assembly completion times of products according to the processing completion times: ； wherein, denotes β h the assembly completion time of the product, β h-1 denotes the number of the h -1 assembly product, denotes β h-1 the assembly completion time of the product, denotes the preparation time before M A on the assembly machine β h-1 after, β h before, denotes the processing time on the assembly machine M A β h ;​ S205: calculating a maximum completion time of the scheduling scheme according to the assembly completion times: ； wherein, Γ denotes a dispatching scheme, C max ( Γ ) denotes a maximum completion time of a dispatching scheme; S206: calculating total energy consumption of the scheduling scheme: ； wherein, TEC Γ represents the total energy consumption of the scheduling scheme, E P Γ represents the total energy consumption of the scheduling scheme, E I Γ represents the total energy consumption of the scheduling scheme, E S Γ represents the total energy consumption of the scheduling scheme;​​​​ S207: constructing the multi-objective scheduling objective function with the maximum completion time and the total energy consumption as targets: ； wherein min represents taking a minimum value.

3. The multi-objective scheduling method for an automobile engine manufacturing plant according to claim 1, wherein, The S3 specifically comprises: S301: initializing the population; S302: initializing network parameters of the actor-critic network; S303: copying actor and critic network parameters respectively to construct corresponding target networks, setting a soft update coefficient, and performing target network parameter updating: ； wherein, denote target network parameters of the actor network, τ denote soft update coefficients, θ π denote current network parameters of the actor network, denote target network parameters of the critic network, θ v denote current network parameters of the critic network.

4. The multi-objective scheduling method for an automobile engine manufacturing plant according to claim 1, wherein, The S5 specifically comprises: S501: performing normalization processing on the state vector; S502: inputting the normalized state vector into an input layer of the actor network to obtain a normalized state vector; S503: performing linear transformation on the normalized state vector through a hidden layer of the actor network to obtain a hidden layer output vector; S504: performing linear transformation on the hidden layer output vector through an output layer of the actor network to obtain an unnormalized score vector of each action; S505: obtaining an action probability distribution through a Softmax activation function according to the unnormalized score vector: ； wherein, denotes the policy network computes the probability of taking action s gen at the current state a gen S506: selecting an action with the highest probability in the action probability distribution and recording an action number; agen denotes the normalized score of action a gen output by the actor network output layer, denotes an index variable in the denominator for traversing all possible actions, denotes the normalized score of the corresponding action at the current state s gen .​ S507: performing neighborhood search on the individual corresponding to the action number to obtain a new solution; S508: updating a position of the individual by comparing multi-objective scheduling objective function values of an old solution and the new solution; S509: forming trajectory experience of the individual based on a result of the neighborhood search. The S8 specifically includes:

5. The multi-objective scheduling method for an automobile engine manufacturing plant according to claim 1, wherein, S801: copying solutions represented by each individual of the population into a temporary array; S802: performing non-dominated sorting on each solution in the temporary array; S803: dividing the solutions into a first preset number of non-dominated layers based on a result of the non-dominated sorting: S804: calculating normalized crowding distance of each solution with respect to a maximum completion time objective and a total energy consumption objective respectively for solutions in the same non-dominated layer: ； wherein, u temp ( g ) denotes a temporary array, S805: adding the normalized crowding distance with respect to the maximum completion time objective and the total energy consumption objective to obtain total crowding distance of each solution: l denotes the non-dominated solution set of the l th non-dominated front, , L denotes the total number of non-dominated layers, denotes the non-dominated solution set of the l th non-dominated front, i th solution in the non-dominated solution set of the th non-dominated front, , l denotes the total number of solutions contained in the non-dominated solution set of the th non-dominated front, solutions are not dominated by each other; S806: performing descending sorting on each solution according to a value of the total crowding distance; ； wherein, denotes α l ( i ) normalized crowding distance with respect to the maximum makespan target, C ml denotes the maximum makespan value of the l th non-dominated front, α l ( i +1) denotes the l th solution in the non-dominated solution set of the i +1th non-dominated front, α l ( i -1) denotes the l -1th solution in the non-dominated solution set of the i th non-dominated front, max denotes taking the maximum value, and min denotes taking the minimum value. ； wherein, denotes α l ( i ) normalized crowding distance with respect to the total energy consumption target, C tl denotes the total energy consumption value of the non-dominated front of the first l non-dominated front. S807: selecting a second preset number of the dominant solutions based on a result of the descending sorting to obtain the dominant sub-population. ； wherein The S9 specifically includes: α l i represents α l i total crowdedness distance​​​ S901: determining the target solution from the current external optimal archive, taking each solution in the dominant sub-population as a starting solution, and constructing a path between the starting solution and the target solution; S902: performing exchange or insertion operation on the path; 6. The multi-objective scheduling method for an automobile engine manufacturing plant according to Claim 1, wherein S903: in the exchange or insertion operation, if a solution obtained by the operation has smaller maximum completion time, the solution is taken as a new starting point, and search is continued on the updated path; S904: dividing an energy-dominant workpiece sub-sequence into multiple workpiece segments, trying to adjust workpiece order in each segment by insertion, calculating total energy consumption after insertion, and if energy consumption is reduced, updating the current solution and continuing search, otherwise, turning to processing of the next segment, and after completing local optimization of all segments, obtaining a final optimized workpiece order, and updating the dominant sub-population. The S10 specifically includes: S1001: constructing a state vector of an agent, the state vector of the agent including a latest end time, an earliest assembly time of a product to which the agent belongs, a current speed of a machine, an idle duration, and a start-stop number; S1003: obtaining a reward after performing an action, and updating a Q table, the reward obtained being as follows: if total energy consumption is reduced after operation, a positive reward 5 is given, otherwise, a negative reward -5 is given.

7. The multi-objective scheduling method for an automobile engine manufacturing plant according to Claim 1, wherein The backtracking reward is specifically: The S13 specifically includes: S1002: According to ​ - a greedy strategy, selecting one of the following actions: postponing the machining, accelerating the machining, slowing down the machining, machine shutdown, and keeping idle; ​ 8. The multi-objective scheduling method for an automobile engine manufacturing plant according to Claim 1, wherein, ​ ； wherein, R extra denotes a backward reward, r final denotes a forward contribution reward, ​ denotes a discount factor, denotes an interval time step.

9. The multi-objective scheduling method for an automobile engine manufacturing plant according to Claim 1, wherein, ​ S1301: uniformly and randomly sampling a plurality of trajectory experiences from the experience pool, and calculating a time difference target of the critic network: ； wherein, y g denotes the timing difference target value for the g th iteration, r g denotes the timing difference target value for the g th iteration in the state vector s g the action number a g the reward obtained, γ denotes the discount factor, s g+1 denotes the state vector obtained for the g + 1th iteration, V target s g+1 denotes the state value computed by the critic network with s g+1 as input;​ S1302: updating network parameters of the critic network by minimizing a mean square error loss function according to the time difference target; S1303: updating network parameters of the actor network by minimizing a mean square error loss function: ； wherein θ π denotes the current network parameters of the actor network, denotes θ π the update quantity of α π denotes the learning rate, denotes the gradient of the loss function with respect to θ π denotes the gradient of the loss function with respect to s g denotes the action probability vector for taking action number a g in a given state vector A s g denotes the reward for selecting action number a g in state vector s g a g χ denotes the entropy regularization coefficient, H π θ s g denotes the action probability distribution output by the policy network s g for a given state vector π θ .​​​​​​ 10. A multi-objective scheduling system for an automotive engine manufacturing plant, characterized by, comprise: a processor and a memory; the memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement the steps of the multi-objective scheduling method for an automobile engine manufacturing workshop according to any one of claims 1 to 9.

Images