Dynamic decision-making method, device and equipment for hot working workshop of aerospace products

Abstract

The invention discloses a dynamic decision-making method, device and equipment for a hot working workshop of aerospace products, and belongs to the technical field of hot working production. The method comprises the following steps: acquiring first production data in the process of producing a target workpiece in a hot processing workshop; abstracting the first production data in the state space to obtain first workshop state information of the hot working workshop; the first workshop state information is input to an intelligent decision-making model, first control action information of an action space output by the intelligent decision-making model is obtained, and the intelligent decision-making model is constructed based on a neural network model; and according to the first control action information, generating a production decision scheme for producing the target workpiece in the hot processing workshop. According to the method, the workshop multi-mode state can be dynamically identified, and efficient decision making is realized.

Description

Technical Field

[0001] The present application belongs to the field of thermal processing production technology, and in particular, relates to a dynamic decision-making method, device and equipment for a thermal processing workshop of aerospace products. Background Art

[0002] The production mode of aerospace hot processing products (such as aircraft main landing gear, rocket engine heads, etc.) is mainly single-piece small-batch production, which is characterized by long processes, slow cycles, high energy consumption, and extremely high quality requirements. The hot processing workshop is the main carrier of the production process of aerospace hot processing products, and its control method directly determines the quality and efficiency of the production process of hot processing products.

[0003] At present, the control of hot working workshops for aerospace products is mainly based on manual experience, such as manual scheduling, process design, manual troubleshooting, etc., which are mainly low-efficiency manual control. There are many disturbance factors in the quality and energy consumption of the hot working production process. When dynamic events such as emergency order insertion and machine failure occur in the hot working workshop, it is difficult to make efficient and intelligent collaborative scheduling decisions for the quality and energy efficiency of the hot working workshop. Automated control is the development trend of hot working workshop control, but the technology of integrated control systems in aerospace hot working workshops is currently developing slowly and has single functions. It only has data collection, transmission, visualization functions, and a small amount of scheduling optimization functions using genetic algorithms. Integrating advanced intelligent optimization algorithms into the control system of aerospace hot working workshops is the underlying key technology to change inefficient manual control to efficient intelligent control.

[0004] However, the current mainstream intelligent optimization algorithms are usually meta-heuristic methods (such as evolution-based algorithms, swarm intelligence-based algorithms, and physical process-based algorithms, etc.). Most of these methods are static methods. Due to their fixed search strategies, large number of iteration requirements, static fitness functions, lack of real-time data fusion, and fixed parameter settings, they are difficult to dynamically identify the multi-modal state of the workshop and make efficient decisions. Therefore, how to improve the dynamic perception and real-time efficient decision-making capabilities of decision-making algorithms has become a key problem that needs to be solved in the intelligent control system technology of the production process of aerospace product hot processing workshops. Summary of the invention

[0005] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a dynamic decision-making method, device and equipment for aerospace product hot processing workshop, which can dynamically identify the multi-modal state of the workshop and achieve efficient decision-making.

[0006] In a first aspect, the present application provides a dynamic decision-making method for an aerospace product hot processing workshop, the method comprising:

[0007] Acquire first production data during the process of producing a target workpiece in a hot working workshop;

[0008] Abstracting the first production data in the state space to obtain first workshop state information of the hot processing workshop;

[0009] Inputting the first workshop state information into an intelligent decision-making model to obtain first control action information of an action space output by the intelligent decision-making model, wherein the intelligent decision-making model is constructed based on a neural network model;

[0010] A production decision plan for the hot working workshop to produce the target workpiece is generated according to the first control action information.

[0011] According to one embodiment of the present application, the first workshop status information in the status space includes product processing quality, order status information, machine utilization rate and overall machine energy consumption average value of the target workpiece during the production process.

[0012] According to one embodiment of the present application, the expression of the product processing quality is:

[0013]

[0014] Wherein, Q(t) represents the processing quality of the product; α, β, γ, δ, ζ are the weights of the state parameters that determine the quality; T represents the state parameter corresponding to the forging temperature; P represents the state parameter corresponding to the forging pressure; S micro-structure Represents the state parameter corresponding to the microstructure; represents the state parameter corresponding to the strain; G represents the state parameter corresponding to the geometric dimension.

[0015] According to one embodiment of the present application, the order status information is characterized as a collection of current orders and urgent orders of the target workpiece:

[0016] W={(p i ,d i ,w i )|i=1,2,3…,n}∪{(p E ,d E ,w E )}

[0017] Where W represents the set of current orders and urgent orders; i is the order number, p i represents the processing time of order i; d i represents the delivery deadline of order i; ω i represents the weight of order i; p E Indicates the processing time of urgent orders; d E Indicates the delivery deadline for urgent orders; E represents the weight of urgent orders; ω E Greater than ωi .

[0018] According to one embodiment of the present application, the neural network model includes an online Q network and a target Q network, the online Q network is used to predict and output the first control action information based on the first workshop status information, and the online Q network is used to update the parameters of the target Q network.

[0019] According to an embodiment of the present application, inputting the first workshop state information into an intelligent decision model to obtain first control action information of an action space output by the intelligent decision model includes:

[0020] Inputting the first workshop status information into the online Q network, performing value evaluation on a plurality of feasible actions corresponding to the first workshop status information through the online Q network, and determining the action combination with the highest Q value among the plurality of feasible actions as the first control action information;

[0021] The first control action information is valued by using the target Q network to obtain a target Q value of the first control action information, and the target Q value is used to update the online Q network.

[0022] According to an embodiment of the present application, after generating a production decision plan for the hot working workshop to produce the target workpiece according to the first control action information, the method further includes:

[0023] When the hot processing workshop executes the production decision plan, obtaining second production data corresponding to the production decision plan;

[0024] Abstracting the second production data in the state space to obtain second workshop state information of the hot processing workshop;

[0025] taking the first workshop state information, the first control action information, a combination of the reward value corresponding to the first control action information and the second workshop state information as a training sample, and storing the training sample in an experience replay pool;

[0026] The training samples are extracted from the experience replay pool to update the parameters of the intelligent decision-making model.

[0027] According to one embodiment of the present application, the reward value is obtained based on a reward function, and the reward function R is:

[0028] R=ω1·R geo +ω2·R mech +ω3·R def +ω4·R eff -P viol

[0029] Among them, R geo represents the size and shape deviation reward value of the target workpiece, R mech represents the mechanical property deviation reward value of the target workpiece, R def represents the reward for defect detection of the target workpiece, R eff represents the reward for the production efficiency of the target workpiece, R viol It indicates that a negative reward is given to the action in the first control action information that may cause quality problems, and ω1, ω2, ω3, and ω4 are weight parameters that characterize relative importance.

[0030] In a second aspect, the present application provides a dynamic decision-making device for a hot working shop of an aerospace product, the device comprising:

[0031] An acquisition module, used for acquiring first production data in a process of producing a target workpiece in a hot working workshop;

[0032] A first processing module, used for abstracting the first production data in a state space to obtain first workshop state information of the hot processing workshop;

[0033] A second processing module is used to input the first workshop state information into an intelligent decision-making model to obtain first control action information of an action space output by the intelligent decision-making model, wherein the intelligent decision-making model is constructed based on a neural network model;

[0034] The third processing module is used to generate a production decision plan for the hot processing workshop to produce the target workpiece according to the first control action information.

[0035] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the dynamic decision-making method for a hot processing workshop of aerospace products as described in the first aspect above is implemented.

[0036] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dynamic decision-making method for aerospace product hot working workshop as described in the first aspect above.

[0037] In a fifth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the dynamic decision-making method for an aerospace product hot processing workshop as described in the first aspect.

[0038] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the dynamic decision-making method for aerospace product hot processing workshop as described in the first aspect above.

[0039] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application.

[0040] The present invention provides a dynamic decision-making method, device and equipment for aerospace product hot processing workshop, which has the following beneficial effects compared with the prior art:

[0041] (1) By abstracting the first production data monitored in real time, the first workshop status information is obtained as the input of the intelligent decision-making model, and a decision-making production plan for dynamically adjusting production parameters is obtained. This can ensure that key parameters such as temperature, pressure, stress and strain in the production process are always kept within the optimal range. In the production process of the hot processing workshop of aerospace products, the multimodal state of the workshop can be dynamically identified to achieve efficient decision-making.

[0042] (2) It can respond quickly to dynamic events in the hot processing workshop of aerospace products, perceive the workshop status in real time, make intelligent decisions on workshop actions based on the status characteristics, and make real-time dynamic feedback adjustments based on the workshop actions after the decision, so as to ensure the processing quality and improve production efficiency. It has practical application value for the development of production process control system for hot processing workshops of aerospace products, and can realize efficient decision-making and intelligent control. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0044] Figure 1 It is one of the flow diagrams of the dynamic decision-making method for aerospace product hot processing workshop provided in the embodiment of the present application;

[0045] Figure 2 It is a schematic diagram of the mapping relationship between the state space and the action space provided in the embodiment of the present application;

[0046] Figure 3 This is the second flow chart of the dynamic decision-making method for aerospace product hot processing workshop provided in the embodiment of the present application;

[0047] Figure 4 It is a structural schematic diagram of a dynamic decision-making device for aerospace product hot processing workshop provided in an embodiment of the present application;

[0048] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0050] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0051] In recent years, in the field of dynamic workshop scheduling, deep reinforcement learning methods based on the fusion of deep learning and reinforcement learning have been proven to be able to solve real-time dynamic scheduling problems in job shops, flexible job shops, flow shops, etc. However, key factors such as the multimodal state of aerospace thermal processing, complex production processes, strict quality indicators, and energy consumption indicators have posed challenges to the application of deep reinforcement learning in scheduling decisions for aerospace thermal processing workshops. It is urgent to establish a general scheduling decision model and method suitable for the field of thermal processing of aerospace products.

[0052] In combination with the accompanying drawings, the dynamic decision-making method for an aerospace product hot processing workshop, the dynamic decision-making device for an aerospace product hot processing workshop, the electronic device and the readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0053] Among them, the dynamic decision-making method of the aerospace product hot processing workshop can be applied to the terminal, and can be specifically executed by the hardware or software in the terminal.

[0054] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0055] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0056] The embodiment of the present application provides a dynamic decision-making method for a hot processing workshop of aerospace products. The execution subject of the dynamic decision-making method for a hot processing workshop of aerospace products can be an electronic device or a functional module or functional entity in an electronic device that can implement the dynamic decision-making method for a hot processing workshop of aerospace products. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The dynamic decision-making method for a hot processing workshop of aerospace products provided in the embodiment of the present application is explained below using electronic devices as the execution subject as an example.

[0057] like Figure 1 As shown, the dynamic decision-making method of the hot processing workshop of aerospace products includes:

[0058] Step 110, obtaining first production data in a process of producing a target workpiece in a hot working workshop;

[0059] Among them, the hot processing workshop is a production area where metals or other materials are subjected to hot processing and is used to produce aerospace products.

[0060] The target workpiece may be an aerospace product.

[0061] The first production data may include output data, equipment utilization, energy consumption data, quality data, etc., which can reflect the production efficiency, resource utilization, and product quality of the hot processing workshop.

[0062] Step 120, abstracting the first production data in a state space to obtain first workshop state information s of the hot processing workshop;

[0063] Among them, the first workshop state information s in the state space may include temperature, pressure, stress strain, geometric dimensions, microstructure, order queue, emergency orders, machine utilization and overall machine energy consumption average value in the production process of aerospace hot processing products.

[0064] In this step, the first production data is preprocessed by data cleaning and standardization, and key indicators such as production efficiency, equipment utilization, and qualified rate are calculated for the preprocessed first production data to obtain the first workshop status information s.

[0065] Step 130: input the first workshop state information s into an intelligent decision-making model to obtain first control action information a in the action space output by the intelligent decision-making model, wherein the intelligent decision-making model is constructed based on a neural network model;

[0066] Among them, the first control action information a is designed based on the action space, and may include: forging temperature adjustment, forging pressure adjustment, forging speed control, product heat treatment parameter adjustment, insertion of emergency orders, adjustment of production sequence, balanced allocation of machine tasks, and balanced allocation of high and low power machine tasks.

[0067] like Figure 2 As shown, there is a mapping relationship between the workshop state information in the state space and the control action information in the action space, and the control action information can affect the corresponding workshop state information.

[0068] The shop status information includes: the production temperature, pressure, stress and strain, geometric dimensions, microstructure, current order queue, priority and arrival time of urgent orders, average machine utilization and overall machine energy consumption average of aerospace hot processing products;

[0069] The control action information includes: forging temperature adjustment, forging pressure adjustment, forging speed control, product heat treatment parameter adjustment, insertion of emergency orders, adjustment of production sequence, balanced distribution of machine tasks, and balanced distribution of high and low power machine tasks.

[0070] In this step, the first workshop status information s is input into the intelligent decision-making model, the intelligent decision-making model extracts features from the first workshop status information s, and generates and outputs the first control action information a in the action space based on the mapping relationship between the workshop status and the control action.

[0071] Step 140: Generate a production decision plan for the hot working workshop to produce the target workpiece according to the first control action information.

[0072] Among them, the production decision plan includes production task allocation, equipment scheduling plan, material requirement list and risk assessment, etc., which are used to control the equipment in the hot processing workshop to control the equipment to execute the first control action information.

[0073] In this step, the first control action information is analyzed to determine the content of the control action information, including equipment status, current production status, process parameters, etc., and to clarify requirements such as specifications, quantity, and delivery time of the target workpiece.

[0074] According to the characteristics of the target workpiece, design the corresponding hot working process flow, including the sequence and operation requirements of each process.

[0075] According to the first control action information, set the optimal process parameters such as temperature, time, pressure, etc. to ensure the quality of the target workpiece, use simulation tools to predict the production effects under different parameters, optimize the process parameters, and form a complete production decision plan.

[0076] According to the dynamic decision-making method for aerospace product hot processing workshop provided in the embodiment of the present application, by abstracting the first production data monitored in real time, the first workshop status information is obtained as the input of the intelligent decision-making model, and a decision-making production plan for dynamically adjusting the production parameters is obtained. It can ensure that key parameters such as temperature, pressure, stress and strain in the production process are always maintained within the optimal range. During the production process of the aerospace product hot processing workshop, the multi-modal state of the workshop is dynamically identified to achieve efficient decision-making.

[0077] In some embodiments, the first workshop status information in the status space includes product processing quality, order status information, machine utilization rate and overall machine energy consumption average value of the target workpiece during the production process.

[0078] In some embodiments, the expression of the product processing quality is:

[0079]

[0080] Wherein, Q(t) represents the processing quality of the product; α, β, γ, δ, ζ are the weights of the state parameters that determine the quality; T represents the state parameter corresponding to the forging temperature; P represents the state parameter corresponding to the forging pressure; S micro-structure Represents the state parameter corresponding to the microstructure; represents the state parameter corresponding to the strain; G represents the state parameter corresponding to the geometric dimension.

[0081] In some embodiments, the order status information is characterized as a collection of current orders and urgent orders of the target workpiece:

[0082] W={(p i ,d i ,w i )|i=1,2,3…,n}∪{(p E ,d E ,w E )}

[0083] Where W represents the set of current orders and urgent orders; i is the order number, p i represents the processing time of order i; d i represents the delivery deadline of order i; ω i represents the weight of order i; p E Indicates the processing time of urgent orders; d E Indicates the delivery deadline for urgent orders; E represents the weight of urgent orders, ω E Greater than ω i .

[0084] In actual implementation, the first workshop status information a={Q(t), W, U std (t), U ave-E (t)}, as follows:

[0085]

[0086] Where Q(t) represents the product processing quality; α, β, γ, δ, ζ are the weights of the various state parameters that determine the quality; T represents the forging temperature; P represents the forging pressure; S micro-structure Indicates the microstructure (such as average grain size); represents strain; G represents geometric dimensions; p i represents the processing time of order i; d i represents the delivery deadline of order i; ω i represents the weight (priority) of order i; p E Indicates the processing time of urgent orders; d E Indicates the delivery deadline for urgent orders; E represents the weight of the urgent order (ω E Greater than ω i ); W represents the set of current orders and urgent orders; U std (t) represents the standard deviation of the average machine utilization; U ave (t) represents the average machine utilization rate; U energy (t) represents the total energy consumption of the workshop in the current state; U ave-E (t) represents the average energy consumption of the whole machine; n represents the total number of workpieces; m represents the total number of machines; i represents the workpiece serial number; j represents the machine serial number; h i represents the total number of processes of the workpiece (o∈(1,2,...,h i ); o represents the process identifier (o∈(1,2,...,h i );t ioj represents the processing time of process o of workpiece i on machine j; represents the startup energy consumption of machine j; represents the shutdown energy consumption of machine j; represents the processing power of machine j per unit time; represents the waiting power of machine j per unit time; ST ioj represents the start time of process o of workpiece i on machine j; FT ioj represents the end processing time of process o of workpiece i on machine j; x ioj Indicates that the process o of workpiece i takes the value of 1 when it is processed by machine j (otherwise it is 0); ioj The value is 1 when machine j is in shutdown state when processing process o of workpiece i (otherwise it is 0); xioi‘o’j The value of "1" indicates that the process o of workpiece i precedes the process o' of workpiece i' when it is processed by machine j (otherwise it is 0).

[0087] In some embodiments, the neural network model includes an online Q network and a target Q network, the online Q network is used to predict and output the first control action information a based on the first workshop status information s, and the online Q network is used to update the parameters of the target Q network.

[0088] In this embodiment, the neural network model is a double deep Q-Network (DDQN), and the intelligent decision-making model can be a decision-making agent based on a framework using the DDQN algorithm, which is reconstructed in combination with aerospace thermal processing states, actions and rewards, and the simulation environment is used for calculation. The computer test environment is: based on Python language and Tensorflow2.0 framework, Intel(R) Core(TM) i5-10400F CPU@2.90GHz, RAM32GB.

[0089] In some embodiments, the step of inputting the first workshop state information s into the intelligent decision model to obtain the first control action information a in the action space output by the intelligent decision model includes:

[0090] Inputting the first workshop state information s into the online Q network, performing value evaluation on a plurality of feasible actions corresponding to the first workshop state information s through the online Q network, and determining the action combination with the highest Q value among the plurality of feasible actions as the first control action information a;

[0091] The first control action information a is evaluated for value through the target Q network to obtain a target Q value of the first control action information, and the target Q value is used to update the online Q network.

[0092] In actual execution, the first control action information is represented as:

[0093] a=[a temp ,a press ,a speed ,a heat_treat ,a prod_order ,a task_bal ,a power_bal ]

[0094] In the formula, a represents the action vector, each element of which represents a specific action, where:

[0095] Forging temperature adjustment action: a temp ∈[T min ,Tmax ], T min and T max Indicates the minimum and maximum permissible temperatures;

[0096] Forging pressure adjustment action: a press ∈[P min ,P max ], P min and P max Indicates the minimum and maximum permissible pressure;

[0097] Forging speed control action: a speed ∈[S min ,S max ],S min and S max Indicates the minimum and maximum permitted speeds;

[0098] Heat treatment parameters (heat treatment furnace) adjustment action: a heat_treat ∈[H min ,H max ], H min and H max Indicates the minimum and maximum permissible heat treatment temperatures;

[0099] Production sequencing action: a prod_order ∈{all possible production sequences};

[0100] Machine task balance allocation action: a task_bal ∈{all possible task assignments};

[0101] High and low power machine task balance allocation action: a power_bal ∈{task balance of all possible high and low power machines}.

[0102] In some embodiments, after generating a production decision plan for producing the target workpiece in the hot working workshop according to the first control action information, the method further includes:

[0103] When the hot processing workshop executes the production decision plan, obtaining second production data corresponding to the production decision plan;

[0104] Abstracting the second production data in the state space to obtain second workshop state information of the hot processing workshop;

[0105] Taking the combination of the first workshop state information, the first control action information, the reward value corresponding to the first control action information and the second workshop state information s' as a training sample, and storing the training sample in an experience replay pool;

[0106] The training samples are extracted from the experience replay pool to update the parameters of the intelligent decision-making model.

[0107] like Figure 3 As shown, in the decision-making agent, there are online Q-Network, target Q-Network, loss function and experience replay pool;

[0108] The current state s (first workshop state information) of the aerospace hot working workshop is input into the online Q-Network, and the value of multiple feasible actions corresponding to the first workshop state information s is evaluated through the online Q-Network. Among the multiple feasible actions, the action combination with the highest Q value is determined as the first control action information a, and the evaluation Q(s, a; θ) and periodic replication parameters are obtained. The online Q-Network can also input the periodic replication parameters into the target Q-Network, wherein the evaluation Q(s, a; θ) includes the first workshop state information s, the first control action information a and the current network weight θ of the online Q-Network.

[0109] The first control action information a is evaluated by the target Q-Network, and the target Q-Network outputs the target Q value, max a′ Q(s', a'; θ') is used as the target Q value, wherein the target Q value includes the second workshop state information s', the second control action information a' and the current network weight θ' of the target Q-Network.

[0110] The loss function evaluates Q(s, a; θ) and the target max a′ Q(s', a'; θ') is calculated to update the network parameters of the online Q-Network.

[0111] After the production decision plan is executed in the hot processing workshop, the reward value r corresponding to the first control action information a is obtained through the reward function, and the second production data of the hot processing workshop is collected, and the second production data is abstracted in the state space to obtain the second workshop state information s' of the hot processing workshop; the combination of the first workshop state information s, the first control action information a, the reward value r corresponding to the first control action information a and the second workshop state information s' (s, a, r, s') is used as a training sample, and the training sample is stored in the experience replay pool.

[0112] Input (s, a) in the experience replay pool into the online Q-Network, and input s' in the experience replay pool into the target Q-Network.

[0113] The steps for training and applying the intelligent decision-making model are shown in Table 1.

[0114] Table 1 Steps for training and applying intelligent decision-making models

[0115]

[0116]

[0117] Initialize the online Q-Network and the target Q-Network, use the experience replay buffer to store the training samples, improve the training efficiency and stability through random sampling, and then use the DDQN method to reduce the deviation of the Q value estimation and improve the training stability; train in a simulated environment, collect samples of state, action, reward and next state, update network parameters, etc., after the model training is completed.

[0118] Double DQN is used to reduce the deviation of Q value estimation, introduce a fixed target network, regularly update the parameters of the target network, and stabilize the training process.

[0119] The output of the online Q network is a vector of length equal to the size of the action space. For each state s, the online Q network will output each possible action combination a i The Q value Q(s,a i ), select the action combination with the highest Q value argmax ai Q(s,a i ), using the experience replay buffer to store the agent’s experience, i.e., samples of state, action, reward, and next state (s, a, r, s′), DDQN uses the target network to calculate the Q value: y = R + γQ target (s′,argmax a′ Q(a,s′)), and update the Q network using the minimized loss function:

[0120] In this embodiment, it is possible to respond quickly to dynamic events occurring in a hot working workshop of aerospace products, perceive the workshop status in real time and dynamically, make intelligent decisions on workshop actions based on status characteristics, and make real-time dynamic feedback adjustments based on the workshop actions after the decision, so as to ensure processing quality and improve production efficiency. This has practical application value for the development of production process control systems in hot working workshops of aerospace products, and can achieve efficient decision-making and intelligent control.

[0121] In some embodiments, the reward value is obtained based on a reward function, and the reward function R is:

[0122] R=ω1·R geo +ω2·R mech +ω3·R def +ω4·R eff -P viol

[0123] Among them, R geo represents the size and shape deviation reward value of the target workpiece, R mech represents the mechanical property deviation reward value of the target workpiece, R def represents the reward for defect detection of the target workpiece, R eff represents the reward for the production efficiency of the target workpiece, P viol It means that a negative reward is given to the action in the first control action information a that may cause quality problems, and ω1, ω2, ω3, and ω4 are weight parameters that characterize relative importance.

[0124] In actual implementation, the reward function is designed as a comprehensive reward function that takes into account both quality indicators and efficiency balance.

[0125] Among them, quality indicators include geometric accuracy, mechanical properties, material uniformity and defect detection, etc. Efficiency balance is to introduce rewards related to order delivery time under the premise of ensuring quality, to ensure that urgent orders are completed on time, while considering production efficiency, reducing production time and energy consumption;

[0126] R=ω1·R geo +ω2·R mech +ω3·R def +ω4·R eff -P viol

[0127] In the formula, R geo Represents the size and shape deviation reward value of the part, R mech Represents the mechanical performance deviation reward value, R def represents the reward for defect detection, R eff It indicates the reward for production efficiency. Under the premise of ensuring quality, production efficiency is taken into consideration, such as reducing time and energy consumption. viol It means punishing actions that violate quality standards and giving negative rewards to actions that may cause quality problems, such as too high or too low temperature, pressure beyond the safe range, etc. ω1, ω2, ω3, and ω4 are weight parameters for adjusting the relative importance of each part.

[0128] In this embodiment, by setting the reward function, the learning behavior of the intelligent decision-making model can be effectively guided, so that better decisions can be made in the complex environment of a hot processing workshop, thereby improving the overall performance of the intelligent decision-making model.

[0129] After the model is trained in a simulation environment, it is loaded into the existing control workshop MES system protocol for communication and application. The present invention can ensure the quality of aerospace thermal processing products and improve production efficiency by dynamically adjusting production parameters in real time.

[0130] Before testing the model in an actual aerospace product hot working workshop, the algorithm model training is completed, and a communication protocol interface between the algorithm and the MES system of the control workshop is built. The interface can read the status of the hot working workshop and issue action instructions to ensure that it can respond to dynamic events such as emergency order insertion and complete offline verification.

[0131] The algorithm performance is tested using the standard set MK01 to MK10. The algorithm test set usually does not include quality factors. Therefore, in this embodiment, the machine sorting individuals in the state vector space and the action vector space are retained, and other individuals are set to 0. The first workshop state information is represented by a state space vector, and the first control action information is represented by an action space vector. The state space vector s and the action space vector a are:

[0132] s=[0,0,0,0,W,0,0]

[0133] a=[0,0,0,0,a prod_order ,0,0]

[0134] The solution results are shown in Table 2, which show that the algorithm of the present invention is better than other algorithms. For example, the optimal value of the completion time in 10 problem cases is better than the genetic algorithm.

[0135] Table 2 Minimum completion time of different algorithms under the standard test set

[0136]

[0137]

[0138] In addition, according to the statistics of the factory's existing manual event control methods, as shown in Table 3, manual scheduling and process design usually take 2 to 3 days, emergency orders require about 6 hours to re-schedule, and manual scheduling during fault handling also takes 0.5 hours. The intelligent control method based on the present invention can greatly improve the control efficiency.

[0139] Table 3 Comparison of the efficiency of different control methods for dynamic control in hot processing workshops of aerospace products

[0140] event Manual control Intelligent control based on deep reinforcement learning Existing production scheduling and process design 2-3 days 0.5~1 hour Urgent orders arrive and reschedule production 6 hours 1 to 5 minutes Scheduling during troubleshooting 0.5~1 hour 5-10 minutes

[0141] In this embodiment, the microstructure in the forging process is optimized by finely controlling the action, reducing production defects, and improving the mechanical properties and geometric accuracy of parts. By introducing an emergency order insertion mechanism, the production demand changes can be quickly responded to, the production sequence can be dynamically adjusted, the order waiting time can be reduced, the production bottleneck and resource waste can be reduced, the production system's ability to respond to emergency orders and emergencies can be enhanced, and timely delivery can be ensured. Improve overall production efficiency, reduce energy consumption, and achieve green manufacturing. Under the premise of ensuring high quality, optimize production efficiency and enhance corporate competitiveness.

[0142] The dynamic decision-making method for aerospace product hot processing workshop provided in the embodiment of the present application can be executed by a dynamic decision-making device for aerospace product hot processing workshop. In the embodiment of the present application, the dynamic decision-making method for aerospace product hot processing workshop is executed by a dynamic decision-making device for aerospace product hot processing workshop as an example to illustrate the dynamic decision-making device for aerospace product hot processing workshop provided in the embodiment of the present application.

[0143] An embodiment of the present application also provides a dynamic decision-making device for an aerospace product hot processing workshop.

[0144] like Figure 4 As shown, the dynamic decision-making device of the hot processing workshop of aerospace products includes:

[0145] An acquisition module 410 is used to acquire first production data in a process of producing a target workpiece in a hot working workshop;

[0146] A first processing module 420 is used to abstract the first production data in a state space to obtain first workshop state information of the hot processing workshop;

[0147] A second processing module 430 is used to input the first workshop state information into an intelligent decision model to obtain first control action information of an action space output by the intelligent decision model, wherein the intelligent decision model is constructed based on a neural network model;

[0148] The third processing module 440 is used to generate a production decision plan for the hot processing workshop to produce the target workpiece according to the first control action information.

[0149] According to the dynamic decision-making device for the hot processing workshop of aerospace products provided in the embodiment of the present application, by abstracting the first production data monitored in real time, the first workshop status information is obtained as the input of the intelligent decision-making model, and a decision-making production plan for dynamically adjusting the production parameters is obtained. It can ensure that key parameters such as temperature, pressure, stress and strain in the production process are always maintained within the optimal range. During the production process of the hot processing workshop of aerospace products, the multimodal state of the workshop is dynamically identified to achieve efficient decision-making.

[0150] The dynamic decision-making device for the hot processing workshop of aerospace products in the embodiment of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or a device other than a terminal. Exemplarily, the electronic device can be a tablet computer, a laptop computer, a PDA, an Ultra-Mobile Personal Computer (UMPC), a netbook, or a Personal Digital Assistant (PDA), etc., which is not specifically limited in the embodiment of the present application.

[0151] The dynamic decision-making device for aerospace product hot processing workshop in the embodiment of the present application may be a device having an operating system. The operating system may be a Linux operating system, a Windows operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0152] The dynamic decision-making device for aerospace product hot processing workshop provided in the embodiment of the present application can realize the various processes realized by the above-mentioned embodiment of the dynamic decision-making method for aerospace product hot processing workshop. To avoid repetition, they will not be described here.

[0153] In some embodiments, Figure 5 As shown, an embodiment of the present application also provides an electronic device 500, including a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, each process of the above-mentioned embodiment of the dynamic decision-making method for a hot working workshop of aerospace products is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0154] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0155] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned embodiment of the dynamic decision-making method for aerospace product hot processing workshop are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0156] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0157] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned dynamic decision-making method for a hot processing workshop of aerospace products.

[0158] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0159] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned embodiment of the dynamic decision-making method for aerospace product hot processing workshop, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0160] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0161] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0162] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the dynamic decision-making method of aerospace product hot processing workshop in each embodiment of the present application.

[0163] In the description of this application, "first feature" or "second feature" may include one or more of the features.

[0164] In the description of the present application, “plurality” means two or more.

[0165] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0166] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0167] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A dynamic decision-making method for aerospace product hot processing workshop, characterized in that: include: Acquire first production data during the process of producing a target workpiece in a hot working workshop; Abstracting the first production data in the state space to obtain first workshop state information of the hot processing workshop; Inputting the first workshop state information into an intelligent decision-making model to obtain first control action information of an action space output by the intelligent decision-making model, wherein the intelligent decision-making model is constructed based on a neural network model; A production decision plan for the hot working workshop to produce the target workpiece is generated according to the first control action information.

2. The dynamic decision-making method for aerospace product hot processing workshop according to claim 1 is characterized in that: The first workshop status information in the status space includes product processing quality, order status information, machine utilization rate and overall machine energy consumption average value of the target workpiece during the production process.

3. The dynamic decision-making method for aerospace product hot processing workshop according to claim 2 is characterized in that: The expression of the product processing quality is: Wherein, Q(t) represents the processing quality of the product; α, β, Y, δ, ζ are the weights of the state parameters that determine the quality; T represents the state parameter corresponding to the forging temperature; P represents the state parameter corresponding to the forging pressure; S micro-structure Represents the state parameter corresponding to the microstructure; represents the state parameter corresponding to the strain; G represents the state parameter corresponding to the geometric dimension.

4. The dynamic decision-making method for aerospace product hot processing workshop according to claim 2 is characterized in that: The order status information is characterized as a collection of current orders and urgent orders for the target workpiece: W={(p i ,d i ,w i )|i=1,2,3…,n}∪{(p E ,d E ,w E )} Where W represents the set of current orders and urgent orders; i is the order number, p i represents the processing time of order i; d i represents the delivery deadline of order i; ω i represents the weight of order i; p E Indicates the processing time of urgent orders; d E Indicates the delivery deadline for urgent orders; E represents the weight of urgent orders; ω E Greater than ω i .

5. The dynamic decision-making method for aerospace product hot processing workshop according to claim 1 is characterized in that: The neural network model includes an online Q network and a target Q network. The online Q network is used to predict and output the first control action information according to the first workshop status information, and the online Q network is used to update the parameters of the target Q network.

6. The dynamic decision-making method for aerospace product hot processing workshop according to claim 5 is characterized in that: The step of inputting the first workshop state information into the intelligent decision model to obtain first control action information of the action space output by the intelligent decision model includes: Inputting the first workshop status information into the online Q network, performing value evaluation on a plurality of feasible actions corresponding to the first workshop status information through the online Q network, and determining the action combination with the highest Q value among the plurality of feasible actions as the first control action information; The first control action information is valued by using the target Q network to obtain a target Q value of the first control action information, and the target Q value is used to update the online Q network.

7. The dynamic decision-making method for aerospace product hot processing workshop according to claim 1 is characterized in that: After generating a production decision plan for producing the target workpiece in the hot working workshop according to the first control action information, the method further includes: When the hot processing workshop executes the production decision plan, obtaining second production data corresponding to the production decision plan; Abstracting the second production data in the state space to obtain second workshop state information of the hot processing workshop; taking the first workshop state information, the first control action information, a combination of the reward value corresponding to the first control action information and the second workshop state information as a training sample, and storing the training sample in an experience replay pool; The training samples are extracted from the experience replay pool to update the parameters of the intelligent decision-making model.

8. The dynamic decision-making method for aerospace product hot processing workshop according to claim 7 is characterized in that: The reward value is obtained based on a reward function, and the reward function R is: R=ω1·R geo +ω2·R mech +ω3·R def +ω4·R eff -P viol Among them, R geo represents the size and shape deviation reward value of the target workpiece, R mech represents the mechanical property deviation reward value of the target workpiece, R def represents the reward for defect detection of the target workpiece, R eff represents the reward for the production efficiency of the target workpiece, P viol It indicates that a negative reward is given to the action in the first control action information that may cause quality problems, and ω1, ω2, ω3, and ω4 are weight parameters that characterize relative importance.

9. A dynamic decision-making device for aerospace product hot processing workshop, characterized in that: include: An acquisition module, used for acquiring first production data in a process of producing a target workpiece in a hot working workshop; A first processing module, used for abstracting the first production data in a state space to obtain first workshop state information of the hot processing workshop; A second processing module is used to input the first workshop state information into an intelligent decision-making model to obtain first control action information of an action space output by the intelligent decision-making model, wherein the intelligent decision-making model is constructed based on a neural network model; The third processing module is used to generate a production decision plan for the hot processing workshop to produce the target workpiece according to the first control action information.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the dynamic decision-making method for aerospace product hot working workshop as described in any one of claims 1-8.

Images