5G communication module batch assembly production line and production method thereof

Abstract

The invention relates to the technical field of 5G communication modules, in particular to a 5G communication module batch assembly production line and a production method thereof.The 5G communication module batch assembly production line comprises a plurality of intelligent manufacturing units working cooperatively, and each unit comprises a material feeding unit, a chip mounter, a reflow soldering machine, a testing unit and the like; the multi-agent reinforcement learning system is used for coordinating the operation of each intelligent manufacturing unit, and the multi-agent reinforcement learning system comprises a plurality of agents respectively corresponding to different intelligent manufacturing units; the environment model is used for simulating various states and events in the production process; the reward function is used for evaluating behaviors of the intelligent agent; the multi-agent reinforcement learning algorithm is used for training the agents so that the agents can learn to make an optimal decision in a complex and changeable production environment; according to the invention, through a plurality of intelligent manufacturing units which work cooperatively and a plurality of intelligent agent reinforcement learning systems, the production strategy can be dynamically adjusted, and the efficient operation of the production process is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of 5G communication modules, and specifically to a 5G communication module batch assembly production line and a production method thereof. Background Art

[0002] In modern industrial production, the batch completion of 5G communication modules involves multiple complex production and precision equipment; traditional production lines rely on manual operation and simple automated equipment, and have problems such as low production efficiency, unstable product quality, and high energy consumption. With the rapid development of artificial intelligence and Internet of Things technologies, intelligent manufacturing has become an effective means to solve these problems.

[0003] As an emerging intelligent manufacturing technology, multi-agent reinforcement learning system can achieve dynamic optimization and precise control of the production process through good work and learning among multiple agents; the application of advanced artificial intelligence algorithms such as deep Q network (DQN), graph neural network (GNN) and long short-term memory network (LSTM) in intelligent manufacturing systems has made it possible to improve the intelligence level and flexibility of production lines.

[0004] Digital twin technology realizes the dual interaction of virtual environment and actual production environment by constructing a digital model of the physical system. It can perform optimization and testing in the virtual environment, thereby greatly reducing the trial and error costs and risks in actual production. The intelligent agent reinforcement learning system further improves the transformation degree and adaptability of the production line. Therefore, a 5G communication module batch assembly production line and a production method thereof are proposed to address the above problems. Summary of the invention

[0005] The object of the present invention is to provide a 5G communication module batch assembly production line and a production method thereof to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A 5G communication module batch assembly production line, comprising:

[0008] Multiple collaborative intelligent manufacturing units: each unit includes a material feeding unit, a placement machine, a reflow machine, a testing unit, etc.

[0009] Multi-agent reinforcement learning system: used to coordinate the operation of each intelligent manufacturing unit, the multi-agent reinforcement learning system includes:

[0010] Multiple intelligent agents: corresponding to different intelligent manufacturing units;

[0011] Environmental model: used to simulate various states and events in the production process;

[0012] Reward function: used to evaluate the behavior of the agent;

[0013] Multi-agent reinforcement learning algorithm: used to train agents to make optimal decisions in complex and changing production environments.

[0014] Preferably, the multi-agent reinforcement learning algorithm is a deep Q network algorithm, and each agent has an independent DQN network. The specific parameters are set as follows:

[0015] Learning rate (α): 0.001;

[0016] Discount factor (γ): 0.99;

[0017] Experience replay buffer size: 100,000;

[0018] Mini-batch size: 64;

[0019] Target network update frequency: every 1000 steps;

[0020] The DQN algorithm formula is as follows:

[0021]

[0022] in:

[0023] Q(s t , a t ) means in state s t Take action a t The value function of

[0024] α = 0.001 is the learning rate;

[0025] r t+1 is the reward obtained at time t+1;

[0026] γ = 0.99 is the discount factor.

[0027] Preferably, the environmental model takes into account factors such as random disturbances, equipment failures, and material shortages in the production process, and the specific parameters include:

[0028] Random disturbance frequency: 10 times per day;

[0029] Equipment failure probability: 0.01 per hour;

[0030] Probability of material shortage: 0.02 per shift.

[0031] Preferably, the reward function comprehensively considers multiple indicators such as production efficiency, product quality, energy consumption, etc., and the specific calculation formula is as follows:

[0032] R=0.5×E+0.3×Q-0.2×C

[0033] in:

[0034] E is production efficiency, measured by the number of qualified products produced per unit time;

[0035] Q is product quality, measured by the pass rate;

[0036] C is energy consumption, measured by the energy consumption per unit product.

[0037] Preferably, it also includes:

[0038] Establish a digital twin model of the 5G communication module production line;

[0039] Conduct multi-agent reinforcement learning training on the digital twin model, so that the agent can learn and optimize production strategies in a virtual environment;

[0040] Deploy the trained multi-agents into the actual production system to achieve closed-loop control of the digital twin and the physical system.

[0041] Preferably, the algorithm is based on a graph neural network, which represents the connection relationship between components as a graph structure and uses the graph neural network to learn the optimal layout. The specific parameter settings are as follows:

[0042] Number of GNN layers: 3 layers, each layer contains 128 nodes;

[0043] Learning rate: 0.001;

[0044] Loss function: cross entropy loss;

[0045] The GNN algorithm formula is as follows:

[0046]

[0047] in:

[0048] represents the hidden representation of node v at layer k;

[0049] W (k) and b (k) are the weight matrix and bias vector of the kth layer respectively;

[0050] N(v) represents the set of neighbor nodes of node v;

[0051] σ is the activation function.

[0052] Preferably, the algorithm is based on a long short-term memory network, which predicts equipment failures and product defects by analyzing historical production data. The specific parameters are set as follows:

[0053] Number of LSTM layers: 2 layers, each containing 64 hidden units;

[0054] Time step: 30;

[0055] Learning rate: 0.001;

[0056] The LSTM algorithm formula is as follows:

[0057] f t =σ(W f ·[h t-1 , x t ]+b f )

[0058] i t =σ(W i ·[h t-1 , x t ]+b i )

[0059]

[0060] o t =σ(W o ·[h t-1 , x t ]+b o )

[0061] h t =o t *tanh(C t )

[0062] in:

[0063] f t 、i t , C t , o t 、h t Represent the forget gate, input gate, candidate cell state, cell state, output gate and hidden state respectively;

[0064] W f , W i , W C , W o are the weight matrices of each gate respectively;

[0065] b f 、b i 、b C 、b o are the bias vectors of each gate respectively;

[0066] σ and tanh are Sigmoid and Tanh activation functions respectively.

[0067] A 5G communication module batch production method, comprising:

[0068] Step 1: Initialize the multi-agent reinforcement learning system;

[0069] Step 2: Train multiple agents in a simulation environment to make them learn to make decisions in various production scenarios. The specific steps include:

[0070] S1. Simulate different production scenarios in a simulation environment;

[0071] S2, the agent accumulates experience by interacting with the simulation environment and is trained through the DQN algorithm;

[0072] S3, using experience replay mechanism and target network to stabilize the training process;

[0073] Step 3: Deploy the trained multi-agent to the actual production system;

[0074] Step 4: Multiple intelligent agents interact to adjust production parameters in real time and optimize the production process.

[0075] Preferably, during the training process, an experience replay mechanism is used to improve sample utilization, and the specific settings are as follows:

[0076] Experience replay buffer size: 100,000;

[0077] Experience replay frequency: once per training step;

[0078] During the training process, the target network is used to stabilize the training process. The specific parameters are set as follows:

[0079] Target network update frequency: every 1000 steps;

[0080] Target network weight update coefficient: 0.001.

[0081] Preferably, including:

[0082] Establish a digital twin model of the 5G communication module production line, and accurately simulate the actual production line by integrating real-time data and virtual models;

[0083] Conduct multi-agent reinforcement learning training on the digital twin model, so that the agent can learn and optimize production strategies in a virtual environment;

[0084] Deploy the trained multi-agents into the actual production system to achieve closed-loop control of the digital twin and the physical system, and improve the intelligence and flexibility of the production line.

[0085] Compared with the prior art, the present invention has the following beneficial effects:

[0086] 1. In the present invention, through multiple collaborative intelligent manufacturing units and multiple intelligent agent reinforcement learning systems, the production strategy can be dynamically adjusted to achieve efficient operation of the production process.

[0087] 2. In the present invention, the intelligent body monitors and adjusts in real time during the production process, comprehensively considering multiple indicators such as production efficiency, product quality, energy consumption, etc., to ensure the production of 5G communication modules with stable quality.

[0088] 3. In the present invention, energy consumption indicators are introduced into the reward function, and unnecessary energy waste is reduced by optimizing production parameters to achieve the purpose of energy saving and consumption reduction.

[0089] 4. In the present invention, the environmental model takes into account factors such as random disturbances, equipment failures, and shutdown interruptions in the production process, so that the intelligent agent can better respond to emergencies and ensure the stability and continuity of the production line.

[0090] 5. In the present invention, the algorithm based on graph neural network can efficiently process the complex connection relationship between devices, optimize the device layout, and improve the rationality of circuit design and the operability of production process.

[0091] 6. In the present invention, by analyzing historical production data, the long short-term memory network can accurately predict equipment failures and product defects, carry out preventive maintenance and quality control in advance, and reduce production time and defective rate.

[0092] 7. In the present invention, a digital twin model is established through training, and multiple agents are tested in a virtual environment to reduce the trial and error costs in actual production, and to achieve closed-loop control of the digital twin and the physical system, thereby improving the standardization and flexibility of the production line.

[0093] 8. In the present invention, multiple agents can flexibly adjust production parameters according to changes in the production environment through real-time interaction to ensure the stability and operation of the production process. DETAILED DESCRIPTION

[0094] Embodiment: The present invention provides a technical solution:

[0095] A 5G communication module batch assembly production line, comprising:

[0096] Multiple collaborative intelligent manufacturing units: each unit includes a material feeding unit, a placement machine, a reflow machine, a testing unit, etc.

[0097] Multi-agent reinforcement learning system: used to coordinate the operation of each intelligent manufacturing unit, the multi-agent reinforcement learning system includes:

[0098] Multiple intelligent agents: corresponding to different intelligent manufacturing units;

[0099] Environmental model: used to simulate various states and events in the production process;

[0100] Reward function: used to evaluate the behavior of the agent;

[0101] Multi-agent reinforcement learning algorithm: used to train agents to make optimal decisions in complex and changing production environments.

[0102] Through the collaborative work of intelligent manufacturing units, precise control and optimization of the production process are achieved; the multi-agent reinforcement learning system can dynamically adjust the production strategy according to the real-time changes in the production environment, improve production efficiency and product quality, reduce energy consumption and waste, and improve the automation level of the production line.

[0103] The multi-agent reinforcement learning algorithm is a deep Q network algorithm, and each agent has an independent DQN network. The specific parameter settings are as follows:

[0104] Learning rate (α): 0.001;

[0105] Discount factor (γ): 0.99;

[0106] Experience replay buffer size: 100,000;

[0107] Mini-batch size: 64;

[0108] Target network update frequency: every 1000 steps;

[0109] The DQN algorithm formula is as follows:

[0110]

[0111] in:

[0112] Q(s t , a t ) means in state s t Take action a t The value function of

[0113] α = 0.001 is the learning rate;

[0114] r t+1 is the reward obtained at time t+1;

[0115] γ = 0.99 is the discount factor.

[0116] By using the DQN algorithm, each intelligent agent can learn and optimize independently, ensuring rapid adaptation and optimal decision-making in complex production environments; specific parameter settings improve the stability and efficiency of the algorithm, allowing the intelligent agent to maintain efficient operation in the face of dynamically changing production environments.

[0117] The environmental model takes into account factors such as random disturbances, equipment failures, and material shortages in the production process. The specific parameters include:

[0118] Random disturbance frequency: 10 times per day;

[0119] Equipment failure probability: 0.01 per hour;

[0120] Probability of material shortage: 0.02 per shift.

[0121] The design of the environmental model fully considers the various uncertainties encountered in the actual production process, which may enable the intelligent agent to better cope with these emergencies during training and application, ensure the stability and continuity of the production line, and reduce production interruptions and losses caused by unexpected situations.

[0122] The reward function comprehensively considers multiple indicators such as production efficiency, product quality, energy consumption, etc. The specific calculation formula is as follows:

[0123] R=0.5×E+0.3×Q-0.2×C

[0124] in:

[0125] E is production efficiency, measured by the number of qualified products produced per unit time;

[0126] Q is product quality, measured by the pass rate;

[0127] C is energy consumption, measured by the energy consumption per unit product.

[0128] The reward function that comprehensively considers production efficiency, product quality and energy consumption can effectively balance various key indicators, not only ensuring the production process, but also producing high-quality products, while reducing energy consumption, achieving the dual effects of economy and environmental protection.

[0129] Also includes:

[0130] Establish a digital twin model of the 5G communication module production line;

[0131] Conduct multi-agent reinforcement learning training on the digital twin model, so that the agent can learn and optimize production strategies in a virtual environment;

[0132] Deploy the trained multi-agents into the actual production system to achieve closed-loop control of the digital twin and the physical system.

[0133] By establishing a digital twin model, the production line can be simulated and optimized in a virtual environment, greatly reducing the trial and error costs and risks in actual production; the trained intelligent agents are deployed in the actual production system, realizing the closed-loop control of the digital twin through the physical system, thereby improving the degree and ability of the production line to be transformed.

[0134] The algorithm is based on graph neural network, which represents the connection relationship between components as a graph structure and uses graph neural network to learn the optimal layout. The specific parameter settings are as follows:

[0135] Number of GNN layers: 3 layers, each layer contains 128 nodes;

[0136] Learning rate: 0.001;

[0137] Loss function: cross entropy loss;

[0138] The GNN algorithm formula is as follows:

[0139]

[0140] in:

[0141] represents the hidden representation of node v at layer k;

[0142] W (k) and b (k) are the weight matrix and bias vector of the kth layer respectively;

[0143] N(v) represents the set of neighbor nodes of node v;

[0144] σ is the activation function.

[0145] By using the GNN algorithm, the complex connection relationship between chips can be efficiently processed, and the chip layout can be optimized on this basis, which improves the rationality of circuit design and the operability of production process, thereby improving the overall performance and reliability of 5G communication modules.

[0146] The algorithm is based on the long short-term memory network and predicts equipment failures and product defects by analyzing historical production data. The specific parameter settings are as follows:

[0147] Number of LSTM layers: 2 layers, each containing 64 hidden units;

[0148] Time step: 30;

[0149] Learning rate: 0.001;

[0150] The LSTM algorithm formula is as follows:

[0151] f t =σ(W f·[h t-1 , x t ]+b f )

[0152] i t =σ(W i ·[h t-1 , x t ]+b i )

[0153]

[0154] o t =σ(W o ·[h t-1 , x t ]+b o )

[0155] h t =o t *tanh(C t )

[0156] in:

[0157] f t 、i t , C t , o t 、h t Represent the forget gate, input gate, candidate cell state, cell state, output gate and hidden state respectively;

[0158] W f , W i , W C , W o are the weight matrices of each gate respectively;

[0159] b f 、b i 、b C 、b o are the bias vectors of each gate respectively;

[0160] σ and tanh are Sigmoid and Tanh activation functions respectively.

[0161] By using the LSTM network, we can effectively capture the long-term dependent features in the production process, so as to accurately predict potential equipment failures and product defects, carry out preventive maintenance and quality control in advance, reduce production time and defective rate, and improve the reliability and stability of the production line.

[0162] A 5G communication module batch production method, comprising:

[0163] Step 1: Initialize the multi-agent reinforcement learning system;

[0164] Step 2: Train multiple agents in a simulation environment to make them learn to make decisions in various production scenarios. The specific steps include:

[0165] S1. Simulate different production scenarios in a simulation environment;

[0166] S2, the agent accumulates experience by interacting with the simulation environment and is trained through the DQN algorithm;

[0167] S3, using experience replay mechanism and target network to stabilize the training process;

[0168] Step 3: Deploy the trained multi-agent to the actual production system;

[0169] Step 4: Multiple intelligent agents interact to adjust production parameters in real time and optimize the production process.

[0170] By training multiple agents in a simulation environment, it is possible to optimize their decision-making capabilities before deploying them to the actual production system, reducing the trial and error costs in actual production. The real-time interaction and parameter adjustment of multiple agents improves the flexibility and efficiency of the production process and ensures the efficient operation of the production line.

[0171] During the training process, the experience replay mechanism is used to improve sample utilization. The specific settings are as follows:

[0172] Experience replay buffer size: 100,000;

[0173] Experience replay frequency: once per training step;

[0174] During the training process, the target network is used to stabilize the training process. The specific parameters are set as follows:

[0175] Target network update frequency: every 1000 steps;

[0176] Target network weight update coefficient: 0.001.

[0177] Replay can improve the utilization efficiency of training samples, reduce data experience, and enable the intelligent agent to repeatedly learn important experience during training, improve training effect and convergence speed, thereby improving the decision-making ability of the intelligent agent; the introduction of the target network can effectively avoid instability during training, maintain the stability of Q-value updates, and prevent unstable decisions caused by data fluctuations, thereby improving the adaptability and decision-making accuracy of the intelligent agent in complex environments.

[0178] include:

[0179] Establish a digital twin model of the 5G communication module production line, and accurately simulate the actual production line by integrating real-time data and virtual models;

[0180] Conduct multi-agent reinforcement learning training on the digital twin model, so that the agent can learn and optimize production strategies in a virtual environment;

[0181] Deploy the trained multi-agents into the actual production system to achieve closed-loop control of the digital twin and the physical system, and improve the intelligence and flexibility of the production line.

[0182] The application of digital twin technology enables production lines to be optimized and tested in a virtual environment, significantly reducing the trial and error costs and risks in actual production. Through multi-agent reinforcement learning training, the automation and adaptability of the production line are further improved, ensuring the efficiency and stability of the production process.

[0183] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. The above is only a preferred implementation of the present invention. It should be pointed out that due to the limitations of textual expression and the objective existence of infinite specific structures, ordinary technicians in this technical field can make several improvements, modifications or changes without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes or combinations, or the direct application of the inventive concept and technical solution to other occasions without improvement, should be regarded as the protection scope of the present invention.

Claims

1. A 5G communication module batch assembly production line, characterized by: include: Multiple collaborative intelligent manufacturing units: each unit includes a material feeding unit, a placement machine, a reflow machine, a testing unit, etc. Multi-agent reinforcement learning system: used to coordinate the operation of each intelligent manufacturing unit, the multi-agent reinforcement learning system includes: Multiple intelligent agents: corresponding to different intelligent manufacturing units; Environmental model: used to simulate various states and events in the production process; Reward function: used to evaluate the behavior of the agent; Multi-agent reinforcement learning algorithm: used to train agents to make optimal decisions in complex and changing production environments.

2. A 5G communication module batch assembly production line according to claim 1, characterized in that: The multi-agent reinforcement learning algorithm is a deep Q network algorithm, and each agent has an independent DQN network. The specific parameter settings are as follows: Learning rate (α): 0.001; Discount factor (γ): 0.99; Experience replay buffer size: 100,000; Mini-batch size: 64; Target network update frequency: every 1000 steps; The DQN algorithm formula is as follows: in: Q(s t , a t ) means in state s t Take action a t The value function of α = 0.001 is the learning rate; r t+1 is the reward obtained at time t+1; γ = 0.99 is the discount factor.

3. A 5G communication module batch assembly production line according to claim 1, characterized in that: The environmental model takes into account factors such as random disturbances, equipment failures, and material shortages in the production process. The specific parameters include: Random disturbance frequency: 10 times per day; Equipment failure probability: 0.01 per hour; Probability of material shortage: 0.02 per shift.

4. A 5G communication module batch assembly production line according to claim 1, characterized in that: The reward function comprehensively considers multiple indicators such as production efficiency, product quality, energy consumption, etc. The specific calculation formula is as follows: R=0.5×E+0.3×Q-0.2×C in: E is production efficiency, measured by the number of qualified products produced per unit time; Q is product quality, measured by the pass rate; C is energy consumption, measured by the energy consumption per unit product.

5. A 5G communication module batch assembly production line according to claim 1, characterized in that: Also includes: Establish a digital twin model of the 5G communication module production line; Conduct multi-agent reinforcement learning training on the digital twin model, so that the agent can learn and optimize production strategies in a virtual environment; Deploy the trained multi-agents into the actual production system to achieve closed-loop control of the digital twin and the physical system.

6. A 5G communication module batch assembly production line according to claim 1, characterized in that: The algorithm is based on graph neural network, which represents the connection relationship between components as a graph structure and uses graph neural network to learn the optimal layout. The specific parameter settings are as follows: Number of GNN layers: 3 layers, each layer contains 128 nodes; Learning rate: 0.001; Loss function: cross entropy loss; The GNN algorithm formula is as follows: in: represents the hidden representation of node v at layer k; W (k) and b (k) are the weight matrix and bias vector of the kth layer respectively; N(v) represents the set of neighbor nodes of node v; σ is the activation function.

7. A 5G communication module batch assembly production line according to claim 1, characterized in that: The algorithm is based on the long short-term memory network and predicts equipment failures and product defects by analyzing historical production data. The specific parameter settings are as follows: Number of LSTM layers: 2 layers, each containing 64 hidden units; Time step: 30; Learning rate: 0.001; The LSTM algorithm formula is as follows: f t =σ(W f ·[h t-1 ,x t ]+b f ) i t =σ(W i ·[h t-1 ,x t ]+b i ) the t =σ(W o ·[h t-1 ,x t ]+b o ) h t =o t *tanh(C t ) in: f t 、i t , C t , o t 、h t Represent the forget gate, input gate, candidate cell state, cell state, output gate and hidden state respectively; W f , W i , W C , W o are the weight matrices of each gate respectively; b f , b i , b C , b o are the bias vectors of each gate respectively; σ and tanh are Sigmoid and Tanh activation functions respectively.

8. A method for mass production of 5G communication modules, characterized in that: include: Step 1: Initialize the multi-agent reinforcement learning system; Step 2: Train multiple agents in a simulation environment to make them learn to make decisions in various production scenarios. The specific steps include: S1. Simulate different production scenarios in a simulation environment; S2, the agent accumulates experience by interacting with the simulation environment and is trained through the DQN algorithm; S3, using experience replay mechanism and target network to stabilize the training process; Step 3: Deploy the trained multi-agent to the actual production system; Step 4: Multiple intelligent agents interact to adjust production parameters in real time and optimize the production process.

9. A method for mass production of 5G communication modules according to claim 8, characterized in that: During the training process, the experience replay mechanism is used to improve sample utilization. The specific settings are as follows: Experience replay buffer size: 100,000; Experience replay frequency: once per training step; During the training process, the target network is used to stabilize the training process. The specific parameters are set as follows: Target network update frequency: every 1000 steps; Target network weight update coefficient: 0.

001.

10. A method for mass production of 5G communication modules according to claim 8, characterized in that: include: Establish a digital twin model of the 5G communication module production line, and accurately simulate the actual production line by integrating real-time data and virtual models; Conduct multi-agent reinforcement learning training on the digital twin model, so that the agent can learn and optimize production strategies in a virtual environment; Deploy the trained multi-agents into the actual production system to achieve closed-loop control of the digital twin and the physical system, and improve the intelligence and flexibility of the production line.

Images