Industrial internet intelligent factory production optimization method based on deep learning

Abstract

The invention provides an industrial internet intelligent factory production optimization method based on deep learning. The method comprises the steps of obtaining a multi-dimensional data vector of an intelligent factory; inputting the multi-dimensional data vector into a deep reinforcement learning model to obtain a basic scheduling strategy; taking the production scheduling optimization strategy and the multi-dimensional data vector as input, inputting the production scheduling optimization strategy and the multi-dimensional data vector into a constructed deep learning model, automatically learning and extracting risk features, and generating a safety risk score; performing online correction on the basic scheduling strategy according to the security risk score to generate a joint optimization strategy; collecting feedback data of the intelligent factory, and designing a target optimization function in combination with the security risk score; the joint optimization strategy is sent to the intelligent factory, so that the intelligent factory executes the joint optimization strategy according to the target optimization function, and feedback data of the intelligent factory is collected in real time.

Description

Technical Field

[0001] This invention belongs to the field of deep learning, and in particular relates to a method for optimizing production in an industrial internet smart factory based on deep learning. Background Technology

[0002] With the rapid development of Industrial Internet technology, smart factories have become an important direction for the transformation and upgrading of the manufacturing industry. Smart factories achieve the digitalization, automation, and informatization of the production process through the application of technologies such as sensors, the Internet of Things (IoT), big data, and artificial intelligence. Traditional manufacturing faces problems such as low production efficiency, untimely equipment maintenance, excessive energy consumption, and poor production line flexibility, which are effectively solved by smart factories. For example, through data collection and analysis, factories can monitor equipment operating status in real time, predict equipment failures promptly, and reduce downtime; through intelligent scheduling, production line resources are allocated more optimally, greatly improving production efficiency. At the same time, the Industrial Internet also facilitates the refined management of the production process, enabling every decision in the production process to be based on real-time data, thereby achieving optimal production.

[0003] However, despite the growing advantages of smart factories, existing technologies still face several bottlenecks that urgently need to be addressed. First, the problem of production process scheduling optimization remains unresolved. Under the constraints of multiple production tasks and complex equipment conditions, achieving real-time dynamic scheduling of the production process remains a highly challenging issue. Existing scheduling methods are often based on experience and historical data, lacking the flexibility to adapt to real-time production data, resulting in low production scheduling efficiency and an inability to quickly adapt to constantly changing production demands. Second, as the complexity of smart factories increases, cybersecurity issues are becoming increasingly apparent. Especially in the context of the Industrial Internet, data transmission between equipment, sensors, and control systems has become more frequent, leading to increasingly serious problems such as data leakage, tampering, and even equipment attacks. Traditional cybersecurity technologies, such as firewalls and intrusion detection systems, primarily target external attacks and do not fully consider the security of data exchange between equipment during production or the collaborative work within the system. Furthermore, existing security protection technologies often cannot be effectively integrated with production scheduling systems, resulting in the inability of the production scheduling system to adjust in a timely manner when a security incident occurs, leading to production interruptions or efficiency reductions. Therefore, ensuring data security and system stability in the industrial internet environment while guaranteeing efficient optimization of the production process has become a key issue that smart factories urgently need to address. Summary of the Invention

[0004] The purpose of this invention is to propose a deep learning-based method for optimizing production in smart factories within the industrial internet, addressing the shortcomings of traditional technologies in efficient production scheduling and safety protection.

[0005] This invention provides a deep learning-based method for optimizing production in an industrial internet-connected smart factory. The method includes:

[0006] S1. Obtain the multi-dimensional data vector of the smart factory;

[0007] S2. Input the multidimensional data vector into the deep reinforcement learning model to obtain the basic scheduling strategy;

[0008] S3. Input the production scheduling optimization strategy and multi-dimensional data vector into the constructed deep learning model and generate a safety risk score.

[0009] S4. Based on the security risk score, the basic scheduling strategy is corrected online to generate a joint optimization strategy;

[0010] S5. Collect feedback data from the smart factory and design a target optimization function based on the safety risk score; send the joint optimization strategy to the smart factory so that the smart factory executes the joint optimization strategy according to the target optimization function and collects feedback data from the smart factory in real time.

[0011] Furthermore, the multidimensional data vector includes a production dataset and a security dataset;

[0012] The acquisition of the multidimensional data vector of the smart factory includes:

[0013] Collect production and safety data from the smart factory and preprocess them;

[0014] The preprocessed data is fused together, and feature construction is performed with the goal of removing redundancy to generate a multidimensional data vector.

[0015] Furthermore, the step of inputting the multidimensional data vector into a deep reinforcement learning model to obtain a basic scheduling strategy specifically includes:

[0016] Obtain the number of tasks actually completed in the current cycle of the smart factory, the total number of tasks planned to be allocated in the current cycle, the cumulative idle time of key equipment, and the total online time of key equipment in the current cycle, and generate a multi-objective reward function;

[0017] The multidimensional data vector is used as the current state of the deep reinforcement learning model, and the action of each production resource unit is used as the corresponding action of the deep reinforcement learning model. The deep reinforcement learning model is trained according to the multi-objective reward function to generate a basic scheduling strategy.

[0018] Furthermore, training the deep reinforcement learning model based on the multi-objective reward function specifically includes:

[0019] The weights of the deep reinforcement learning model are initialized, and the last layer of the deep reinforcement learning model uses the Softmax activation function to ensure that the output follows a probability distribution; the result of the deep reinforcement learning model includes an input layer, several hidden layers, and an output layer.

[0020] Based on the current state, sample an action from the policy distribution and execute the corresponding production scheduling operation;

[0021] Based on the production results after the execution of the actions, the reward of the multi-objective reward function is calculated to comprehensively evaluate production efficiency, resource utilization and safety risks;

[0022] The policy gradient method is adopted to maximize the expected cumulative reward and directly optimize the policy network parameters.

[0023] Furthermore, the security risk score Represented as:

[0024]

[0025] Among them, L t L represents the total length of the task queue in the current system. max Indicates the maximum task queue length supported by the system; T t T represents the current temperature of the critical equipment as collected by the temperature sensor; T0 represents the recommended operating temperature of the equipment; T crit Indicates the critical superheat temperature; B t Indicates the current total upload network bandwidth usage of the factory; B max Indicates the maximum bandwidth capacity of this network port; H t H0 indicates the humidity level in the workshop; H0 indicates the recommended humidity value; H tol Indicates the allowable deviation tolerance value; α, β, γ, δ are hyperparameters;

[0026] All components are normalized to the interval [0,1]. The highest value indicates the greatest security risk.

[0027] Furthermore, S3 also includes:

[0028] If the security risk score exceeds a preset security threshold, an early warning mechanism will be triggered to send an alarm to the operator; the alarm includes at least one of the following: adjusting the priority or scheduling order of production tasks, performing equipment checks or adjusting equipment load, or activating network security protection measures.

[0029] Furthermore, the step of online revising the basic scheduling strategy based on the security risk score to generate a joint optimization strategy specifically includes:

[0030] The basic scheduling policy is modified by introducing equipment scheduling constraint strength and safety risk score as weighting factors to generate a joint optimization policy; wherein the joint optimization policy is also regularized based on the safety risk score, and the maximum safety risk score results in the maximum regularization penalty.

[0031] Furthermore, the ultimate optimization objective of the joint optimization strategy is:

[0032]

[0033] Among them, R t For the reward value, π' t For the joint optimization strategy, Ω(π') t ) represents the regularization operation.

[0034] Furthermore, the smart factory feedback data includes average equipment load rate and production task overdue rate;

[0035] The joint optimization strategy is sent to the smart factory so that the smart factory executes the joint optimization strategy according to the objective optimization function and collects feedback data from the smart factory in real time, specifically including:

[0036] The joint optimization strategy is continuously fine-tuned according to the objective optimization function to generate a new strategy. At the same time, a continuous regularization term is added during the fine-tuning process to encourage the strategy to maintain continuous change during adjustment, prevent drastic scheduling changes caused by small feedback fluctuations, and improve the overall stability of the system.

[0037] After fine-tuning, the new strategy will be used for actual production control.

[0038] Furthermore, the step of continuously fine-tuning the joint optimization strategy according to the objective optimization function to generate a new strategy specifically includes:

[0039] If the difference between the average load rate of the equipment at two consecutive time points is found to be more than 10%, it indicates that some equipment is overloaded. The system will then automatically reduce the scheduling probability of the corresponding equipment to 80% of the original value.

[0040] If the overdue rate of a task is detected to exceed 5%, the system will prioritize increasing the scheduling probability of the equipment related to the delayed task to 120% of the original value.

[0041] If the security risk score exceeds 0.7, all scheduling probabilities will be reduced to 90% of their original values ​​to enhance system defense stability and prioritize the retention of low-risk devices.

[0042] The beneficial technical effects of the present invention are at least as follows:

[0043] To address the shortcomings of existing technologies in production scheduling optimization and cybersecurity, this invention proposes an innovative intelligent production optimization scheme aimed at overcoming the deficiencies of traditional technologies in efficient production scheduling and security protection. First, this invention organically combines production scheduling with security protection, proposing an intelligent algorithm capable of real-time optimization of production scheduling, and integrating real-time security monitoring and emergency response mechanisms. This combination enables real-time monitoring of equipment and data security status during production while optimizing production scheduling. When the system detects risks such as network attacks or data breaches, it can immediately adjust production scheduling strategies to prevent potential security threats from impacting production efficiency. Second, this invention introduces an adaptive learning mechanism, enabling the production scheduling system to intelligently adjust based on real-time production data and network security status. By analyzing factors such as equipment load and production task priorities in real time, the system can dynamically adjust task allocation and equipment operation strategies during production, significantly improving the flexibility and responsiveness of the production line. This method not only improves production efficiency but also effectively ensures the security of the production system in high-risk environments. Furthermore, this invention proposes a highly efficient security protection algorithm that, through dynamic evaluation and adaptive adjustment mechanisms, ensures effective network security between devices in a smart factory environment without affecting the real-time performance and efficiency of production scheduling. This innovative technology overcomes the problem of production scheduling and safety protection not being able to work together in existing technologies, enabling smart factories to achieve comprehensive protection of data security and equipment stability while improving production efficiency.

[0044] Through the above innovations, this invention can achieve dual optimization of production scheduling and safety protection in smart factories, providing a safer, more flexible and efficient solution for the manufacturing industry in the era of the Industrial Internet. Attached Figure Description

[0045] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0046] Figure 1 This is a flowchart of a deep learning-based industrial internet smart factory production optimization method according to the present invention. Detailed Implementation

[0047] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0048] In one embodiment, Figure 1 As shown, a deep learning-based method for optimizing production in an industrial internet smart factory is provided, including the following steps:

[0049] S1. Obtain a multidimensional data vector of the smart factory; wherein the multidimensional data vector includes a production dataset and a safety dataset.

[0050] Specifically, in the production scheduling and safety protection processes of a smart factory, data is the most fundamental and crucial input. To ensure the effective operation of the production scheduling model and safety assessment model in subsequent steps, the goal of this step is to provide high-quality, well-structured input data for these models through efficient data acquisition and preprocessing. Data sources in a smart factory are highly diverse, including operational data from production equipment, environmental data collected by sensors, and network traffic data. Therefore, this step not only needs to handle the various data types involved in the production process but also needs to ensure the timeliness and completeness of the data, providing a reliable foundation for model training and optimization in subsequent steps.

[0051] First, data collection is fundamental to a smart factory. This step involves collecting production and cybersecurity data in real time using Internet of Things (IoT) devices and sensor networks. Specifically:

[0052] Production data includes, but is not limited to, equipment operating status (such as load and speed), temperature and humidity, production progress, energy consumption, etc., from smart sensors and production equipment.

[0053] Security data includes communication status between devices, bandwidth usage, latency, potential attack events, etc., and comes from the network security monitoring system.

[0054] This data is obtained through real-time monitoring of sensors and equipment deployed throughout the factory. The operational data and safety status of each device are uploaded in real time to a centralized processing system and connected to the production control system.

[0055] Furthermore, the collected raw data typically needs to be preprocessed before it can be used in the subsequent model training process. Especially in the industrial internet environment, the data is diverse, large in volume, and noisy, requiring appropriate processing and transformation through feature engineering to facilitate further model analysis.

[0056] Denoising and data cleaning: Remove outliers, duplicates, and missing data to ensure data accuracy.

[0057] Standardization and Normalization: Different types of data (such as temperature, humidity, energy consumption, and bandwidth) may have different ranges and units. To avoid the impact of unit and scale differences on subsequent models, data standardization is necessary to ensure that all data have a uniform scale. For production data such as temperature and current, Z-score standardization is used; for proportional data such as network bandwidth, min-max normalization is used. These processes ensure that data from different dimensions can be compared and analyzed within the same computational framework.

[0058] For example, by standardizing production data such as temperature and humidity, and security data such as bandwidth and latency, they can be merged into a standardized data vector x. t And for use in subsequent steps:

[0059] x t ={D t ,S t} (1)

[0060] Where: D t This represents the production dataset, including equipment operation data, temperature and humidity, load, current, voltage, etc.; S t This represents a security dataset, including cybersecurity data such as bandwidth, latency, and data traffic; x t This is the final fused data vector, used in subsequent steps.

[0061] Furthermore, to ensure that subsequent steps can consistently utilize both production and security data, we merge these two types of data into a multi-dimensional data vector x. t This information is then used in subsequent models. This process ensures that production scheduling and safety issues are considered simultaneously, rather than being handled separately.

[0062] Data fusion is not just about simply piecing together production and security data; it involves merging, cleaning, and standardizing the two types of data to ensure their validity and consistency.

[0063] Further, after data preprocessing, we need to perform feature selection and feature construction. The goal of feature selection is to remove redundancy and retain the most meaningful features. The purpose of feature construction is to generate new features to improve model performance. For example, based on equipment load and production tasks, a new feature, "equipment load rate," can be constructed, which better reflects the operating status of the equipment. Common feature construction methods include:

[0064] A moving average based on a time window, for example, taking the average current over the past 5 minutes;

[0065] The construction of hysteresis features is used to capture patterns in time series (such as "average current over the first 5 minutes").

[0066] By constructing these additional features, we can capture potential patterns in the production environment, further improving the prediction and optimization capabilities of subsequent models.

[0067] Furthermore, the processed data needs to be stored in a suitable format and transmitted to subsequent systems. For efficient data access, the data will be stored in a distributed database; common storage methods include time-series databases (such as InfluxDB) or relational databases (such as PostgreSQL). The stored data can then be transmitted to the model training system and scheduling system via API interfaces, ensuring the system's efficiency and real-time performance.

[0068] In this step, data preprocessing goes beyond traditional data cleaning and standardization. Through innovative data processing methods, it solves the long-standing problem of data separation in production scheduling and safety protection. In particular, the integration of production and safety data ensures data consistency and provides clear and efficient input for subsequent steps. This provides a solid data foundation for subsequent joint optimization, avoids the data silo problem of traditional methods, and provides strong support for scheduling optimization and safety protection in smart factories.

[0069] S2. Input the multidimensional data vector into the deep reinforcement learning model to obtain the basic scheduling strategy.

[0070] Specifically, in the production scheduling of a smart factory, the core objective is to dynamically optimize resource allocation based on real-time production data and task requirements to maximize production efficiency and reduce resource waste. Unlike traditional production scheduling methods, this patented solution introduces additional factors such as network security and equipment health status, enabling production scheduling to not only focus on production efficiency but also respond in real time to factors such as equipment failures and network attacks, thereby improving the overall operational security and production efficiency of the factory.

[0071] In such a complex environment, traditional optimization methods and static scheduling strategies are no longer sufficient. Deep reinforcement learning (DRL), as a technique capable of self-learning and optimizing decisions in dynamic environments, perfectly meets this need. Therefore, this step employs deep reinforcement learning to train the production scheduling model, enabling it to make optimal scheduling decisions in dynamically changing production and safety environments.

[0072] This step uses a Deep Reinforcement Learning (DRL) framework for production scheduling optimization. The core model is a Deep Neural Network (DNN) with a multi-layer fully connected network structure, where each layer uses the ReLU activation function. This structure can capture the complex nonlinear relationships in production scheduling. The model architecture is designed as follows:

[0073] Input layer: fused vector x from step 1t When input into the model, the vector has a multidimensional feature number dimension, which can reflect the status of factory equipment and network.

[0074] Multiple hidden layers: Several fully connected hidden layers are used, employing the ReLU activation function for non-linear transformation to progressively extract useful features from the data. The number of neurons in each layer and the number of layers are selected experimentally; typically, deeper networks are used to enhance the model's expressive power.

[0075] Output layer: The final output is the production scheduling strategy π. t That is, based on the input data at the current moment, it determines how to schedule production tasks, allocate equipment, and prioritize tasks.

[0076] To ensure that the scheduling strategy improves production efficiency while also considering equipment utilization and system security, this invention proposes the following multi-objective reward function:

[0077]

[0078] Among them, N done,t : The number of tasks actually completed during this period; N total,t The total number of tasks planned for this period; Indicates the percentage of production tasks completed; T idle,t : Cumulative idle time of key equipment, collected from industrial control systems; T total,t Total online time of key equipment within a cycle; Indicates the resource idle ratio;

[0079] To adapt to the target preferences of different production scenarios, the system supports simple and controllable adjustment of hyperparameters α1 and β1, with the following adjustment rules:

[0080] When the primary goal of the scheduling strategy is to complete production tasks quickly (e.g., with tight delivery cycles), α1 = 0.6 and β1 = 0.2 can be set.

[0081] When the system focuses more on resource utilization and cost control (such as nighttime operation and low load periods), α1 = 0.4 and β1 = 0.4 can be set.

[0082] The default configuration is α1 = 0.5, β1 = 0.3, which remains unchanged without human intervention;

[0083] The parameters can be set by the operation and maintenance personnel in the scheduling module via configuration files, or they can be dynamically written by the upper-level business system through the API interface.

[0084] This strategy enables flexible switching between "efficiency first" and "energy saving first" scheduling optimization objectives, ensuring the adaptability and stability of the system after deployment.

[0085] Furthermore, during the training process of reinforcement learning, the model will continuously interact with the production environment and update its scheduling strategy π based on the feedback of the reward function. t Unlike traditional methods, the π in this invention... t Instead of directly outputting a definite action, it outputs the probability distribution of each action in the action space, i.e., π. t (i) represents the probability of choosing action i (such as scheduling to device i) at time t.

[0086] Furthermore, action space and policy representation:

[0087] Action space definition: set of actions Each action a i This corresponds to a production resource unit (such as a piece of equipment or a production line).

[0088] Strategy definition: π t (a i |x t ) indicates that in state x t Choose action a i The probability of each action is calculated, and the policy network outputs a probability vector for all actions, which is then normalized using Softmax.

[0089] Training process adjustments:

[0090] Initialization: Initialize the weights of the deep neural network. The last layer of the network uses the Softmax activation function to ensure that the output follows a probability distribution.

[0091] Choose an action: based on the current state x t From the strategy distribution π t Sample an action a t , and execute the corresponding production scheduling operations.

[0092] Feedback and reward: based on action a t Calculate the reward R based on the production results after execution. t A comprehensive assessment of production efficiency, resource utilization, and safety risks is conducted.

[0093] Policy update: Employ policy gradient methods (such as REINFORCE, PPO, etc.) to maximize the expected cumulative reward. Optimize the policy network parameters directly.

[0094] Convergence determination: By monitoring the cumulative reward curve and policy stability index on the validation set, determine whether the model has converged to the optimal policy.

[0095] Furthermore, experience replay: randomly sample past interaction records to break sample correlation.

[0096] Target network (if using an Actor-Critic structure): mitigates policy fluctuations and improves training stability.

[0097] L2 regularization and Dropout: Prevent overfitting and improve model generalization performance.

[0098] Adaptive optimizer (Adam): Automatically adjusts the learning rate to accelerate convergence and avoid local optima traps.

[0099] After training, the output policy π t The scheduling actions can be flexibly adjusted at any time based on the real-time status.

[0100] S3. The production scheduling optimization strategy and multi-dimensional data vector are input into the constructed deep learning model, and a safety risk score is generated.

[0101] Specifically, in the production scheduling process of a smart factory, the core task of safety risk assessment is to evaluate potential safety risks during production based on real-time production and safety data. This includes factors such as equipment health status, equipment load during production task execution, and potential cyberattacks. By conducting real-time risk assessments during production scheduling, the model can provide timely warnings, preventing production stoppages or safety accidents caused by equipment failures or safety issues.

[0102] To enhance the scheduling system's real-time awareness of production safety, we propose a low-complexity, high-stability safety risk scoring method. This method assigns a risk score to the current state at each scheduling time t, as shown in the following formula:

[0103]

[0104] The definitions of each item are as follows:

[0105] L t : Total length of the task queue in the current system, source: Task scheduling system API;

[0106] L max : The maximum task queue length supported by the system (static configuration parameter);

[0107] T t Temperature of the critical equipment as collected by the temperature sensor (unit: °C), source: built-in sensor of the equipment;

[0108] T0: Recommended operating temperature of the equipment (e.g., 60℃), source: equipment manufacturer's instructions;

[0109] T crit Critical superheat temperature (e.g., 85℃), source: standard specifications;

[0110] B t Total upload network bandwidth usage in the factory (unit: Mbps), source: real-time port traffic of industrial switches;

[0111] B max The maximum bandwidth capacity of this network port (e.g., 100Mbps);

[0112] H t Workshop humidity (e.g., 65% RH), source: environmental sensor;

[0113] H0: Recommended humidity value (e.g., 50% RH);

[0114] H tol Tolerance value for permissible deviation (e.g., ±10% RH);

[0115] All components are normalized to the interval [0,1]. The larger the value, the higher the security risk.

[0116] Recommendations for setting hyperparameters α, β, γ, δ:

[0117] These weights determine the system's sensitivity to risks across different dimensions and need to be optimized based on the specific scenario. The following suggestions are offered:

[0118] Initial setup suggestions:

[0119] α = 0.3 (queue congestion has the greatest impact on efficiency)

[0120] β = 0.25 (Equipment temperature is an important signal of early failure)

[0121] γ = 0.25 (Network congestion will affect task upload and scheduling instructions)

[0122] δ = 0.2 (Cumulative impact of environmental conditions on long-term operation)

[0123] Furthermore, a dynamic adjustment mechanism is recommended:

[0124] Daily statistics of all In which dimensions did the samples with scores >0.8 exceed the standard?

[0125] If most of the anomalies are due to temperature abnormalities, then increase β to 0.3, and vice versa.

[0126] Use a sliding window (e.g., the past 72 hours) to statistically analyze the distribution of risk dimensions, and periodically reset the four weights;

[0127] If the system supports automatic parameter tuning, it can be self-tuned using a simple Bayesian optimization algorithm or a rolling regression method.

[0128] When security risk score After calculation, the system will determine whether to trigger a safety alert based on the set threshold. If the score exceeds the preset safety threshold, an alert mechanism will be triggered, sending an alarm to the operator. The alarm content may include:

[0129] It is recommended to adjust the priority of production tasks or the order of task scheduling;

[0130] It is recommended to check the equipment or adjust the equipment load;

[0131] If the current total upload network bandwidth usage of the factory is significantly abnormal, such as approaching the maximum bandwidth capacity of the network port for several consecutive days, network security protection measures should be activated immediately.

[0132] S4. Based on the security risk score, the basic scheduling strategy is modified online to generate a joint optimization strategy.

[0133] In deep reinforcement learning (step two), we have trained the basic scheduling policy π. t It possesses some global optimization capabilities, but lacks fine-grained response capabilities to current real-time conditions (such as security risks, equipment fluctuations, and environmental disturbances). The security score output in step three... It can reflect the system's risk status in real time. Therefore, to make the scheduling strategy more robust and deployable, a joint optimization strategy π' combining security scoring and task constraints is proposed. t , for π t Make fine-tuning at the decision space level to adapt it to the real-time industrial production environment.

[0134] Final scheduling policy π' t Based on the original strategy π t An optimized version that, without changing the training model structure and parameters, introduces two weight factors. and c i,t This enables "online correction" of the strategy output. The optimization formula is as follows:

[0135]

[0136] Where, π t (i) represents the original allocation probability of the scheduling strategy for device i; This is the system's current security risk score, output by the model in step three; c i,t ∈[0,1] represents the equipment scheduling constraint strength, for example, when the equipment is under maintenance, c i,t ≈1, when normal scheduling is possible, c i,t ≈0; λ1, λ2 are adjustment coefficients for scheduling intensity, generally set to 0.2~0.5; Z t It is a normalization factor that makes all π' t(i) constitutes a valid probability distribution.

[0137] For example, at a certain time t: π t = [0.5, 0.3, 0.2]; c i,t =[0.1,0.4,0.8]; λ1=λ2=0.3.

[0138] After factor adjustment, the intermediate results for the three devices are approximately 0.846, 0.756, and 0.544, respectively. Multiplying by the original π... t After normalization, we get π' t .

[0139] To prevent excessive scheduling concentration under high-risk conditions, a regularization term is also introduced:

[0140]

[0141] Where η is the regularity coefficient, typically taken as 0.1 to 0.3; π' t (i) 2 Punishment for centralized task scheduling behavior; The larger the value, the stronger the regular expression penalty.

[0142] The ultimate optimization goal during deployment is:

[0143]

[0144] For example, if π' t (i)>0.6 and Then the scheduling weight can be forcibly multiplied by 0.8 and renormalized to distribute the scheduling pressure.

[0145] S5. Collect feedback data from the smart factory and design a target optimization function based on the safety risk score; send the joint optimization strategy to the smart factory so that the smart factory executes the joint optimization strategy according to the target optimization function and collects feedback data from the smart factory in real time.

[0146] Specifically, in the dynamic production process of a smart factory, production task progress, equipment load, and overall operational safety are constantly changing. Any delay, overload, or anomaly may lead to production interruption or resource waste. Therefore, this invention utilizes the joint optimization strategy π' obtained through prior training. t Based on this, a real-time feedback adjustment mechanism is introduced to form a closed-loop, continuous production scheduling optimization process to ensure the system operates continuously, efficiently, and safely.

[0147] This step involves real-time collection of key system status indicators, combined with the current strategy output, to design a fine-tuning mechanism to dynamically correct the scheduling strategy, ultimately forming a practically executable strategy π″. tIt also enables real-time control and task scheduling of production resources.

[0148] To achieve this goal, this step revolves around the following core processes:

[0149] First, in each feedback cycle t, the system collects core feedback indicators in real time to form a feedback vector e. t This invention defines the requirement for only two concise yet core feedback data:

[0150] Equipment average load rate U t Data is collected through industrial control systems (such as PLCs or equipment sensors), and the unit is %.

[0151] Production task overdue rate D t This is automatically calculated by the Production Execution System (MES), and the unit is %.

[0152] Both of these metrics are native data from within the system and can be collected directly, avoiding the introduction of additional interface complexity. Simultaneously, they are combined with the real-time security risk score output in step three. The system can comprehensively perceive the current health status of the production environment.

[0153] Secondly, the system is based on the set expected state criteria. (Where U0 = 75%, D0 = 5%) Evaluate the current deviation and design the following real-time optimization objective function:

[0154]

[0155] Where, θ, μ controls the impact of equipment load, task latency, and safety risk on the overall evaluation value, with default configurations of α = 0.5, β = 0.3, and γ = 0.2. This formula innovatively integrates production performance and safety risk into a unified scoring system, ensuring a balance between the two through dynamic optimization.

[0156] Next, based on the current U t D t , Calculated For the joint optimization strategy π' t Perform continuous fine-tuning to generate a new strategy π″ t The fine-tuning mechanism is detailed below:

[0157] If U is found t If the value is more than 10% higher than U0, it indicates that some devices are overloaded. The system will then automatically reduce the scheduling probability of the corresponding device to 80% of its original value. For example, if the original scheduling probability of a device is 0.3, it will be adjusted to 0.24.

[0158] If the task overdue rate D is detectedt If the probability exceeds 5%, the system will prioritize increasing the scheduling probability of equipment related to the delayed task to 120% of the original value, that is, adjusting the original value of 0.2 to 0.24, and renormalizing other actions proportionally.

[0159] If the overall security risk score If the probability exceeds 0.7, all scheduling probabilities will be compressed to 90% of their original value to enhance system defense stability and prioritize the retention of low-risk devices.

[0160] For example, within a certain period, U t =88%, D t =8% and The system will simultaneously trigger three fine-tuning logics: downgrading the weight of all high-load devices, increasing the priority of delayed tasks, and compressing the scheduling weight in high-risk situations, thereby forming a new strategy that balances efficiency and security.

[0161] To further ensure the stability of the system over continuous time, this invention introduces a continuity regularization term during the strategy adjustment process:

[0162]

[0163] Where η is the regularization coefficient, with a suggested initial value of 0.1, which can be dynamically adjusted according to the system's response sensitivity. This regularization term encourages the strategy to remain continuously changing during adjustment, preventing drastic scheduling changes caused by small feedback fluctuations and improving the overall stability of the system.

[0164] Finally, the objective function is adjusted in real time to maximize the following comprehensive index:

[0165]

[0166] After fine-tuning, the system will use π″ t To conduct actual production control, including:

[0167] Dynamically redistribute tasks to devices with lower loads;

[0168] Reduce the workload of high-load equipment or temporarily switch to a backup machine;

[0169] Based on real-time scheduling results, adjust the equipment operating mode (frequency reduction, standby) appropriately to prevent overload;

[0170] Delayed tasks are prioritized and processed to ensure that the overall delivery cycle is not affected.

[0171] Finally, after executing the control action, the system immediately collects new feedback data e. t+1 And then proceed through the same process to the next adjustment cycle, forming the following closed loop:

[0172] π' t →e t →π″ t →Control Execution→e t+1 →π″ t+1 (10)

[0173] Through the aforementioned closed-loop mechanism, this invention utilizes a lightweight, real-time feedback fine-tuning strategy to continuously adapt to changing environments, enabling a truly efficient, safe, and adaptive production scheduling optimization system in smart factories.

[0174] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application.

[0175] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the system described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0176] In the description of this application, it should be noted that the terms "upper" and "lower" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use. They are only used to facilitate the description of this application and to simplify the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0177] In the description of this application, it should also be noted that, unless otherwise expressly specified and limited, the terms "set up," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for optimizing production in an industrial internet smart factory based on deep learning, characterized in that, The method includes: S1. Obtain the multi-dimensional data vector of the smart factory; S2. Input the multidimensional data vector into the deep reinforcement learning model to obtain the basic scheduling strategy; S3. Input the production scheduling optimization strategy and multi-dimensional data vector into the constructed deep learning model to generate a safety risk score; S4. Based on the security risk score, the basic scheduling strategy is corrected online to generate a joint optimization strategy; S5. Collect feedback data from the smart factory and design a target optimization function based on the safety risk score; send the joint optimization strategy to the smart factory so that the smart factory executes the joint optimization strategy according to the target optimization function and collects feedback data from the smart factory in real time.

2. The method for optimizing production in an industrial internet smart factory based on deep learning according to claim 1, characterized in that, in, The multidimensional data vector includes production datasets and security datasets; The acquisition of the multidimensional data vector of the smart factory includes: Collect production and safety data from the smart factory and preprocess them; The preprocessed data is fused together, and feature construction is performed with the goal of removing redundancy to generate a multidimensional data vector.

3. The method for optimizing production in an industrial internet smart factory based on deep learning according to claim 1, characterized in that, The step of inputting the multidimensional data vector into a deep reinforcement learning model to obtain a basic scheduling strategy specifically includes: Obtain the number of tasks actually completed in the current cycle of the smart factory, the total number of tasks planned to be allocated in the current cycle, the cumulative idle time of key equipment, and the total online time of key equipment in the current cycle, and generate a multi-objective reward function; The multidimensional data vector is used as the current state of the deep reinforcement learning model, and the action of each production resource unit is used as the corresponding action of the deep reinforcement learning model. The deep reinforcement learning model is trained according to the multi-objective reward function to generate a basic scheduling strategy.

4. The method for optimizing production in an industrial internet smart factory based on deep learning according to claim 3, characterized in that, The step of training the deep reinforcement learning model according to the multi-objective reward function specifically includes: The weights of the deep reinforcement learning model are initialized, and the last layer of the deep reinforcement learning model uses the Softmax activation function to ensure that the output follows a probability distribution; the result of the deep reinforcement learning model includes an input layer, several hidden layers, and an output layer. Based on the current state, sample an action from the policy distribution and execute the corresponding production scheduling operation; Based on the production results after the execution of the actions, the reward of the multi-objective reward function is calculated to comprehensively evaluate production efficiency, resource utilization and safety risks; The policy gradient method is adopted to maximize the expected cumulative reward and directly optimize the policy network parameters.

5. The method for optimizing production in an industrial internet smart factory based on deep learning according to claim 1, characterized in that, The security risk score Represented as: Among them, L t L represents the total length of the task queue in the current system. max Indicates the maximum task queue length supported by the system; T t T represents the current temperature of the critical equipment as collected by the temperature sensor; T0 represents the recommended operating temperature of the equipment; T crit Indicates the critical superheat temperature; B t Indicates the current total upload network bandwidth usage of the factory; B max Indicates the maximum bandwidth capacity of this network port; H t H0 indicates the humidity level in the workshop; H0 indicates the recommended humidity value; H tol Indicates the allowable deviation tolerance value; α, β, γ, δ are hyperparameters; All components are normalized to the interval [0,1]. The highest value indicates the greatest security risk.

6. The method for optimizing production in an industrial internet smart factory based on deep learning according to claim 5, characterized in that, S3 further includes: If the security risk score exceeds a preset security threshold, an early warning mechanism will be triggered to send an alarm to the operator; the alarm includes at least one of the following: adjusting the priority or scheduling order of production tasks, performing equipment checks or adjusting equipment load, or activating network security protection measures.

7. The method for optimizing production in an industrial internet smart factory based on deep learning according to claim 1, characterized in that, The basic scheduling strategy is corrected online based on the security risk score to generate a joint optimization strategy, specifically: The basic scheduling policy is modified by introducing equipment scheduling constraint strength and safety risk score as weighting factors to generate a joint optimization policy; wherein the joint optimization policy is also regularized based on the safety risk score, and the maximum safety risk score results in the maximum regularization penalty.

8. The method for optimizing production in an industrial internet smart factory based on deep learning according to claim 7, characterized in that, The ultimate optimization objective of the joint optimization strategy is: Among them, R t For the reward value, π' t For the joint optimization strategy, Ω(π') t ) represents the regularization operation.

9. The method for optimizing production in an industrial internet smart factory based on deep learning according to claim 1, characterized in that, The smart factory feedback data includes average equipment load rate and production task overdue rate; The joint optimization strategy is sent to the smart factory so that the smart factory executes the joint optimization strategy according to the objective optimization function and collects feedback data from the smart factory in real time, specifically including: The joint optimization strategy is continuously fine-tuned according to the objective optimization function to generate a new strategy. At the same time, a continuous regularization term is added during the fine-tuning process to encourage the strategy to maintain continuous change during adjustment, prevent drastic scheduling changes caused by small feedback fluctuations, and improve the overall stability of the system. After fine-tuning, the new strategy will be used for actual production control.

10. A method for optimizing production in an industrial internet smart factory based on deep learning, as described in claim 9, characterized in that, The step of continuously fine-tuning the joint optimization strategy according to the objective optimization function to generate a new strategy specifically includes: If the difference between the average load rate of the equipment at two consecutive time points is found to be more than 10%, it indicates that some equipment is overloaded. The system will then automatically reduce the scheduling probability of the corresponding equipment to 80% of the original value. If the overdue rate of a task is detected to exceed 5%, the system will prioritize increasing the scheduling probability of the equipment related to the delayed task to 120% of the original value. If the security risk score exceeds 0.7, all scheduling probabilities will be reduced to 90% of their original values ​​to enhance system defense stability and prioritize the retention of low-risk devices.

Images