Glass preparation raw material ratio automatic control system and method

Abstract

The invention relates to the technical field of glass, in particular to a glass preparation raw material ratio automatic control system and method, and the system comprises a formula management module, a data processing module, an initial ratio determination module, a state information acquisition module, an RBC model establishment module, a raw material ratio determination module, a raw material ratio control module and an abnormity management module. The raw material formula is managed through the formula management module; the data processing module collects and preprocesses historical data; the initial ratio determination module and the RBC model establishment module are combined with expert experience to generate an initial ratio and store the initial ratio into an experience playback pool; the state information acquisition module acquires current state information according to an equipment request, and the raw material ratio determination module generates a ratio behavior through a TD3 model; the control module is used for generating and storing an experience sample for using a TD3 model so as to optimize a matching behavior; and the exception management module is responsible for monitoring and processing exceptions. The accuracy of system control is effectively improved, and the glass production efficiency and the product quality are improved.

Description

Technical Field

[0001] The invention relates to the technical field of glass, and in particular to an automatic control system and method for glass preparation raw material ratio. Background Art

[0002] As a key component of modern industrial and building materials, the quality and performance of glass are directly related to the reliability and safety of the final product. In glass production, the accuracy of raw material ratio is the core factor to ensure the quality of glass. The traditional ratio method relies on manual weighing of raw materials, and then putting the prepared materials into the batching machine for mixing. However, since many raw materials are in powder or granular form, it is easy to inhale dust during manual batching, which not only threatens the health of workers, but also increases the risk and labor cost of production. In addition, there are many types of raw materials and huge quantities. It is difficult to manage manual batching accurately, and batching errors are prone to occur, which not only affects the quality of glass, but also increases management costs.

[0003] With the continuous development of industrial automation technology, many solutions have been proposed for automatic control systems in the field of glass manufacturing. These systems have shown significant advantages in improving production efficiency, optimizing resource allocation and ensuring product quality. For example, high-precision process control can be achieved and production stability can be improved through real-time data monitoring. However, in the process of raw material proportioning in the glass production line, the actual situation of the raw materials may not be completely consistent with the set results, or all influencing factors are not fully considered, such as changes in the system environment or process conditions, which may lead to unsatisfactory proportioning effects or even negative effects. In addition, the processing solutions for multiple recipe changes are not considered. If the manpower and time required to frequently update and optimize the raw material proportioning control model will further increase the maintenance cost of the system, it will lack sufficient accuracy.

[0004] Therefore, an automatic control system and method for glass preparation raw material ratio are proposed. Summary of the invention

[0005] The purpose of the present invention is to provide an automatic control system and method for glass preparation raw material ratio, which manages raw material formula through formula management module; data processing module collects and preprocesses historical data; initial ratio determination module calculates glass raw material demand in combination with expert experience; state information acquisition module obtains current state information according to device request; RBC model establishment module inputs glass raw material demand into device according to current state information to obtain first experience sample and stores it into experience playback pool; raw material ratio determination module generates current ratio behavior through TD3 model; control module inputs current ratio behavior into device, calculates ratio reward, generates and stores second experience sample, and uses TD3 model in combination with first experience sample to optimize ratio strategy; exception management module is responsible for monitoring and handling exceptions. The above system and method effectively improve the accuracy of system control.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] An automatic control system for glass preparation raw material ratio, comprising:

[0008] Recipe management module, used to add and delete raw material recipes for glass preparation;

[0009] A data processing module, used for collecting raw material ratio historical data, adding the raw material formula to the raw material ratio historical data, and performing data preprocessing to obtain a first data set;

[0010] An initial ratio determination module is used to obtain the glass raw material demand according to the raw material formula and expert experience;

[0011] A status information acquisition module, which acquires current status information of the device according to the first data set;

[0012] An RBC model building module, configured to generate a first experience sample according to the current state information and the glass raw material demand, and store the first experience sample in an experience playback pool;

[0013] A raw material ratio determination module, using the current state information and the current raw material ratio information and determining the current ratio behavior according to the TD3 model;

[0014] A raw material ratio control module inputs the current ratio behavior into the device to obtain a ratio reward and next batch state information; combines the current state information, the current ratio behavior, the ratio reward and the next batch state information to obtain a second experience sample, and stores the second experience sample in the experience replay pool; obtains a trained TD3 model according to the experience replay pool, and obtains the optimized raw material ratio according to the trained TD3 model;

[0015] The exception management module is used to detect and handle abnormal situations in the batching process.

[0016] Furthermore, the initial ratio determination module includes:

[0017] The expert experience includes glass raw material information and other raw material information, wherein the glass raw material information includes raw material conversion rate, raw material purity and raw material volatilization rate, and the other raw material information includes the content of additives;

[0018] The raw material formula includes the target ratio of the raw materials and the total mass of the glass to be prepared;

[0019] According to the expert experience and the raw material formula, the demand for the glass raw materials is calculated and expressed as:

[0020]

[0021] Among them, O i is the demand for the i-th glass raw material, P i is the target ratio of the i-th raw material, T is the total mass of the glass prepared, C i is the purity of the i-th raw material, E i is the conversion rate of the ith raw material, V i is the volatilization rate of the i-th raw material.

[0022] Furthermore, the current status information includes:

[0023] The device status includes the device on status and the device off status;

[0024] Environmental information includes the temperature, humidity, and pressure of the production environment;

[0025] The raw material formula information includes the raw material formula and a digital serial number, and the digital serial number corresponds to the raw material formula one by one;

[0026] The recipe change information includes a recipe non-change flag and a recipe change flag.

[0027] Furthermore, the raw material ratio determination module includes:

[0028] The current raw material ratio information includes n kinds of glass raw material ratios and j kinds of additive contents. The current raw material ratio information is expressed as:

[0029] A={p1,p2,…p n ,k1,k2…k j};

[0030] Where A is the current raw material ratio information, p1, p2, ... p nare the first glass raw material ratio, the second glass raw material ratio and the nth glass raw material ratio, k1, k2…k j They are the first additive content, the second additive content and the jth additive content respectively;

[0031] Inputting the current state information into a strategy network, wherein the strategy network generates a first raw material ratio according to the current raw material ratio information;

[0032] Euler noise is added to the first raw material ratio to obtain the current ratio behavior, which is expressed as:

[0033] a=m(s)+noise;

[0034] Wherein, a is the current ratio behavior, m(s) is the first raw material ratio under the current state information s, and noise is the Euler noise.

[0035] Furthermore, the matching reward includes:

[0036] According to the actual ratio of raw materials and the target ratio of raw materials, the basic reward R is calculated. base , the basic reward function is expressed as:

[0037]

[0038] Among them, P actual,i is the actual ratio of the i-th raw material, P target,i is the target ratio of the i-th raw material;

[0039] According to the actual ratio of the raw materials and the current state information, the volatility reward R is calculated. stability ;

[0040] If the recipe change information is 0, it means that the recipe has not changed, and the volatility reward R stability is equal to the sum of the variances of the actual ratios of n raw materials; if the formula change information is 1, it is a formula change flag, and the volatility reward R stability Equal to 0, the volatility reward function is expressed as:

[0041]

[0042] Wherein, Var is the variance of the actual proportion of the i-th raw material, O change For recipe change information, O change =0 means the formula has not been changed, O change =1 is the formula change sign.

[0043] Furthermore, the calculation method of the matching reward is to obtain the matching reward by weighted calculation based on the basic reward, the volatility reward, equipment energy consumption and raw material loss, which is expressed as:

[0044] R=-w1×R base -w2×R stability -w3×E efficiency -w4×C waste ;

[0045] Among them, R is the matching reward, E efficiency is the energy consumption of the equipment, C waste is the raw material loss, w1, w2, w3 and w4 are weight factors of the basic reward, the volatility reward, the equipment energy consumption and the raw material loss respectively.

[0046] Furthermore, the deep neural network training process of the TD3 model includes:

[0047] The deep neural network includes a policy network, a first value network, a second value network, a target policy network, a first target value network, and a second target value network;

[0048] Randomly extract a batch of the first experience samples and the second experience samples from the experience replay pool;

[0049] According to the extracted experience samples, using the first target value network and the second target value network to respectively calculate a first target Q value and a second target Q value, wherein the target Q value is the minimum value between the first target Q value and the second target Q value;

[0050] Updating parameters of the first value network and the second value network by minimizing the mean square error between the actual Q values ​​of the first value network and the second value network and the target Q value;

[0051] Updating the parameters of the policy network by maximizing the Q value of the first value network;

[0052] Every q steps, the parameters of the target policy network, the first target value network and the second target value network are updated according to the parameters of the policy network, the first value network and the second value network using a soft update method.

[0053] Furthermore, the first experience sample generating step includes:

[0054] Step S20, initializing the experience replay pool, randomly extracting data including all recipes according to the first data set, to obtain a second data set;

[0055] Step S21, obtaining the glass raw material demand according to the second data set and current state information;

[0056] Step S22, assigning the glass raw material demand to the current proportioning behavior, inputting the current proportioning behavior into the device, and obtaining the proportioning reward and the next batch status information;

[0057] Step S23, combining the current state information, the current matching behavior, the matching reward and the next batch of state information to obtain the first experience sample, and storing the first experience sample in the experience replay pool;

[0058] Step S24, if the first experience sample reaches the first threshold, the matching process is terminated, otherwise the next batch of state information is used as the current state information of the next batch of matching process;

[0059] Step S25, determining the recipe change information and the raw material recipe information according to the current state information. If a recipe non-change mark is obtained, it is not necessary to obtain the glass raw material demand, and repeat steps S22 to S24; if a recipe change mark is obtained, repeat steps S21 to S24.

[0060] Furthermore, the abnormal situation includes:

[0061] The ratio deviation is calculated based on the actual ratio of the raw materials and the target ratio of the raw materials, and is expressed as:

[0062]

[0063] Among them, Δp i is the ratio deviation of the i-th raw material, P actual,i is the actual ratio of the i-th raw material, P target,i is the target ratio of the i-th raw material;

[0064] If the matching deviation is less than a second threshold, the current matching behavior is not updated;

[0065] If the ratio deviation is greater than or equal to the second threshold, the first abnormality is automatically recorded;

[0066] If the ratio deviation still exceeds the second threshold after f time steps, notify manual intervention;

[0067] The raw material loss is calculated according to the actual ratio of the raw materials and the current ratio behavior, which is expressed as:

[0068] C waste,i =P i -P actual,i ;

[0069] Among them, C waste,i is the loss of the i-th raw material, P i is the current matching behavior of the ith type;

[0070] If the raw material loss is greater than a third threshold, a second abnormality is automatically recorded;

[0071] If the ratio deviation still exceeds the third threshold after g time steps, the manual intervention is notified.

[0072] A method for automatically controlling the ratio of raw materials for glass preparation, comprising:

[0073] Collecting the raw material formula for preparing the glass;

[0074] Collecting raw material ratio historical data, adding the raw material formula to the raw material ratio historical data, and performing data preprocessing to obtain a first data set;

[0075] According to the raw material formula and expert experience, the demand for glass raw materials is obtained;

[0076] Acquire current status information of the device according to the first data set;

[0077] Generate a first experience sample according to the current state information and the glass raw material demand, and store the first experience sample in an experience playback pool;

[0078] Using the current state information and the current raw material ratio information and according to the TD3 model to determine the current ratio behavior;

[0079] Input the current proportioning behavior into the device to obtain a proportioning reward and the next batch of state information; combine the current state information, the current proportioning behavior, the proportioning reward and the next batch of state information to obtain a second experience sample, and store the second experience sample in the experience replay pool; obtain a trained TD3 model according to the experience replay pool, and obtain the optimized raw material proportion according to the trained TD3 model;

[0080] Detect and handle anomalies in the batching process.

[0081] Compared with the prior art, the present invention has the following beneficial effects:

[0082] 1. In the process of glass raw material ratio, the traditional ratio system mainly relies on expert experience and manual adjustment. Although this method is simple to implement, its accuracy is limited. The demand for glass raw materials is calculated through expert experience, combined with the RBC (rule-based control) model, and preset rules are used to interact with the equipment to obtain ratio rewards and the next batch of status information. The first batch of experience samples is generated based on the information obtained from the interaction, which not only provides high-quality data support for subsequent optimization, but also accelerates the optimization process of the TD3 algorithm, significantly improving the accuracy of the ratio of glass raw materials.

[0083] 2. Traditional proportioning control systems often find it difficult to fully optimize proportioning schemes when faced with complex production environments and dynamically changing data, resulting in insufficient proportioning accuracy and waste of raw materials. By introducing the TD3 model and combining it with the environmental status monitoring system, it is possible to capture and evaluate environmental changes in production equipment in real time. The system uses a strategy network and a value network to optimize proportioning rewards, intelligently adjust the current raw material proportioning scheme, and ensure that accurate raw material proportioning can still be achieved in a complex environment, thereby improving overall production efficiency and resource utilization.

[0084] 3. During the production process, traditional systems are unable to respond to abnormal situations such as ratio deviation and raw material loss in a timely manner, which may affect the stability of production and product quality. In order to meet these challenges, the abnormal management module can monitor abnormal situations that may occur in the production process in real time, such as ratio deviation, raw material waste or equipment failure. The system can not only automatically record these abnormalities, but also provide manual assistance processing suggestions to ensure the continuity and stability of production, effectively improve the stability of product quality, and reduce production interruptions and resource waste caused by abnormalities. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 A schematic diagram of a process flow of an automatic control system for glass preparation raw material ratio provided by the present disclosure;

[0086] Figure 2 A schematic diagram of the process of the reinforcement learning model provided by the present disclosure;

[0087] Figure 3 A schematic diagram of a process for generating a first experience sample provided by the present disclosure;

[0088] Figure 4 A schematic diagram of the deep neural network training process provided by the present disclosure;

[0089] Figure 5 A schematic flow chart of an automatic control method for raw material ratio in glass preparation provided by the present invention. DETAILED DESCRIPTION

[0090] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0091] In the glass manufacturing industry, the raw material ratio is a key factor affecting product quality and production efficiency. The composition of glass is usually a mixture of multiple raw materials, such as silica sand, alkali metal oxides, calcium materials, etc. The raw material ratio directly determines the physical properties of glass, such as transparency, strength and heat resistance, so accurate ratio is crucial to the production of high-quality glass.

[0092] Traditional automatic control systems for glass preparation raw material ratios usually rely on manual experience to set up fixed feedback mechanisms. Although this method can meet basic production requirements, the limitations of manual experience will significantly affect the accuracy of the ratio when faced with a complex and dynamically changing production environment. This method may lead to instability in the raw material ratio, thereby affecting the quality of glass products and the consistency of production. In addition, with the improvement of production requirements and the complexity of formulas, traditional systems do not consider the need to adjust multiple formulas in real time, which limits their ability to achieve precise and intelligent control in modern production. Therefore, there is an urgent need for a new system and method to improve the accuracy of raw material ratios and adapt to the requirements of modern production for efficient, stable and intelligent control.

[0093] like Figures 1 to 5 As shown, the present invention provides an automatic control system and method for glass preparation raw material ratio, and the technical scheme is as follows:

[0094] Embodiment 1

[0095] This embodiment uses a glass factory as the experimental environment and adopts the hardware equipment of the traditional glass raw material ratio automatic control system, including industrial computers, PLCs, touch screens, batching controllers and weighing sensors. This embodiment aims to solve the shortcomings of the traditional glass preparation raw material ratio control system in dealing with complex and dynamic production environments, especially the challenges of dealing with multiple recipes, thereby improving the accuracy of raw material ratios.

[0096] like Figure 1 As shown, an automatic control system for glass preparation raw material ratio includes:

[0097] The recipe management module is used to add and delete raw material recipes for glass preparation.

[0098] The raw material formula refers to the specific proportions of various raw materials used to produce glass. These raw materials are precisely proportioned and mixed to produce a specific type of glass. The choice and proportion of the formula directly affects the physical and chemical properties of the glass. For example, ordinary soda-lime glass requires a SiO2 ratio of 70%, a Na2O ratio of 12%, a CaO ratio of 8%, a MgO ratio of 3%, and an Al2O3 ratio of 1%.

[0099] When a new type of glass needs to be produced, the user can add a new raw material formula through the formula management module. For example, the production of a high-temperature resistant glass requires increasing the proportion of B2O3 based on the existing formula, so the user can add this new formula to the system. If a certain type of glass is no longer produced, the user can delete the corresponding raw material formula from the system through this module to keep the system simple and efficient.

[0100] The data processing module is used to collect historical data of raw material ratios, add the raw material formula to the historical data of raw material ratios, and perform data preprocessing to obtain a first data set.

[0101] The raw material ratio historical data refers to the specific ratio information of various raw materials used at different time points in the glass production process. The raw material ratio historical data includes:

[0102] The proportioning timestamp includes the time record of each proportioning operation;

[0103] Raw material types include the names of all raw materials used in glass production, such as silica sand, soda ash and limestone;

[0104] The production batch number includes the batch number of each production and associates the raw material formula with a specific production batch;

[0105] Production parameters include other production parameters related to each mix, such as equipment status, temperature and pressure;

[0106] The quality inspection results include the quality inspection data of the glass products for each mix, such as transparency, strength, heat resistance and other quality indicators.

[0107] Remove or correct outliers, missing values ​​or erroneous data from the historical data of raw material ratios to ensure the quality and consistency of the data. Divide the cleaned historical data of raw material ratios into training sets and test sets in proportion, with the training set accounting for 70% and the test set accounting for 30%.

[0108] The initial ratio determination module obtains the glass raw material demand according to the raw material formula and expert experience.

[0109] Furthermore, the initial ratio determination module includes:

[0110] The expert experience includes glass raw material information and other raw material information, wherein the glass raw material information includes raw material conversion rate, raw material purity and raw material volatilization rate, and the other raw material information includes the content of additives, which are obtained through the accumulation of historical data and the long-term experience summary of experts;

[0111] Among them, the content of the additives is introduced in the form of addition, such as the oxidant is introduced as sodium nitrate at 4% of the batch, the colorant is introduced as copper oxide at 1.5% of the batch, and the clarifier is introduced as cerium dioxide at 0.5% of the batch;

[0112] The raw material formula includes the target ratio of the raw materials and the total mass of the glass to be prepared;

[0113] According to the expert experience and the raw material formula, the glass raw material requirement is calculated, and the calculation formula of the glass raw material requirement is:

[0114]

[0115] Among them, O i is the demand for the i-th glass raw material, P i is the target ratio of the i-th raw material, T is the total mass of the glass prepared, C i is the purity of the i-th raw material, E i is the conversion rate of the ith raw material, V i is the volatilization rate of the i-th raw material.

[0116] Specifically, the raw material of SiO2 in the glass composition is quartz sand, with a conversion rate of 100%, a purity of 99.7%, and a volatilization rate of 0. If the target ratio is set to 60%, and the total mass of glass preparation is 1000g, then the required amount of quartz sand can be calculated to be 601.80g. The initial ratio determination module can accurately consider factors such as purity, conversion rate, and volatilization rate to ensure the accuracy of the raw material ratio, thereby improving the quality of glass products and the stability of the production process.

[0117] A status information acquisition module acquires current status information of the device according to the first data set.

[0118] Furthermore, the current status information includes:

[0119] The equipment status includes the equipment on state and the equipment off state. When the equipment is on state, the raw materials can be added or mixed, while when it is off state, maintenance or cleaning work can be carried out;

[0120] Environmental information includes the temperature, humidity, and pressure of the production environment;

[0121] The raw material formula information includes the raw material formula and a digital serial number, and the digital serial number corresponds to the raw material formula one by one;

[0122] The recipe change information includes a recipe non-change flag and a recipe change flag.

[0123] Specifically, the system records multiple raw material formulas, each with a unique digital serial number. For example, the digital serial number of formula 1 is 001, which corresponds to a standard glass formula; the digital serial number of formula 2 is 002, which corresponds to a high-strength glass formula. During the production process, the system retrieves the corresponding raw material formula from the formula library and applies it according to the input ratio request. Through the status information acquisition module, the system dynamically adjusts the ratio parameters according to the real-time monitoring of equipment status and environmental information to ensure the accuracy of the ratio under different conditions. In addition, the system can quickly respond to formula changes, ensure smooth switching between different formulas, avoid quality fluctuations caused by formula changes, and enable the production system to better meet the needs of modern manufacturing.

[0124] The RBC model building module is used to generate a first experience sample according to the current state information and the glass raw material demand, and store the first experience sample in an experience playback pool.

[0125] Further, if Figure 2 As shown, the first experience sample generating step includes:

[0126] Step S20, initializing the experience replay pool, randomly extracting data including all recipes according to the first data set, to obtain a second data set;

[0127] The second data set includes sample data of all recipes, and 30 data are randomly selected for each recipe to ensure that each recipe has at least 30 samples in the second data set;

[0128] Step S21, obtaining the glass raw material demand according to the second data set and current state information;

[0129] Step S22, assigning the glass raw material demand to the current proportioning behavior, inputting the current proportioning behavior into the device, and obtaining a proportioning reward and the next batch status information;

[0130] Step S23, combining the current state information, the current matching behavior, the matching reward and the next batch of state information to obtain the first experience sample, and storing the first experience sample in the experience replay pool;

[0131] Step S24, if the first experience sample reaches the first threshold, the matching process is terminated, otherwise the next batch of state information is used as the current state information of the next batch of matching process;

[0132] Among them, the first threshold is set to 300;

[0133] Step S25, determining the recipe change information and the raw material recipe information according to the current state information. If a recipe non-change mark is obtained, it is not necessary to obtain the glass raw material demand, and repeat steps S22 to S24; if a recipe change mark is obtained, repeat steps S21 to S24.

[0134] Specifically, the RBC (Rule-Based Control) model is a control system based on predefined rules, which is used to generate the first experience sample after interacting with the equipment. In this model, the current proportioning behavior is directly set to the demand for glass raw materials based on expert experience and formula information. Only when the formula changes, the current proportioning behavior will be adjusted accordingly. By inputting the first experience sample generated based on expert experience into the experience playback pool, the RBC model can use these experiences in subsequent operations for more effective control and optimization, thereby improving the accuracy of raw material proportioning.

[0135] Furthermore, the matching reward includes:

[0136] According to the actual ratio of raw materials and the target ratio of raw materials, the basic reward R is calculated. base , the basic reward function is expressed as:

[0137]

[0138] Among them, P actual,i is the actual ratio of the i-th raw material, P target,i is the target ratio of the i-th raw material;

[0139] According to the actual ratio of the raw materials and the current state information, the volatility reward R is calculated. stability ;

[0140] If the recipe changes information O change If the formula change information is 0, it is a sign that the formula has not changed, and the volatility reward is equal to the sum of the variances of the actual proportions of the n raw materials; if the formula change information is 1, it is a sign that the formula has changed, and the volatility reward R stability Equal to 0, the volatility reward function is expressed as:

[0141]

[0142] Wherein, Var is the variance of the actual proportion of the i-th raw material, Ochange For recipe change information, O change =0 means the formula has not been changed, O change =1 is the formula change mark, the raw material type is the sum of the glass raw material ratio and the additives, and n is set to 20.

[0143] Specifically, the basic reward function improves the quality of glass products by penalizing the deviation between the actual ratio and the target ratio. This function encourages the model to reduce the ratio error, thereby ensuring that the final product meets the expected quality standards. The volatility reward function focuses on penalizing situations where the ratio fluctuates too much and can adapt to changes in the formula. For example, if there are 10 time steps O change =0, then calculate the variance of the actual ratio of all raw materials in these 10 time steps, add them up to get 0.2, and assign 0.2 to the volatility reward. change = 1, then recalculate the reward. Calculating the volatility reward function helps stabilize the production process, reduce drastic changes in raw material ratios, ensure production stability and consistency, and thus improve the accuracy of raw material ratios.

[0144] Furthermore, the calculation method of the matching reward is to obtain the matching reward by weighted calculation based on the basic reward, the volatility reward, equipment energy consumption and raw material loss, which is expressed as:

[0145] R=w1×R base -w2×R stability -w3×E efficiency -w4×C waste ;

[0146] Among them, R is the matching reward, E efficiency is the energy consumption of the equipment, C waste is the raw material loss, w1, w2, w3 and w4 are weight factors of the basic reward, the volatility reward, the equipment energy consumption and the raw material loss respectively.

[0147] Among them, after normalizing all reward items, w1, w2, w3 and w4 are set to 0.6, 0.2, 0.1 and 0.1 respectively.

[0148] Specifically, the basic reward has a higher weight, which ensures that the deviation between the actual ratio and the target ratio is fully considered. This setting emphasizes the importance of ratio accuracy, which helps to improve the quality of the final glass product and ensure that it meets production standards and user requirements. The setting of equipment energy consumption rewards encourages the optimization of equipment energy consumption, and the setting of raw material loss rewards prompts the model to reduce raw material waste in the production process. By incorporating multiple reward items into the total reward calculation and setting weights, such a comprehensive optimization strategy makes the production process focus not only on quality, but also on cost control and resource utilization, thereby improving the accuracy of raw material ratios.

[0149] The raw material ratio determination module uses the current state information and the current raw material ratio information and determines the current ratio behavior according to the TD3 model.

[0150] Furthermore, the raw material ratio determination module includes:

[0151] The current raw material ratio information includes n kinds of glass raw material ratios and j kinds of additive contents. The current raw material ratio information is expressed as:

[0152] A={p1,p2,…p n ,k1,k2…k j};

[0153] Where A is the current raw material ratio information, p1, p2, ... p n are the first glass raw material ratio, the second glass raw material ratio and the nth glass raw material ratio, k1, k2…k j are the first additive content, the second additive content and the jth additive content, n is set to 15, and j is set to 5;

[0154] Inputting the current state information into a strategy network, wherein the strategy network generates a first raw material ratio according to the current raw material ratio information;

[0155] Specifically, the strategy network generates the first raw material ratio process as follows:

[0156] The input layer receives the current state information and the current raw material ratio information as input. The hidden layer adjusts the ratio of the glass raw materials by capturing the complex relationship between the input information and its influence on the ratio. The output layer generates a vector, which is the first raw material ratio. Each element represents the recommended ratio of the corresponding glass raw material.

[0157] Euler noise is added to the first raw material ratio to obtain the current ratio behavior, which is expressed as:

[0158] a=m(s)+noise;

[0159] Wherein, a is the current ratio behavior, m(s) is the first raw material ratio under the current state information, and noise is the Euler noise;

[0160] The Euler noise formula is:

[0161] d(noise t )=b×(c-noise t )×dt+f×dW t ;

[0162] Among them, the initial value of Euler noise is set to 0, the termination value of Euler noise is set to 1e-3, b is the regression speed, b is set to 0.15, c is the long-term average value of noise, c is set to 0, f is the noise intensity, f is set to 0.2, dW t For Brownian motion;

[0163] Specifically, Euler noise (Ornstein-Uhlenbeck Noise) is a noise model used to enhance the ability of policy exploration, especially widely used in the TD3 model. Its introduction helps the policy network avoid falling into local optimality during the exploration stage of raw material ratios, so as to better cope with complex production environments. For example, in glass preparation, if the current environmental conditions change slightly (such as a temperature increase of 1°C), the current ratio behavior with Euler noise can gradually adjust the ratio in continuous time instead of drastically jumping, maintaining the stability of the ratio behavior. The policy network takes into account the current environmental conditions when generating the first raw material ratio. With the assistance of Euler noise, the action selection is further refined to ensure that the ratio behavior remains accurate in a dynamically changing production environment.

[0164] The raw material ratio control module inputs the current ratio behavior into the device to obtain the ratio reward and the next batch of status information; combines the current status information, the current ratio behavior, the ratio reward and the next batch of status information to obtain a second experience sample, and stores the second experience sample in the experience replay pool; obtains a trained TD3 model according to the experience replay pool, and obtains the optimized raw material ratio according to the trained TD3 model.

[0165] Specifically, the TD3 (Twin Delayed Deep Deterministic Policy Gradient) model is a reinforcement learning algorithm for continuous action space, which belongs to the category of policy gradient methods. Figure 3 As shown in the figure, the TD3 model selects a current matching behavior at each time step and acts on the device. After receiving the current matching behavior, the current state information of the device will change accordingly, and a matching reward signal will be generated and fed back to the TD3 model. The TD3 model further selects the next batch of matching behaviors based on the obtained matching rewards and the next batch of state information. The principle of selecting matching behaviors is not only to maximize the current matching rewards, but also to consider the impact on a batch of state information in the environment, so as to maximize the final cumulative matching rewards.

[0166] Further, if Figure 4 As shown, the deep neural network training process of the TD3 model includes:

[0167] The deep neural network includes a policy network m u, First Value Network Second Value Network Target Policy Network u' , First target value network and the second target value network Where u is the policy network parameter, u1 and u2 are the policy network parameters, u' is the target policy network parameter, u1' and u2' are the target policy network parameters;

[0168] A batch of the first experience samples and the second experience samples are randomly selected from the experience replay pool, and the fused samples are represented as (s, a, r, s'), where s is the current state information, a is the current matching behavior, r is the matching reward, and s' is the next batch of state information;

[0169] Based on the experience sample, the target strategy network is used to generate the next matching behavior:

[0170] a'=m u' (s')+noise;

[0171] Among them, a' is the next matching behavior, noise is the Euler noise;

[0172] Next, the first target value network and the second target value network are used to calculate the first target Q value and the second target Q value respectively. The target Q value is the minimum value between the first target Q value and the second target Q value, which is expressed as:

[0173]

[0174] Wherein, y is the target Q value, k is the discount factor, which is set to 0.99;

[0175] The parameters of the first value network and the second value network are updated by minimizing the mean square error between the actual Q value and the target Q value of the first value network and the second value network. The updated objective function is expressed as:

[0176]

[0177] Where N is the batch size of the empirical samples, which is set to 64. is the mean square error of the i-th value network, b Q is the value network learning rate, set to 0.001, j is the sample number, is the gradient of the i-th value network, i is 1 or 2;

[0178] The parameters of the policy network are updated by maximizing the Q value of the first value network, which is expressed as:

[0179]

[0180]

[0181] Among them, b m is the policy network learning rate, set to 0.001, is the Q value of the policy network, is the gradient of the policy network;

[0182] Every q steps, the parameters of the target policy network, the first target value network and the second target value network are updated according to the parameters of the policy network, the first value network and the second value network using a soft update method, which is expressed as:

[0183] u'←zu+(1-z)u';

[0184] u i ←zu i +(1-z)u i ';

[0185] Among them, q is set to 2 and the soft update coefficient z is set to 0.005.

[0186] Specifically, for example, the current state is that the equipment is running at high temperature and the demand is 1000kg of raw materials. The RBC model generates a preliminary raw material ratio based on the state information, 40% silica sand, 30% sodium ash, and 30% lime. This ratio is recorded as the first experience sample. After TD3 optimization, the first experience sample is not suitable for the current equipment state. The ratio is adjusted to 38% silica sand, 32% sodium ash, and 30% lime. The ratio reward fed back by the equipment is +3 points, indicating that the production quality has improved. The current state information is updated to a slight decrease in the equipment temperature and the raw material consumption is in line with expectations. This information is recorded as the second experience sample. By combining the first experience sample generated by the RBC model and the TD3 model for optimization, the raw material ratio can be optimized in real time to adapt to changes in production, thereby improving the accuracy of the raw material ratio.

[0187] The exception management module is used to detect and handle abnormal situations in the batching process.

[0188] Furthermore, the abnormal situation includes:

[0189] The ratio deviation is calculated based on the actual ratio of the raw materials and the target ratio of the raw materials, and is expressed as:

[0190]

[0191] Among them, Δp i is the ratio deviation of the i-th raw material, P actual,i is the actual ratio of the i-th raw material, Ptarget,i is the target ratio of the i-th raw material;

[0192] If the matching deviation is less than a second threshold, the current matching behavior is not updated;

[0193] If the ratio deviation is greater than or equal to the second threshold, the first abnormality is automatically recorded;

[0194] If the ratio deviation still exceeds the second threshold after f time steps, notify manual intervention;

[0195] Among them, the second threshold is set to 1%, and f is set to 5;

[0196] The raw material loss is calculated according to the actual ratio of the raw materials and the current ratio behavior, which is expressed as:

[0197] C waste,i =P i -P actual,i ;

[0198] Among them, C waste,i is the loss of the i-th raw material, P i is the current matching behavior of the ith type;

[0199] If the raw material loss is greater than a third threshold, a second abnormality is automatically recorded;

[0200] If the ratio deviation still exceeds the third threshold after g time steps, notify manual intervention;

[0201] Among them, the third threshold is set to 500kg and g is set to 10;

[0202] Specifically, assuming that SiO2 is 70%, Na2O is 15%, and CaO is 15% during the glass production process, the actual ratios extracted from the production data are 69%, 15%, and 14.2%, and the ratio deviation of SiO2 is greater than the second threshold, so the abnormality needs to be recorded. If the ratio deviation is large, the reward value may be low. The TD3 model receives a negative value of the ratio reward during the control process and will optimize the ratio behavior to try to correct the deviation. If the ratio deviation of SiO2 is greater than the second threshold after 5 time steps, it can be considered that the optimization model is abnormal or the equipment itself is abnormal. It is necessary to manually confirm whether the parameter settings of the TD3 model are reasonable, whether the model needs to be retrained, and check whether the raw material delivery system, sensor, and control system are faulty or inaccurately adjusted. Ensure that the problem is resolved after troubleshooting, thereby improving the accuracy of the raw material ratio.

[0203] In summary, the formula management module can flexibly manage glass preparation formulas, quickly add and delete formulas, and improve the adaptability of the production line; the data processing module combines historical data and new formulas to ensure data accuracy, laying the foundation for optimizing raw material ratios; the initial ratio determination module uses expert experience to provide reliable ratio references; the RBC model and status information acquisition module generates high-quality experience samples to accelerate the optimization of raw material ratios; the TD3 model intelligently adjusts the ratio behavior according to the current status, dynamically adapts to production changes, and ultimately achieves higher-precision raw material ratio control; the abnormal management module provides real-time detection and processing functions to promptly respond to abnormal situations that may occur in the production process, ensuring the stability of system operation and the high quality of glass preparation products.

[0204] Embodiment 2

[0205] This embodiment solves the problem of accuracy and intelligence of raw material ratio in the glass preparation process by realizing an automatic control method for raw material ratio in glass preparation.

[0206] like Figure 5 As shown, a method for automatically controlling the ratio of raw materials for glass preparation comprises:

[0207] Collecting the raw material formula for preparing the glass;

[0208] Collecting raw material ratio historical data, adding the raw material formula to the raw material ratio historical data, and performing data preprocessing to obtain a first data set;

[0209] According to the raw material formula and expert experience, the demand for glass raw materials is obtained;

[0210] Acquire current status information of the device according to the first data set;

[0211] Generate a first experience sample according to the current state information and the glass raw material demand, and store the first experience sample in an experience playback pool;

[0212] Using the current state information and the current raw material ratio information and according to the TD3 model to determine the current ratio behavior;

[0213] Input the current proportioning behavior into the device to obtain a proportioning reward and the next batch of state information; combine the current state information, the current proportioning behavior, the proportioning reward and the next batch of state information to obtain a second experience sample, and store the second experience sample in the experience replay pool; obtain a trained TD3 model according to the experience replay pool, and obtain the optimized raw material proportion according to the trained TD3 model;

[0214] Detect and handle anomalies in the batching process.

[0215] Specifically, in order to prove the effectiveness of an automatic control method for the ratio of raw materials in glass preparation, the method is recorded as Scheme 4, and three comparative schemes are proposed for testing. Scheme 1 uses the RBC model to adjust the control strategy according to the predetermined formula and the demand for glass raw materials, and does not use the reinforcement learning model; Scheme 2 only uses the DDPG model, and the state information, ratio information and ratio reward are the same as those of Scheme 4; Scheme 3 only uses the TD3 model, and the state information, ratio information and ratio reward are the same as those of Scheme 4.

[0216] As shown in Table 1, Scheme 4 is superior to the comparative scheme in all indicators. The ratio deviation of Scheme 4 is 0.9%, which is significantly lower than other schemes, indicating that this method has higher accuracy in ratio control. The raw material loss of Scheme 4 is controlled at 500kg, which significantly reduces the waste of raw materials and improves the utilization rate of raw materials. The abnormality rate of Scheme 4 is 2.1%, which is significantly lower than other schemes, indicating that the frequency of abnormal situations in actual production is greatly reduced. The energy consumption of Scheme 4 is 400kWh / ton of glass, and the energy-saving effect is significantly better than other schemes. Scheme 4 combines the TD3 model and prior knowledge for optimization strategy, realizes precise control in the glass production process, significantly improves production efficiency, reduces energy consumption, and effectively reduces raw material waste and the occurrence of abnormal situations, thereby improving the accuracy of raw material ratio.

[0217] Table 1. Performance comparison of different solutions

[0218] plan Ratio deviation % Raw material loss kg Abnormality rate% Energy consumption kWh / ton of glass Solution 1 1.0 1002 2.4 435 Solution 2 1.8 850 4.7 432 Option 3 1.4 742 4.5 425 Option 4 0.9 500 2.1 401

[0219] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An automatic control system for glass preparation raw material ratio, characterized in that: include: Recipe management module, used to add and delete raw material recipes for glass preparation; A data processing module, used for collecting raw material ratio historical data, adding the raw material formula to the raw material ratio historical data, and performing data preprocessing to obtain a first data set; An initial ratio determination module is used to obtain the glass raw material demand according to the raw material formula and expert experience; A status information acquisition module, which acquires current status information of the device according to the first data set; An RBC model building module, configured to generate a first experience sample according to the current state information and the glass raw material demand, and store the first experience sample in an experience playback pool; A raw material ratio determination module, using the current state information and the current raw material ratio information and determining the current ratio behavior according to the TD3 model; A raw material ratio control module inputs the current ratio behavior into the device to obtain a ratio reward and next batch status information; The current state information, the current matching behavior, the matching reward and the next batch of state information are combined to obtain a second experience sample, and the second experience sample is stored in the experience replay pool; Obtaining a trained TD3 model according to the experience playback pool, and obtaining the optimized raw material ratio according to the trained TD3 model; The exception management module is used to detect and handle abnormal situations in the batching process.

2. The automatic control system for glass preparation raw material ratio according to claim 1, characterized in that: The initial ratio determination module comprises: The expert experience includes glass raw material information and other raw material information, wherein the glass raw material information includes raw material conversion rate, raw material purity and raw material volatilization rate, and the other raw material information includes the content of additives; The raw material formula includes the target ratio of the raw materials and the total mass of the glass to be prepared; The required amount of the glass raw materials is calculated based on the expert experience and the raw material formula.

3. The automatic control system for glass preparation raw material ratio according to claim 1, characterized in that: The current status information includes: The device status includes the device on status and the device off status; Environmental information includes the temperature, humidity, and pressure of the production environment; The raw material formula information includes the raw material formula and a digital serial number, and the digital serial number corresponds to the raw material formula one by one; The recipe change information includes a recipe non-change flag and a recipe change flag.

4. The automatic control system for glass preparation raw material ratio according to claim 1, characterized in that: The raw material ratio determination module comprises: The current raw material ratio information includes n kinds of glass raw material ratios and j kinds of additive contents; Inputting the current state information into a strategy network, wherein the strategy network generates a first raw material ratio according to the current raw material ratio information; Euler noise is added to the first raw material ratio to obtain the current ratio behavior.

5. The automatic control system for glass preparation raw material ratio according to claim 1, characterized in that: The matching rewards include: The basic reward is calculated based on the actual ratio of raw materials and the target ratio of raw materials; Calculate a volatility reward based on the actual ratio of the raw materials and the current state information; If the recipe change information is 0, it is a sign that the recipe has not changed, and the volatility reward is equal to the sum of the variances of the actual proportions of the n raw materials; if the recipe change information is 1, it is a sign that the recipe has changed, and the volatility reward is equal to 0.

6. The automatic control system for glass preparation raw material ratio according to claim 5, characterized in that: The calculation method of the matching reward is: according to the basic reward, the volatility reward, equipment energy consumption and raw material loss, the matching reward is obtained by weighted calculation.

7. The automatic control system for glass preparation raw material ratio according to claim 1, characterized in that: The deep neural network training process of the TD3 model includes: The deep neural network includes a policy network, a first value network, a second value network, a target policy network, a first target value network, and a second target value network; Randomly extract a batch of the first experience samples and the second experience samples from the experience replay pool; According to the extracted experience samples, using the first target value network and the second target value network to respectively calculate a first target Q value and a second target Q value, wherein the target Q value is the minimum value between the first target Q value and the second target Q value; Updating parameters of the first value network and the second value network by minimizing the mean square error between the actual Q values ​​of the first value network and the second value network and the target Q value; Updating the parameters of the policy network by maximizing the Q value of the first value network; Every q steps, the parameters of the target policy network, the first target value network and the second target value network are updated according to the parameters of the policy network, the first value network and the second value network using a soft update method.

8. The automatic control system for glass preparation raw material ratio according to claim 1, characterized in that: The first experience sample generating step comprises: Step S20, initializing the experience replay pool, randomly extracting data including all recipes according to the first data set, to obtain a second data set; Step S21, obtaining the glass raw material demand according to the second data set and current state information; Step S22, assigning the glass raw material demand to the current proportioning behavior, inputting the current proportioning behavior into the device, and obtaining the proportioning reward and the next batch status information; Step S23, combining the current state information, the current matching behavior, the matching reward and the next batch of state information to obtain the first experience sample, and storing the first experience sample in the experience replay pool; Step S24, if the first experience sample reaches the first threshold, the matching process is terminated, otherwise the next batch of state information is used as the current state information of the next batch of matching process; Step S25, determining the recipe change information and the raw material recipe information according to the current state information. If a recipe non-change mark is obtained, it is not necessary to obtain the glass raw material demand, and repeat steps S22 to S24; if a recipe change mark is obtained, repeat steps S21 to S24.

9. The automatic control system for glass preparation raw material ratio according to claim 1, characterized in that: The abnormal situations include: Calculating a ratio deviation according to an actual ratio of the raw materials and a target ratio of the raw materials, and if the ratio deviation is less than a second threshold, not updating the current ratio behavior; If the ratio deviation is greater than or equal to the second threshold, the first abnormality is automatically recorded; If the ratio deviation still exceeds the second threshold after f time steps, notify manual intervention; Calculating the raw material loss according to the actual ratio of the raw materials and the current ratio behavior, and automatically recording a second abnormality if the raw material loss is greater than a third threshold; If the ratio deviation still exceeds the third threshold after g time steps, the manual intervention is notified.

10. A method for automatically controlling the ratio of raw materials for glass preparation, characterized in that: include: Collecting the raw material formula for preparing the glass; Collecting raw material ratio historical data, adding the raw material formula to the raw material ratio historical data, and performing data preprocessing to obtain a first data set; According to the raw material formula and expert experience, the demand for glass raw materials is obtained; According to the first data set, obtaining current status information of the device; Generate a first experience sample according to the current state information and the glass raw material demand, and store the first experience sample in an experience playback pool; Using the current state information and the current raw material ratio information and according to the TD3 model to determine the current ratio behavior; Input the current matching behavior into the device to obtain matching rewards and next batch status information; The current state information, the current matching behavior, the matching reward and the next batch of state information are combined to obtain a second experience sample, and the second experience sample is stored in the experience replay pool; Obtaining a trained TD3 model according to the experience playback pool, and obtaining the optimized raw material ratio according to the trained TD3 model; Detect and handle anomalies in the batching process.

Images