Glass raw material proportioning automatic control system and method

Abstract

This invention relates to the field of glass technology, specifically to an automatic control system and method for the proportioning of raw materials in glass preparation. The system includes a formula management module, a data processing module, an initial proportioning determination module, a status information acquisition module, an RBC model establishment module, a raw material proportioning determination module, a raw material proportioning control module, and an anomaly management module. The invention manages raw material formulas through the formula management module; the data processing module collects and preprocesses historical data; the initial proportioning determination module and the RBC model establishment module combine expert experience to generate preliminary proportions and store them in an experience playback pool; the status information acquisition module obtains current status information based on equipment requests; the raw material proportioning determination module generates proportioning behavior through a TD3 model; the control module generates and stores experience samples for use with the TD3 model, thereby optimizing proportioning behavior; and the anomaly management module is responsible for monitoring and handling anomalies. This effectively improves the accuracy of system control and enhances glass production efficiency and product quality.

Description

Technical Field

[0001] This invention relates to the field of glass technology, specifically to an automatic control system and method for the proportioning of raw materials in glass preparation. Background Technology

[0002] As a key component of modern industry and building materials, the quality and performance of glass directly affect the reliability and safety of the final product. In glass production, the accuracy of raw material proportioning is a core factor in ensuring glass quality. Traditional proportioning methods rely on manually weighing raw materials and then feeding the prepared materials into a batching machine for mixing. However, since many raw materials are in powder or granular form, manual batching easily leads to dust inhalation, posing a threat to worker health and increasing production risks and labor costs. Furthermore, the wide variety and large quantity of raw materials make precise management difficult with manual batching, easily resulting in errors that affect glass quality and increase management costs.

[0003] With the continuous development of industrial automation technology, many solutions have been proposed for automatic control systems in the glass manufacturing industry. These systems have demonstrated significant advantages in improving production efficiency, optimizing resource allocation, and ensuring product quality. For example, they can achieve high-precision process control and enhance production stability through real-time data monitoring. However, in the raw material proportioning process of glass production lines, the actual situation of the raw materials may not be entirely consistent with the set results, or all influencing factors may not be fully considered. For example, changes in the system environment or process conditions may lead to unsatisfactory proportioning results or even negative impacts. Furthermore, the lack of consideration for handling multiple formula variations, coupled with the increased manpower and time required for frequent updates and optimizations of the raw material proportioning control model, further increases the system's maintenance costs, resulting in a lack of sufficient accuracy.

[0004] Therefore, an automatic control system and method for the proportioning of raw materials in glass preparation are proposed. Summary of the Invention

[0005] The purpose of this invention is to provide an automatic control system and method for the proportioning of raw materials in glass preparation. The system comprises: a formula management module to manage raw material formulas; a data processing module to collect and preprocess historical data; an initial proportioning determination module to calculate the required amount of glass raw materials based on expert experience; a status information acquisition module to obtain current status information based on equipment requests; an RBC model building module to input the required amount of glass raw materials into the equipment based on the current status information to obtain a first experience sample and store it in an experience playback pool; a raw material proportioning determination module to generate the current proportioning behavior through a TD3 model; a control module to input the current proportioning behavior into the equipment, calculate the proportioning reward, generate and store a second experience sample, and combine it with the first experience sample to use the TD3 model to optimize the proportioning strategy; and an anomaly management module to monitor and handle anomalies. This system and method effectively improve the accuracy of system control.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An automatic control system for the proportioning of raw materials in glass preparation, comprising:

[0008] The recipe management module is used to add and delete raw material recipes for glass preparation;

[0009] The data processing module is used to collect historical data on raw material ratios, add the raw material formula to the historical data on raw material ratios, and perform data preprocessing to obtain the first dataset;

[0010] The initial proportioning determination module determines the required amount of glass raw materials based on the raw material formula and expert experience;

[0011] The status information acquisition module acquires the current status information of the device based on the first dataset;

[0012] The RBC model building module is used to generate a first experience sample based on the current state information and the glass raw material demand, and to store the first experience sample in the experience playback pool.

[0013] The raw material ratio determination module uses the current status information and the current raw material ratio information and determines the current ratio behavior based on the TD3 model;

[0014] The raw material proportioning control module inputs the current proportioning behavior into the equipment to obtain proportioning rewards and next batch status information; it combines the current status information, the current proportioning behavior, the proportioning rewards, and the next batch status information to obtain a second experience sample, and stores the second experience sample in the experience playback pool; it obtains a trained TD3 model based on the experience playback pool, and obtains the optimized raw material proportions based on the trained TD3 model;

[0015] The anomaly management module is used to detect and handle abnormal situations during the ingredient preparation process.

[0016] Furthermore, the initial proportion determination module includes:

[0017] The expert experience includes information on glass raw materials and other raw materials. The glass raw material information includes raw material conversion rate, raw material purity, and raw material volatility. The other raw material information includes the content of additives.

[0018] The raw material formulation includes the target proportions of the raw materials and the total mass of glass preparation;

[0019] Based on the expert experience and the raw material formula, the required amount of glass raw materials is calculated and expressed as follows:

[0020]

[0021] Among them, O i Let P be the demand for the i-th type of glass raw material. i Let T be the target proportion of the i-th raw material, T be the total mass of the glass prepared, and C be the total mass of the glass prepared. i E represents the purity of the i-th raw material. i Let V be the conversion rate of the i-th raw material. i Let be the volatilization rate of the i-th raw material.

[0022] Furthermore, the current status information includes:

[0023] Device status includes device on status and device off status;

[0024] Environmental information includes the temperature, humidity, and pressure of the production environment;

[0025] The raw material formula information includes the raw material formula and a numerical serial number, wherein the numerical serial number corresponds one-to-one with the raw material formula;

[0026] The formula change information includes indicators for no formula change and indicators for formula change.

[0027] Furthermore, the raw material proportioning determination module includes:

[0028] The current raw material ratio information includes the ratios of n glass raw materials and the content of j additives. The current raw material ratio information is expressed as follows:

[0029] A = {p1, p2, ..., p} n ,k1,k2…k j};

[0030] Where A represents the current raw material ratio information, p1, p2, ... p nThese represent the proportions of the first type of glass raw material, the second type of glass raw material, and the nth type of glass raw material, k1, k2…k j These are the contents of the first type of admixture, the second type of admixture, and the jth type of admixture, respectively.

[0031] The current status information is input into the strategy network, and the strategy network generates a first raw material ratio based on the current raw material ratio information.

[0032] Adding Euler noise to the first raw material ratio yields the current ratio behavior, which is represented as follows:

[0033] a = m(s) + noise;

[0034] Where a represents the current proportioning behavior, m(s) represents the first raw material proportioning under the current state information s, and noise represents the Euler noise.

[0035] Furthermore, the matching bonus includes:

[0036] Based on the actual and target proportions of the raw materials, the basic reward R is calculated. base The basic reward function is expressed as:

[0037]

[0038] Among them, P actual,i Let P be the actual proportion of the i-th raw material. target,i The target proportion for the i-th raw material;

[0039] Based on the actual proportions of the raw materials and the current state information, the volatility reward R is calculated. stability ;

[0040] If the formula change information is 0, it indicates that the formula has not changed, and the volatility reward R... stability It equals the sum of the variances of the actual proportions of the n raw materials; if the formula change information is 1, it is a formula change indicator, and the volatility reward R stability When the value equals 0, the volatility reward function is expressed as:

[0041]

[0042] Where Var is the variance of the actual proportion of the i-th raw material, O change For formula change information, O change =0 indicates that the formula has not changed, O change =1 indicates a formula change.

[0043] Furthermore, the ratio bonus is calculated by weighting the base bonus, the volatility bonus, equipment energy consumption, and raw material loss, and is expressed as follows:

[0044] R = -w1 × R base -w2×R stability -w3×E efficiency -w4×C waste ;

[0045] Where R is the matching bonus, E efficiency For equipment energy consumption, C waste For raw material loss, w1, w2, w3, and w4 are the weighting factors for the basic reward, the volatility reward, the equipment energy consumption, and the raw material loss, respectively.

[0046] Furthermore, the deep neural network training process of the TD3 model includes:

[0047] The deep neural network includes a policy network, a first value network, a second value network, a target policy network, a first target value network, and a second target value network.

[0048] Randomly select a batch of the first experience samples and the second experience samples from the experience replay pool;

[0049] Based on the extracted experience samples, the first target Q value and the second target Q value are calculated using the first target value network and the second target value network, respectively, and the target Q value is the minimum value between the first target Q value and the second target Q value;

[0050] The parameters of the first value network and the second value network are updated by minimizing the mean square error between the actual Q value and the target Q value of the first value network and the second value network.

[0051] The parameters of the policy network are updated by maximizing the Q-value of the first value network;

[0052] Every q steps, a soft update is used to update the parameters of the target policy network, the first target value network, and the second target value network based on the parameters of the policy network, the first value network, and the second value network.

[0053] Furthermore, the first empirical sample generation step includes:

[0054] Step S20: Initialize the experience replay pool, and randomly extract data including all recipes from the first dataset to obtain the second dataset;

[0055] Step S21: Obtain the required amount of glass raw materials based on the second dataset and the current status information;

[0056] Step S22: Assign the glass raw material demand to the current mixing ratio behavior, input the current mixing ratio behavior into the equipment, and obtain the mixing ratio reward and the status information of the next batch;

[0057] Step S23: Combine the current status information, the current allocation behavior, the allocation reward, and the next batch status information to obtain the first experience sample, and store the first experience sample in the experience replay pool;

[0058] Step S24: If the first empirical sample reaches the first threshold, the matching process ends; otherwise, the status information of the next batch is used as the current status information of the next batch matching process.

[0059] Step S25: Determine the formula change information and raw material formula information based on the current status information. If a formula change flag is obtained, it is not necessary to obtain the glass raw material requirement, and steps S22 to S24 are repeated. If a formula change flag is obtained, steps S21 to S24 are repeated.

[0060] Furthermore, the abnormal situations include:

[0061] The proportioning deviation is calculated based on the actual proportion of raw materials and the target proportion of raw materials, and is expressed as follows:

[0062]

[0063] Where, Δp i Let P be the deviation in the proportion of the i-th raw material. actual,i Let P be the actual proportion of the i-th raw material. target,i The target proportion for the i-th raw material;

[0064] If the ratio deviation is less than the second threshold, the current ratio behavior is not updated;

[0065] If the ratio deviation is greater than or equal to the second threshold, the first anomaly is automatically recorded;

[0066] If the ratio deviation still exceeds the second threshold after f time steps, then manual intervention is notified;

[0067] The raw material loss is calculated based on the actual proportion of the raw materials and the current proportioning behavior, and is expressed as follows:

[0068] C waste,i =P i -P actual,i ;

[0069] Among them, C waste,i For the loss of the i-th raw material, P i This refers to the current mixing ratio behavior of the i-th type;

[0070] If the raw material loss exceeds the third threshold, a second anomaly will be automatically recorded.

[0071] If the ratio deviation still exceeds the third threshold after g time steps, then the manual intervention is notified.

[0072] An automatic control method for the proportioning of raw materials in glass preparation, comprising:

[0073] Collect the raw material formula for the glass preparation;

[0074] Collect historical data on raw material ratios, add the raw material formula to the historical data on raw material ratios, and perform data preprocessing to obtain the first dataset;

[0075] Based on the aforementioned raw material formula and expert experience, the required amount of glass raw materials is obtained;

[0076] Based on the first dataset, obtain the current status information of the device;

[0077] A first experience sample is generated based on the current status information and the glass raw material demand, and the first experience sample is stored in the experience playback pool;

[0078] The current state information and current raw material ratio information are used to determine the current ratio behavior based on the TD3 model;

[0079] The current proportioning behavior is input into the device to obtain proportioning reward and next batch status information; the current status information, the current proportioning behavior, the proportioning reward and the next batch status information are combined to obtain a second experience sample, and the second experience sample is stored in the experience replay pool; a trained TD3 model is obtained based on the experience replay pool, and the optimized raw material proportion is obtained based on the trained TD3 model;

[0080] Detect and handle any abnormalities during the ingredient preparation process.

[0081] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0082] 1. In the glass preparation raw material proportioning process, traditional proportioning systems mainly rely on expert experience and manual adjustments. While this method is simple to implement, its accuracy is limited. This method calculates the required amount of glass raw materials based on expert experience and combines this with an RBC (Rule-Based Control) model. By interacting with the equipment using preset rules, proportioning rewards and the status information for the next batch are obtained. The information obtained from this interaction generates the first batch of experience samples, which not only provides high-quality data support for subsequent optimization but also accelerates the optimization process of the TD3 algorithm, significantly improving the accuracy of glass preparation raw material proportioning.

[0083] 2. Traditional proportioning control systems often struggle to fully optimize proportioning schemes when faced with complex production environments and dynamically changing data, leading to insufficient proportioning accuracy and raw material waste. By introducing the TD3 model and combining it with an environmental condition monitoring system, the system can capture and evaluate environmental changes in production equipment in real time. The system uses policy networks and value networks to optimize proportioning rewards, intelligently adjusting the current raw material proportioning scheme to ensure accurate raw material proportioning even in complex environments, thereby improving overall production efficiency and resource utilization.

[0084] 3. During the production process, traditional systems struggle to respond promptly to anomalies such as proportioning deviations and raw material losses, which can impact production stability and product quality. To address these challenges, an anomaly management module enables real-time monitoring of potential anomalies during production, such as proportioning deviations, raw material waste, or equipment malfunctions. The system not only automatically records these anomalies but also provides manual assistance suggestions, ensuring production continuity and stability, effectively improving product quality stability, and reducing production interruptions and resource waste caused by anomalies. Attached Figure Description

[0085] Figure 1 This is a schematic diagram of an automatic control system for the proportioning of raw materials for glass preparation provided in this disclosure;

[0086] Figure 2 This is a flowchart illustrating the reinforcement learning model provided in this disclosure;

[0087] Figure 3 A flowchart illustrating the process of generating the first empirical sample provided in this disclosure;

[0088] Figure 4 This is a schematic diagram of the deep neural network training process provided in this disclosure;

[0089] Figure 5 This is a schematic flowchart of an automatic control method for the proportioning of raw materials in glass preparation, as provided in this disclosure. Detailed Implementation

[0090] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0091] In the glass manufacturing industry, raw material ratios are a key factor affecting product quality and production efficiency. Glass is typically composed of a mixture of various raw materials, such as silica sand, alkali metal oxides, and calcium carbonate materials. The ratio of these raw materials directly determines the physical properties of the glass, such as transparency, strength, and heat resistance; therefore, precise ratios are crucial for producing high-quality glass.

[0092] Traditional automated control systems for glass manufacturing raw material proportioning typically rely on human experience to set fixed feedback mechanisms. While this method can meet basic production requirements, its limitations significantly impact the accuracy of proportioning when facing complex and dynamically changing production environments. This can lead to instability in raw material proportions, affecting the quality and consistency of glass products. Furthermore, with increasing production demands and more complex formulations, traditional systems do not consider the need for real-time adjustments to multiple formulations, limiting their ability to achieve precise and intelligent control in modern production. Therefore, a new system and method are urgently needed to improve the accuracy of raw material proportioning and meet the demands of modern production for efficient, stable, and intelligent control.

[0093] like Figures 1 to 5 As shown, the present invention provides an automatic control system and method for the proportioning of raw materials in glass preparation, the technical solution of which is as follows:

[0094] Example 1

[0095] This embodiment uses a glass factory as the experimental environment and employs the hardware equipment of a traditional automatic control system for glass raw material proportioning, including an industrial computer, PLC, touch screen, batching controller, and weighing sensors. This embodiment aims to address the shortcomings of traditional glass preparation raw material proportioning control systems in handling complex and dynamic production environments, particularly the challenges of processing multiple formulations, thereby improving the accuracy of raw material proportioning.

[0096] like Figure 1 As shown, an automatic control system for the proportioning of raw materials for glass preparation includes:

[0097] The recipe management module is used to add and delete raw material recipes for glass preparation.

[0098] The raw material formulation refers to the specific proportions of various raw materials used in glass production. These raw materials must be precisely proportioned and mixed to produce a specific type of glass. The selection and proportion of the formulation directly affect the physical and chemical properties of the glass. For example, ordinary soda-lime glass requires 70% SiO2, 12% Na2O, 8% CaO, 3% MgO, and 1% Al2O3.

[0099] When a new type of glass needs to be produced, users can add a new raw material formula through the formula management module. For example, to produce a high-temperature resistant glass, the proportion of B2O3 needs to be increased based on the existing formula, so users can add this new formula to the system. If a certain type of glass is no longer produced, users can delete the corresponding raw material formula from the system through this module to maintain the system's simplicity and efficient management.

[0100] The data processing module is used to collect historical data on raw material ratios, add the raw material formula to the historical data on raw material ratios, and perform data preprocessing to obtain the first dataset.

[0101] Historical raw material proportioning data refers to the specific proportioning information of various raw materials used at different points in time during the glass production process. Historical raw material proportioning data includes:

[0102] The matching timestamp includes the time record of each matching operation;

[0103] The types of raw materials include all the names of raw materials used in glass production, such as silica sand, soda ash, and limestone.

[0104] The production batch number includes the batch number for each production run and associates the raw material formula with a specific production batch.

[0105] Production parameters include other production parameters related to each batching, such as equipment status, temperature, and pressure.

[0106] The quality inspection results include quality inspection data for each batch of glass products, such as transparency, strength, and heat resistance.

[0107] Remove or correct outliers, missing values, or erroneous data from the historical raw material ratio data to ensure data quality and consistency. Divide the cleaned historical raw material ratio data into training and testing sets proportionally, with the training set accounting for 70% and the testing set accounting for 30%.

[0108] The initial proportioning determination module determines the required amount of glass raw materials based on the raw material formula and expert experience.

[0109] Furthermore, the initial proportion determination module includes:

[0110] The expert experience includes information on glass raw materials and other raw materials. The glass raw material information includes raw material conversion rate, raw material purity, and raw material volatility. The other raw material information includes the content of additives. This information is derived from the accumulation of historical data and the long-term experience of experts.

[0111] Among them, the content of additives is introduced in the form of external additions, such as sodium nitrate as an oxidant introduced at 4% of the batch, copper oxide as a colorant introduced at 1.5% of the batch, and cerium dioxide as a clarifying agent introduced at 0.5% of the batch.

[0112] The raw material formulation includes the target proportions of the raw materials and the total mass of glass preparation;

[0113] Based on the expert experience and the raw material formula, the required amount of glass raw materials is calculated, and the formula for calculating the required amount of glass raw materials is as follows:

[0114]

[0115] Among them, O i Let P be the demand for the i-th type of glass raw material. i Let T be the target proportion of the i-th raw material, T be the total mass of the glass prepared, and C be the total mass of the glass prepared. i E represents the purity of the i-th raw material. i Let V be the conversion rate of the i-th raw material. i Let be the volatilization rate of the i-th raw material.

[0116] Specifically, the raw material for SiO2 in the glass composition is quartz sand, with a conversion rate of 100%, a purity of 99.7%, and a volatility of 0. If the target proportion is set to 60%, and the total mass of glass produced is 1000g, then the required amount of quartz sand can be calculated to be 601.80g. The initial proportion determination module can accurately consider factors such as purity, conversion rate, and volatility, ensuring the accuracy of the raw material proportions, thereby improving the quality of the glass products and the stability of the production process.

[0117] The status information acquisition module acquires the current status information of the device based on the first dataset.

[0118] Furthermore, the current status information includes:

[0119] The equipment status includes the equipment on state and the equipment off state. When the equipment is on, raw materials can be fed or mixed, while when it is off, maintenance or cleaning work can be carried out.

[0120] Environmental information includes the temperature, humidity, and pressure of the production environment;

[0121] The raw material formula information includes the raw material formula and a numerical serial number, wherein the numerical serial number corresponds one-to-one with the raw material formula;

[0122] The formula change information includes indicators for no formula change and indicators for formula change.

[0123] Specifically, the system records multiple raw material formulas, each with a unique numerical serial number. For example, formula 1 has the serial number 001, corresponding to a standard glass formula; formula 2 has the serial number 002, corresponding to a high-strength glass formula. During production, the system retrieves the corresponding raw material formula from the formula library and applies it based on the input mixing ratio request. Through the status information acquisition module, the system dynamically adjusts the mixing parameters based on real-time monitoring of equipment status and environmental information, ensuring the accuracy of the mixing ratio under different conditions. In addition, the system can quickly respond to formula changes, ensuring a smooth switch between different formulas and avoiding quality fluctuations caused by formula changes, enabling the production system to better meet the needs of modern manufacturing.

[0124] The RBC model building module is used to generate a first experience sample based on the current state information and the glass raw material demand, and to store the first experience sample in the experience playback pool.

[0125] Furthermore, such as Figure 2 As shown, the first experience sample generation step includes:

[0126] Step S20: Initialize the experience replay pool, and randomly extract data including all recipes from the first dataset to obtain the second dataset;

[0127] The second dataset includes sample data for all recipes, with 30 data points randomly selected from each recipe to ensure that each recipe has at least 30 samples in the second dataset.

[0128] Step S21: Obtain the required amount of glass raw materials based on the second dataset and the current status information;

[0129] Step S22: Assign the required amount of glass raw materials to the current mixing ratio behavior, input the current mixing ratio behavior into the equipment, and obtain the mixing ratio reward and the status information of the next batch;

[0130] Step S23: Combine the current status information, the current allocation behavior, the allocation reward, and the next batch status information to obtain the first experience sample, and store the first experience sample in the experience replay pool;

[0131] Step S24: If the first empirical sample reaches the first threshold, the matching process ends; otherwise, the status information of the next batch is used as the current status information of the next batch matching process.

[0132] The first threshold is set to 300.

[0133] Step S25: Determine the formula change information and raw material formula information based on the current status information. If a formula change flag is obtained, it is not necessary to obtain the glass raw material requirement, and steps S22 to S24 are repeated. If a formula change flag is obtained, steps S21 to S24 are repeated.

[0134] Specifically, the RBC (Rule-Based Control) model is a control system based on predefined rules, used to generate an initial experience sample after interacting with the equipment. In this model, the current mixing ratio is directly set to the required amount of glass raw materials based on expert experience and formula information. The current mixing ratio is only adjusted when the formula changes. By inputting the initial experience sample generated based on expert experience into the experience playback pool, the RBC model can use this experience for more effective control and optimization in subsequent operations, thereby improving the accuracy of raw material mixing.

[0135] Furthermore, the matching bonus includes:

[0136] Based on the actual and target proportions of the raw materials, the basic reward R is calculated. base The basic reward function is expressed as:

[0137]

[0138] Among them, P actual,i Let P be the actual proportion of the i-th raw material. target,i The target proportion for the i-th raw material;

[0139] Based on the actual proportions of the raw materials and the current state information, the volatility reward R is calculated. stability ;

[0140] If the formula change information is O change If the value is 0, it indicates that the formula has not changed, and the volatility reward is equal to the sum of the variances of the actual proportions of the n raw materials; if the formula change information is 1, it indicates that the formula has changed, and the volatility reward R... stability When the value equals 0, the volatility reward function is expressed as:

[0141]

[0142] Where Var is the variance of the actual proportion of the i-th raw material, Ochange For formula change information, O change =0 indicates that the formula has not changed, O change =1 indicates a formula change, the type of raw material is the sum of glass raw material ratio and additives, and n is set to 20.

[0143] Specifically, the basic reward function improves the quality of glass products by penalizing deviations between the actual and target proportions. This function prompts the model to reduce proportioning errors, thereby ensuring the final product meets expected quality standards. The volatility reward function, on the other hand, focuses on penalizing excessive proportioning fluctuations and can adapt to formula changes. For example, if there are 10 time steps O... change =0, then calculate the variance of the actual proportions of all raw materials in these 10 time steps, sum them up to get 0.2, then assign 0.2 to the volatility reward. If O change If the value is 1, the reward is recalculated. Calculating the volatility reward function helps stabilize the production process, reduce drastic changes in raw material ratios, ensure production stability and consistency, and thus improve the accuracy of raw material ratios.

[0144] Furthermore, the ratio bonus is calculated by weighting the base bonus, the volatility bonus, equipment energy consumption, and raw material loss, and is expressed as follows:

[0145] R = w1 × R base -w2×R stability -w3×E efficiency -w4×C waste ;

[0146] Where R is the matching bonus, E efficiency For equipment energy consumption, C waste For raw material loss, w1, w2, w3, and w4 are the weighting factors for the basic reward, the volatility reward, the equipment energy consumption, and the raw material loss, respectively.

[0147] After normalizing all reward items, w1, w2, w3, and w4 are set to 0.6, 0.2, 0.1, and 0.1, respectively.

[0148] Specifically, the high weighting of the base reward ensures that the deviation between the actual and target proportions is fully considered. This setting emphasizes the importance of proportioning accuracy, helping to improve the quality of the final glass product and ensuring it meets production standards and user requirements. The equipment energy consumption reward encourages optimized energy use, while the raw material loss reward prompts the model to reduce raw material waste during production. By incorporating multiple reward items into the total reward calculation and assigning weights, this comprehensive optimization strategy ensures that the production process not only focuses on quality but also emphasizes cost control and resource utilization, thereby improving the accuracy of raw material proportioning.

[0149] The raw material ratio determination module uses the current status information and the current raw material ratio information and determines the current ratio behavior based on the TD3 model.

[0150] Furthermore, the raw material proportioning determination module includes:

[0151] The current raw material ratio information includes the ratios of n glass raw materials and the content of j additives. The current raw material ratio information is expressed as follows:

[0152] A = {p1, p2, ..., p} n ,k1,k2…k j};

[0153] Where A represents the current raw material ratio information, p1, p2, ... p n These represent the proportions of the first type of glass raw material, the second type of glass raw material, and the nth type of glass raw material, k1, k2…k j The contents of the first type of admixture, the second type of admixture, and the j-th type of admixture are respectively, with n set to 15 and j set to 5;

[0154] The current status information is input into the strategy network, and the strategy network generates a first raw material ratio based on the current raw material ratio information.

[0155] Specifically, the process of generating the first raw material ratio using the strategy network is as follows:

[0156] The input layer receives the current state information and the current raw material ratio information as input. The hidden layer adjusts the ratio of glass raw materials by capturing the complex relationship between the input information and its influence on the ratio. The output layer generates a vector, which is the first raw material ratio, and each element represents the suggested ratio of the corresponding glass raw material.

[0157] Adding Euler noise to the first raw material ratio yields the current ratio behavior, which is represented as follows:

[0158] a = m(s) + noise;

[0159] Where a represents the current proportioning behavior, m(s) represents the first raw material proportioning under the current state information, and noise represents the Euler noise;

[0160] The Euler noise formula is:

[0161] d(noise t ) = b × (c - noise) t )×dt+f×dW t ;

[0162] Where the initial value of Euler noise is set to 0, the termination value of Euler noise is set to 1e-3, b is the regression rate, b is set to 0.15, c is the long-term average value of noise, c is set to 0, f is the noise intensity, f is set to 0.2, and dW t It is Brownian motion;

[0163] Specifically, Ornstein-Uhlenbeck noise is a noise model used to enhance policy exploration capabilities, particularly widely used in the TD3 model. Its introduction helps the policy network avoid getting trapped in local optima during the raw material proportioning exploration phase, thus better coping with complex production environments. For example, in glass manufacturing, if the current environmental conditions change slightly (e.g., the temperature increases by 1°C), the current proportioning behavior with Ornstein-Uhlenbeck noise can gradually adjust the proportions over a continuous period, rather than jumping drastically, maintaining the stability of the proportioning behavior. The policy network considers the current environmental conditions when generating the first raw material proportions, and with the assistance of Ornstein-Uhlenbeck noise, further refines the action selection, ensuring that the proportioning behavior remains accurate even in dynamically changing production environments.

[0164] The raw material proportioning control module inputs the current proportioning behavior into the equipment to obtain proportioning rewards and next batch status information; combines the current status information, the current proportioning behavior, the proportioning rewards, and the next batch status information to obtain a second experience sample, and stores the second experience sample in the experience playback pool; obtains a trained TD3 model based on the experience playback pool, and obtains the optimized raw material proportion based on the trained TD3 model.

[0165] Specifically, the TD3 (Twin Delayed Deep Deterministic Policy Gradient) model is a reinforcement learning algorithm for continuous action spaces, belonging to the category of policy gradient methods. For example... Figure 3 As shown, the TD3 model selects a current matching behavior at each time step and applies it to the device. After receiving the current matching behavior, the device's current state information changes accordingly, generating a matching reward signal that is fed back to the TD3 model. Based on the obtained matching reward and the next batch of state information, the TD3 model further selects the next batch of matching behaviors. The principle of selecting matching behaviors aims not only to maximize the current matching reward but also to consider the impact on a batch of state information in the environment, in order to maximize the final cumulative matching reward.

[0166] Furthermore, such as Figure 4 As shown, the deep neural network training process of the TD3 model includes:

[0167] The deep neural network includes a policy network m. uFirst Value Network Second Value Network Target policy network m u' First Target Value Network Second target value network Where u is the policy network parameter, u1 and u2 are policy network parameters, u' is the target policy network parameter, and u1' and u2' are target policy network parameters;

[0168] A batch of the first experience samples and the second experience samples are randomly selected from the experience replay pool and fused together as (s,a,r,s'), where s is the current state information, a is the current matching behavior, r is the matching reward, and s' is the state information of the next batch.

[0169] Based on the aforementioned empirical samples, the next matching behavior is generated using the target policy network:

[0170] a'=m u' (s')+noise;

[0171] Where a' is the next matching behavior and noise is Euler noise;

[0172] Next, the first target Q-value and the second target Q-value are calculated using the first target value network and the second target value network, respectively. The target Q-value is the minimum value between the first target Q-value and the second target Q-value, expressed as:

[0173]

[0174] Where y is the target Q value, and k is the discount factor, set to 0.99;

[0175] The parameters of the first and second value networks are updated by minimizing the mean square error between the actual Q-values ​​and the target Q-values ​​of the first and second value networks. The objective function for the update is expressed as:

[0176]

[0177] Where N is the batch size of the empirical sample, set to 64. b is the mean squared error of the i-th value network. Q The learning rate for the value network is set to 0.001, and j is the sample number. It is the gradient of the i-th value network, where i is 1 or 2;

[0178] The parameters of the policy network are updated by maximizing the Q-value of the first value network, as follows:

[0179]

[0180]

[0181] Among them, b m The learning rate for the policy network is set to 0.001. It is the Q-value of the policy network. It is the gradient of the policy network;

[0182] Every q steps, a soft update is used to update the parameters of the target policy network, the first target value network, and the second target value network based on the parameters of the policy network, the first value network, and the second value network, as shown below:

[0183] u'←zu+(1-z)u';

[0184] u i '←zu i +(1-z)u i ';

[0185] Where q is set to 2 and the soft update coefficient z is set to 0.005.

[0186] Specifically, for example, the current state is that the equipment is operating at a high temperature and requires 1000 kg of raw materials. The RBC model generates a preliminary raw material ratio based on the state information: 40% silica sand, 30% sodium ash, and 30% lime. This ratio is recorded as the first empirical sample. After TD3 optimization, the first empirical sample is found to be unsuitable for the current equipment state, and the ratio is adjusted to 38% silica sand, 32% sodium ash, and 30% lime. The equipment provides a +3 point bonus, indicating improved production quality. The current state information is updated to show a slight decrease in equipment temperature and that raw material consumption is in line with expectations. This information is recorded as the second empirical sample. By combining the first empirical sample generated by the RBC model with the TD3 model for optimization, the raw material ratio can be optimized in real time to adapt to changes in production, thereby improving the accuracy of the raw material ratio.

[0187] The anomaly management module is used to detect and handle abnormal situations during the ingredient preparation process.

[0188] Furthermore, the abnormal situations include:

[0189] The proportioning deviation is calculated based on the actual proportion of raw materials and the target proportion of raw materials, and is expressed as follows:

[0190]

[0191] Where, Δp i Let P be the deviation in the proportion of the i-th raw material. actual,i Let P be the actual proportion of the i-th raw material.target,i The target proportion for the i-th raw material;

[0192] If the ratio deviation is less than the second threshold, the current ratio behavior is not updated;

[0193] If the ratio deviation is greater than or equal to the second threshold, the first anomaly is automatically recorded;

[0194] If the ratio deviation still exceeds the second threshold after f time steps, then manual intervention is notified;

[0195] The second threshold is set to 1%, and f is set to 5.

[0196] The raw material loss is calculated based on the actual proportion of the raw materials and the current proportioning behavior, and is expressed as follows:

[0197] C waste,i =P i -P actual,i ;

[0198] Among them, C waste,i For the loss of the i-th raw material, P i This refers to the current mixing ratio behavior of the i-th type;

[0199] If the raw material loss exceeds the third threshold, a second anomaly will be automatically recorded.

[0200] If the ratio deviation still exceeds the third threshold after g time steps, then manual intervention is notified;

[0201] The third threshold is set to 500 kg and g is set to 10.

[0202] Specifically, assuming that in the glass production process, SiO2 accounts for 70%, Na2O 15%, and CaO 15%, the actual proportions extracted from the production data are 69%, 15%, and 14.2%, respectively. The deviation in the SiO2 proportion exceeds the second threshold, thus requiring an anomaly to be recorded. If the proportion deviation is large, the reward value may be low. The TD3 model, receiving a negative proportion reward during the control process, will optimize its proportioning behavior to attempt to correct the deviation. If the SiO2 proportion deviation exceeds the second threshold after 5 time steps, it can be considered an anomaly in the optimization model or the equipment itself. Manual verification of the TD3 model's parameter settings is necessary, whether model retraining is required, and checking for faults or inaccurate adjustments in the raw material feeding system, sensors, and control system. After investigation, ensuring the problem is resolved will improve the accuracy of the raw material proportioning.

[0203] In summary, the formula management module flexibly manages glass preparation formulas, enabling rapid addition and deletion of formulas and improving the adaptability of the production line; the data processing module combines historical data and new formulas to ensure data accuracy, laying the foundation for optimizing raw material ratios; the initial ratio determination module provides reliable ratio references using expert experience; the RBC model and status information acquisition module generate high-quality experience samples, accelerating the optimization of raw material ratios; the TD3 model intelligently adjusts ratio behavior based on the current status, dynamically adapting to production changes and ultimately achieving higher precision in raw material ratio control; the anomaly management module provides real-time detection and processing functions, promptly addressing any anomalies that may occur during production, ensuring the stability of system operation and the high quality of glass products.

[0204] Example 2

[0205] This embodiment implements an automatic control method for the proportioning of raw materials in glass preparation, thereby solving the problems of accuracy and intelligence in the proportioning of raw materials during the glass preparation process.

[0206] like Figure 5 As shown, an automatic control method for the proportioning of raw materials in glass preparation includes:

[0207] Collect the raw material formula for the glass preparation;

[0208] Collect historical data on raw material ratios, add the raw material formula to the historical data on raw material ratios, and perform data preprocessing to obtain the first dataset;

[0209] Based on the aforementioned raw material formula and expert experience, the required amount of glass raw materials is obtained;

[0210] Based on the first dataset, obtain the current status information of the device;

[0211] A first experience sample is generated based on the current status information and the glass raw material demand, and the first experience sample is stored in the experience playback pool;

[0212] The current state information and current raw material ratio information are used to determine the current ratio behavior based on the TD3 model;

[0213] The current proportioning behavior is input into the device to obtain proportioning reward and next batch status information; the current status information, the current proportioning behavior, the proportioning reward and the next batch status information are combined to obtain a second experience sample, and the second experience sample is stored in the experience replay pool; a trained TD3 model is obtained based on the experience replay pool, and the optimized raw material proportion is obtained based on the trained TD3 model;

[0214] Detect and handle any abnormalities during the ingredient preparation process.

[0215] Specifically, to demonstrate the effectiveness of an automatic control method for the proportioning of raw materials in glass preparation, the method is designated as Scheme 4, and three comparative schemes are proposed for testing. Scheme 1 uses the RBC model to adjust the control strategy according to the predetermined formula and the required amount of glass raw materials, without using a reinforcement learning model. Scheme 2 uses only the DDPG model, with the same state information, proportioning information, and proportioning reward as Scheme 4. Scheme 3 uses only the TD3 model, with the same state information, proportioning information, and proportioning reward as Scheme 4.

[0216] As shown in Table 1, Scheme 4 outperforms the comparison schemes in all indicators. Scheme 4's proportioning deviation is 0.9%, significantly lower than other schemes, indicating higher precision in proportioning control. Scheme 4 controls raw material loss to 500 kg, significantly reducing waste and improving utilization. Scheme 4's anomaly rate is 2.1%, significantly lower than other schemes, indicating a substantial reduction in the frequency of anomalies in actual production. Scheme 4's energy consumption is 400 kWh / ton of glass, demonstrating significantly better energy efficiency than other schemes. Scheme 4 combines the TD3 model with prior knowledge for optimization strategies, achieving precise control in the glass production process, significantly improving production efficiency, reducing energy consumption, and effectively minimizing raw material waste and anomalies, thus enhancing the accuracy of raw material proportioning.

[0217] Table 1. Performance Comparison of Different Solutions

[0218] plan Mixing deviation % Raw material loss kg Abnormality rate % Energy consumption kWh / ton of glass Option 1 1.0 1002 2.4 435 Option 2 1.8 850 4.7 432 Option 3 1.4 742 4.5 425 Option 4 0.9 500 2.1 401

[0219] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An automatic control system for the proportioning of raw materials in glass preparation, characterized in that, include: The recipe management module is used to add and delete raw material recipes for glass preparation; The data processing module is used to collect historical data on raw material ratios, add the raw material formulas to the historical data on raw material ratios, and perform data preprocessing to obtain the first dataset; The initial proportioning determination module determines the required amount of glass raw materials based on the raw material formula and expert experience; The status information acquisition module acquires the current status information of the device based on the first dataset; The RBC model building module is used to generate a first empirical sample based on the current state information and the glass raw material demand, and to store the first empirical sample in the empirical playback pool. The raw material ratio determination module uses the current status information and the current raw material ratio information and determines the current ratio behavior based on the TD3 model. The raw material proportioning control module inputs the current proportioning behavior into the equipment and obtains proportioning rewards and status information for the next batch; The matching bonus includes: The basic reward is calculated based on the actual ratio of raw materials and the target ratio of raw materials. The volatility reward is calculated based on the actual proportions of the raw materials and the current state information. If the formula change information is 0, it indicates that the formula has not changed, and the volatility reward is equal to the sum of the variances of the actual proportions of the n raw materials; if the formula change information is 1, it indicates that the formula has changed, and the volatility reward is equal to 0. The ratio bonus is calculated by weighting the base bonus, volatility bonus, equipment energy consumption, and raw material loss. The current state information, current ratio behavior, ratio reward, and next batch state information are combined to obtain a second experience sample, which is then stored in the experience replay pool. The trained TD3 model is obtained based on the experience replay pool, and the optimized raw material ratio is obtained by combining the first experience sample generated by the RBC model. The anomaly management module is used to detect and handle abnormal situations during the ingredient preparation process.

2. The automatic control system for the proportioning of raw materials for glass preparation according to claim 1, characterized in that, The initial proportion determination module includes: The expert experience includes information on glass raw materials and other raw materials. The glass raw material information includes raw material conversion rate, raw material purity, and raw material volatility. The other raw material information includes the content of additives. The raw material formulation includes the target proportions of the raw materials and the total mass of glass preparation; Based on the expert experience and the raw material formula, the required amount of glass raw materials was calculated.

3. The automatic control system for the proportioning of raw materials for glass preparation according to claim 1, characterized in that, The current status information includes: Device status includes device on status and device off status; Environmental information includes the temperature, humidity, and pressure of the production environment; The raw material formula information includes the raw material formula and a numerical serial number, wherein the numerical serial number corresponds one-to-one with the raw material formula; The formula change information includes indicators for no formula change and indicators for formula change.

4. The automatic control system for the proportioning of raw materials for glass preparation according to claim 1, characterized in that, The raw material proportioning determination module includes: The current raw material ratio information includes the ratios of n glass raw materials and the content of j additives; The current status information is input into the strategy network, and the strategy network generates a first raw material ratio based on the current raw material ratio information. Euler noise is added to the first raw material ratio to obtain the current ratio behavior.

5. The automatic control system for the proportioning of raw materials for glass preparation according to claim 1, characterized in that, The deep neural network training process of the TD3 model includes: The deep neural network includes a policy network, a first value network, a second value network, a target policy network, a first target value network, and a second target value network; Randomly select a batch of the first experience samples and the second experience samples from the experience replay pool; Based on the extracted experience samples, the first target Q value and the second target Q value are calculated using the first target value network and the second target value network, respectively. The target Q value is the minimum value between the first target Q value and the second target Q value. The parameters of the first value network and the second value network are updated by minimizing the mean square error between the actual Q value and the target Q value of the first value network and the second value network. The parameters of the policy network are updated by maximizing the Q-value of the first value network; Every q steps, a soft update is used to update the parameters of the target policy network, the first target value network, and the second target value network based on the parameters of the policy network, the first value network, and the second value network.

6. The automatic control system for the proportioning of raw materials for glass preparation according to claim 1, characterized in that, The first empirical sample generation step includes: Step S20: Initialize the experience replay pool, and randomly extract data including all recipes from the first dataset to obtain the second dataset; Step S21: Obtain the required amount of glass raw materials based on the second dataset and the current status information; Step S22: Assign the required amount of glass raw materials to the current mixing ratio behavior, input the current mixing ratio behavior into the equipment, and obtain the mixing ratio reward and the status information of the next batch; Step S23: Combine the current status information, the current allocation behavior, the allocation reward, and the next batch status information to obtain the first experience sample, and store the first experience sample in the experience replay pool; Step S24: If the first empirical sample reaches the first threshold, the matching process ends; otherwise, the status information of the next batch is used as the current status information of the next batch matching process. Step S25: Determine the formula change information and raw material formula information based on the current status information. If a formula change flag is obtained, it is not necessary to obtain the glass raw material requirement, and steps S22 to S24 are repeated. If a formula change flag is obtained, steps S21 to S24 are repeated.

7. The automatic control system for the proportioning of raw materials for glass preparation according to claim 1, characterized in that, The abnormal situations include: The ratio deviation is calculated based on the actual ratio of raw materials and the target ratio of raw materials. If the ratio deviation is less than the second threshold, the current ratio behavior is not updated. If the ratio deviation is greater than or equal to the second threshold, the first anomaly is automatically recorded; If the ratio deviation still exceeds the second threshold after f time steps, then manual intervention is notified; The raw material loss is calculated based on the actual ratio of the raw materials and the current ratio behavior. If the raw material loss is greater than the third threshold, the second anomaly is automatically recorded. If the ratio deviation still exceeds the third threshold after g time steps, then the manual intervention is notified.

8. An automatic control method for the proportioning of raw materials in glass preparation, characterized in that, include: Collect the raw material formula for the glass preparation; Collect historical data on raw material ratios, add the raw material formula to the historical data on raw material ratios, and perform data preprocessing to obtain the first dataset; Based on the aforementioned raw material formula and expert experience, the required amount of glass raw materials is obtained; Based on the first dataset, obtain the current status information of the device; A first experience sample is generated based on the current status information and the glass raw material demand, and the first experience sample is stored in the experience playback pool; The current state information and current raw material ratio information are used to determine the current ratio behavior based on the TD3 model; The current proportioning behavior is input into the device to obtain proportioning rewards and the status information for the next batch; The matching bonus includes: The basic reward is calculated based on the actual ratio of raw materials and the target ratio of raw materials. The volatility reward is calculated based on the actual proportions of the raw materials and the current state information. If the formula change information is 0, it indicates that the formula has not changed, and the volatility reward is equal to the sum of the variances of the actual proportions of the n raw materials; if the formula change information is 1, it indicates that the formula has changed, and the volatility reward is equal to 0. The ratio bonus is calculated by weighting the base bonus, volatility bonus, equipment energy consumption, and raw material loss. The current state information, the current proportioning behavior, the proportioning reward, and the next batch of state information are combined to obtain a second experience sample, which is then stored in the experience replay pool. The trained TD3 model is obtained based on the experience replay pool, and the optimized raw material proportion is obtained by combining the first experience sample generated by the RBC model. Detect and handle any abnormalities during the ingredient preparation process.

Images