Intelligent ultrathin glass tempering control method based on graph neural network and reinforcement learning

Abstract

The invention discloses an autonomous positioning method based on a human body space displacement model and an inertial navigation system. The autonomous positioning method comprises the following steps: S1, data acquisition and cleaning; s2, data processing; s3, constructing a graph neural network; s4, constructing a time sequence convolutional network model; and S5, reinforcement learning optimization control. According to the method, temperature, pressure, stress and other data in the glass tempering process are collected and cleaned, spatial characteristics are extracted by utilizing a graph neural network, time sequence characteristics are captured in combination with a time sequence convolutional network model, and finally, control parameters are optimized by adopting a reinforcement learning algorithm, so that intelligent adjustment of argon flow velocity, crucible rotating speed and other parameters in the cooling process is realized; the temperature difference between the glass surface and the center is reduced, stress deformation is reduced, cooling uniformity is improved, and therefore the quality of tempered glass is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of a glass tempering production process, and in particular to an intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning. Background Art

[0002] In modern industrial production, the tempering process of ultra-thin glass is highly complex and requires refinement. Tempered glass needs to be heated at high temperatures and then cooled rapidly to form a compressive stress layer on its surface, thereby improving the strength and impact resistance of the glass. However, in the actual production process, the control of physical parameters such as temperature, pressure and stress is crucial to the quality of the final product.

[0003] Traditional glass tempering control methods usually rely on experience and simple rule control, and cannot adjust production parameters in real time and accurately, resulting in unstable product quality, severe stress deformation, low energy utilization, etc. In addition, with the increasing application of ultra-thin glass, its thickness is getting thinner and thinner, which puts higher requirements on the control accuracy of the production process.

[0004] In recent years, the development of machine learning and deep learning technologies has provided new ideas for the optimization control of industrial processes. Among them, graph neural networks (GNNs) can effectively process data with topological structures and capture spatial correlation characteristics; reinforcement learning (RL) can learn the optimal control strategy through interaction with the environment. However, applying these advanced artificial intelligence technologies to the glass tempering process still faces challenges such as complex data processing and difficulty in real-time application of models. Summary of the invention

[0005] In view of this, the present invention provides an intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning, aiming to solve the problems of insufficient control accuracy and difficulty in real-time optimization of the glass tempering process in the prior art.

[0006] An intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning provided by an embodiment of the present invention includes the following steps: S1, data acquisition and cleaning: various parameter data in the glass tempering process are collected through temperature sensors, pressure sensors and stress sensors installed on the production line, and the collected various parameter data are cleaned, missing values, abnormal values ​​and repeated values ​​are processed to obtain cleaned parameter data, wherein the collected various parameter data include temperature, pressure and stress; S2, data processing: according to the importance of the data dimension, the cleaned parameter data are divided into three levels: low level, medium level and high level, and corresponding level labels are added to the cleaned parameter data, and the cleaned parameter data are divided into a training set, a verification set and a test set according to a preset ratio; S3, constructing a graph neural network: selecting a graph convolutional neural network as the architecture of the graph neural network, and inputting the collected various parameter data as node features into the graph neural network for spatial feature capture to output node high-dimensional features, wherein each node represents a production process. The position of the online sensor, and the edge represents the physical relationship between different nodes; S4, constructing a time series convolutional network model: based on the high-dimensional features of the output nodes, the time series characteristics of various parameter data are captured through the constructed time series convolutional network model, and the temperature, pressure and stress in the glass tempering process are predicted in time to generate prediction results, wherein the time series convolutional network model is trained through the training set, the hyperparameters of the time series convolutional network model are adjusted through the validation set, and the prediction effect of the time series convolutional network model is evaluated through the test set; S5, reinforcement learning optimization control: based on the prediction results of the time series convolutional network model, the reinforcement learning algorithm is used to optimize the control parameters of the cooling system in the glass tempering process and design a reward function to reduce the temperature difference between the glass surface and the center, reduce stress deformation, and improve cooling uniformity, thereby improving the quality of tempered glass, wherein the state of the cooling system is the future changes of temperature, pressure, and stress of each node predicted by the time series convolutional network model, and the action of the cooling system is to adjust the control parameters in the cooling process.

[0007] In one implementation, the processing of missing values, outliers and duplicate values ​​in step S1 includes supplementing missing values ​​through interpolation or context information, removing outliers through statistical analysis, and processing duplicate values ​​through comparison and deduplication.

[0008] In another implementation, in step S2, the cleaned parameter data is divided into a training set, a validation set and a test set according to a preset ratio of 70%:15%:15%.

[0009] In another implementation, the specific operation steps of selecting the graph convolutional neural network as the architecture of the graph neural network in step S3 are as follows:

[0010] Define each sensor location as a node vi , the feature vector of each node is:

[0011] x i =[T i , P i , S i ]

[0012] Among them, T i Indicates the temperature sensor reading, P i Indicates the pressure sensor reading, S i Indicates the stress sensor reading;

[0013] Construct an adjacency matrix A, where A ij =1 indicates that there is a connection between node i and node j, otherwise it is 0. In the graph convolutional neural network, the node features are updated by information interaction with neighboring nodes. The node features of the lth layer are represented by H (l) , the update rule is:

[0014]

[0015] in, Add self-loops to include the node’s own information, I is the unit matrix, yes The degree matrix of H(l) is the node feature matrix of layer l, W(l) is the learnable weight matrix, and σ is the activation function;

[0016] Use the Adam optimizer to update the model parameters of the graph convolutional neural network.

[0017] In another implementation, step S4 specifically includes the following steps:

[0018] The high-dimensional features of the nodes output by the graph convolutional neural network are used as the input of the time series convolutional network model to form a time series dataset:

[0019] X TCN =[H (L) (t-n+1), H (L) (t-n+2), ..., H (L) (t)]

[0020] Among them, n is the length of the time window, t is the current time, H (L) (t) represents the node features at time t;

[0021] The temporal convolutional network model consists of multiple stacked causal convolutional layers. The convolution kernel size of each layer is k, and the expansion factor is d. The formula is:

[0022]

[0023] Among them, O (l) is the output of layer l, Denotes the expansion factor as d l The convolution operation uses the mean square error to measure the difference between the predicted value and the true value:

[0024]

[0025] in, is the predicted output of the temporal convolutional network model, Y i is the true value, N is the number of samples;

[0026] Use the Adam optimizer to update the model parameters of the temporal convolutional network model.

[0027] In another implementation, the state of the cooling system in step S5 is composed of the temperature, pressure, and stress changes of each node within a specified period of time in the future predicted by the temporal convolutional network model, and the state of the cooling system is expressed as:

[0028]

[0029] Here, k is the time step of prediction.

[0030] In another implementation, the action of the cooling system in step S5 is to adjust the control parameters in the cooling process, the control parameters include the argon flow rate F and the crucible rotation speed R, and the action of the cooling system is expressed as:

[0031] a t =[ΔF t , ΔR t ]

[0032] Where, ΔF t is the adjustment amount of argon gas flow rate, ΔR t is the adjustment amount of the crucible speed.

[0033] In another implementation, the reward function formula in step S5 is:

[0034] R(s t , a t )=-(α1·ΔT t +α1·ΔS t +α3·U t )

[0035] Among them, α1, α2, α3 are weight coefficients, ΔT t Indicates the temperature difference between the glass surface and the center, ΔS t Indicates stress deformation, U t Indicates uneven cooling.

[0036] In another implementation, the reinforcement learning algorithm used in step S5 is specifically a deep deterministic policy gradient algorithm.

[0037] In another implementation, the specific cycle process of the deep deterministic policy gradient algorithm is as follows:

[0038] For each state s in time step t , perform the following steps:

[0039] a. Get the current status s t :Using the temporal convolutional network model to predict future changes in physical parameters Combined with the current control parameter F t and R t , forming state s t ;

[0040] b. Select and execute action a t :Use the policy network to generate action a t =μ(s t |θ μ ) adjusting the argon gas flow rate and the crucible speed;

[0041] c. Environmental feedback: obtain actual temperature, pressure and stress sensor data and update environmental status;

[0042] d. Calculate the reward function R(s t , a t ): According to the new physical parameters obtained in steps ac, calculate the temperature difference ΔT between the glass surface and the center t , stress deformation ΔS t and cooling non-uniformity U t ;

[0043] e. State transfer: update state s t to t+1 , prepare the decision for the next time step;

[0044] f.Store experience: store the experience tuple (s t , a t ,R(s t , a t ), s t+1 ) is stored in the experience replay buffer;

[0045] g. Based on the decision made by the policy network after feedback iteration, the control parameters are continuously optimized and updated until the temporal convolutional network model converges to obtain the optimal control parameters.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] (1) The present invention discloses an intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning. Under the conditions of ensuring data privacy and real-time performance, it realizes high-precision and intelligent control of the glass tempering process, and solves the problem that traditional glass tempering control methods usually rely on manual experience and simple rule control, which makes it difficult to adjust production parameters in real time and accurately, resulting in unstable product quality, severe stress deformation and other problems.

[0048] (2) The present invention effectively handles the spatial correlation of sensor data in the production process by introducing a graph neural network (GCN). Each sensor location is modeled as a node in the graph, and the edges represent the physical relationships between sensors, such as spatial proximity and heat conduction paths. Through the spatial feature extraction capability of GCN, the spatial dependencies of various physical parameters can be deeply explored, improving the model's ability to characterize complex production processes.

[0049] (3) The present invention uses the temporal convolutional network (TCN) model to capture the time series characteristics of sensor data and make high-precision predictions of key parameters such as temperature, pressure, and stress at future moments. TCN uses a combination of causal convolution and dilated convolution, which can effectively capture dependencies over a long period of time.

[0050] (4) On this basis, the present invention also uses a reinforcement learning algorithm to learn the optimal control strategy, so that the cooling system can adjust the control parameters in the cooling process, such as argon gas flow rate and crucible speed, in real time according to the predicted physical parameter changes. Through a carefully designed reward function, the cooling system can maximize the reduction of the temperature difference between the glass surface and the center, reduce stress deformation, and improve cooling uniformity, thereby improving the quality of tempered glass.

[0051] (5) The solution of the present invention significantly improves the accuracy and real-time performance of the glass tempering process control, reduces manual intervention and reliance on experience, optimizes energy utilization, and reduces production costs. At the same time, the present invention ensures the stability of the production process and the consistency of product quality by making full use of sensor data and integrating advanced algorithms, and has important industrial application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flowchart of the steps of the intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning in an embodiment of the present invention.

[0053] Figure 2 For Figure 1 The corresponding overall flow chart of the intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning according to the embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] For ease of understanding, before describing the specific embodiments of the present invention in detail, the prior art of the present invention is first exemplified. The prior art solutions include:

[0056] (1) Data collection and preprocessing: By installing sensors on the production line, real-time data on key parameters such as temperature, pressure, and stress are collected. The collected data is then cleaned and missing values, outliers, and duplicate values ​​are processed to ensure data quality and reliability.

[0057] (2) Model construction and training: Use traditional neural network models, such as feedforward neural networks (FNN) or convolutional neural networks (CNN), to train the cleaned data and establish prediction models for parameters such as temperature and pressure.

[0058] (3) Control strategy optimization: Based on the output of the prediction model, a simple optimization algorithm (such as PID control) is used to adjust the control parameters during the cooling process to minimize temperature difference and stress deformation.

[0059] In summary, although the existing technology has applied machine learning and deep learning methods to a certain extent to optimize the control of the glass tempering process, there are still significant deficiencies in spatial feature capture, time series prediction, control strategy optimization, real-time performance, and energy efficiency. Therefore, a new method that can comprehensively, accurately, and intelligently control the ultra-thin glass tempering process is urgently needed to overcome the limitations of the existing technology and improve product quality and production efficiency.

[0060] Specifically, according to Figure 1 , Figure 2 Further describing the solution of the present invention, an intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning provided by an embodiment of the present invention mainly includes the following steps:

[0061] S1. Data collection and cleaning: various parameter data in the glass tempering process are collected through temperature sensors, pressure sensors and stress sensors installed on the production line, and the collected parameter data are cleaned, missing values, abnormal values ​​and repeated values ​​are processed to obtain cleaned parameter data. The collected parameter data include temperature, pressure and stress;

[0062] It should be understood that the temperature sensor, pressure sensor and stress sensor are used to record the glass surface temperature, air pressure changes and stress conditions. All collected data will be recorded and stored.

[0063] It should also be understood that the data cleaning process will deal with issues such as missing values, outliers, and duplicate values ​​in the data to ensure the quality and reliability of the data.

[0064] S2. Data processing: The cleaned parameter data is divided into three levels: low, medium, and high according to the importance of the data dimension, and corresponding level labels are added to the cleaned parameter data as a reference for subsequent model training. The cleaned parameter data is divided into a training set, a validation set, and a test set according to a preset ratio;

[0065] S3. Build a graph neural network: Select the graph convolutional neural network (GCN) as the architecture of the graph neural network (GNN), and input the collected parameter data as node features into the graph neural network for spatial feature capture to output node high-dimensional features, where each node represents the location of the sensor on the production line, and the edge represents the physical relationship between different nodes;

[0066] S4. Construct a time series convolutional network (TCN) model: Based on the high-dimensional features of nodes output by the graph convolutional neural network (GCN), the constructed time series convolutional network model captures the time series characteristics of various parameter data, and predicts the temperature, pressure and stress in the glass tempering process, and generates prediction results. The training set is used to train the time series convolutional network model, the validation set is used to adjust the hyperparameters of the time series convolutional network model, and the test set is used to evaluate the prediction effect of the time series convolutional network model.

[0067] S5. Reinforcement learning optimization control: Based on the prediction results of the time series convolutional network model, the reinforcement learning algorithm is used to optimize the control parameters of the cooling system in the glass tempering process and design a reward function to reduce the temperature difference between the glass surface and the center, reduce stress deformation, and improve cooling uniformity, thereby improving the quality of tempered glass. The state of the cooling system is the future changes in temperature, pressure, and stress of each node predicted by the time series convolutional network model, and the action of the cooling system is to adjust the control parameters in the cooling process.

[0068] Optionally, the processing of missing values, outliers and duplicate values ​​in step S1 includes supplementing missing values ​​through interpolation or context information, removing outliers through statistical analysis, and processing duplicate values ​​through comparison and deduplication.

[0069] Optionally, in step S2, the cleaned parameter data is divided into a training set, a validation set and a test set according to a preset ratio of 70%:15%:15%.

[0070] Optionally, the specific operation steps of selecting the graph convolutional neural network as the architecture of the graph neural network in step S3 are as follows:

[0071] Define each sensor location as a node v i , the feature vector of each node is:

[0072] x i =[T i , P i , S i ]

[0073] Among them, T i Indicates the temperature sensor reading, P i Indicates the pressure sensor reading, S i Indicates the stress sensor reading;

[0074] Construct an adjacency matrix A, where A ij =1 indicates that there is a connection between node i and node j, otherwise it is 0. In the graph convolutional neural network, the node features are updated by information interaction with neighboring nodes. The node features of the lth layer are represented by H (l) , the update rule is:

[0075]

[0076] in, Add self-loops to include the node’s own information, I is the unit matrix, yes The degree matrix of H (l) is the node feature matrix of the lth layer, w (l) is the learnable weight matrix, σ is the activation function;

[0077] Use the Adam optimizer to update the model parameters of the graph convolutional neural network.

[0078] Optionally, step S4 specifically includes the following steps:

[0079] The high-dimensional features of the nodes output by the graph convolutional neural network are used as the input of the time series convolutional network model to form a time series dataset:

[0080] X TCN =[H (L) (t-n+1), H (L) (t-n+2), ..., H (L) (t)]

[0081] Among them, n is the length of the time window, t is the current time, H (L) (t) represents the node features at time t;

[0082] The temporal convolutional network model consists of multiple stacked causal convolutional layers. The convolution kernel size of each layer is k, and the expansion factor is d. The formula is:

[0083]

[0084] Among them, O (l) is the output of layer l, Denotes the expansion factor as d l The convolution operation uses the mean square error (MSE) to measure the difference between the predicted value and the true value:

[0085]

[0086] in, is the predicted output of the temporal convolutional network model, Y i is the true value, N is the number of samples;

[0087] Use the Adam optimizer to update the model parameters of the temporal convolutional network model.

[0088] Optionally, the state of the cooling system in step S5 is composed of the temperature, pressure, and stress changes of each node within a specified period of time in the future predicted by the temporal convolutional network model, and the state of the cooling system is expressed as:

[0089]

[0090] Here, k is the time step of prediction.

[0091] Optionally, the action of the cooling system in step S5 is to adjust the control parameters in the cooling process, the control parameters include the argon flow rate F and the crucible rotation speed R, and the action of the cooling system is expressed as:

[0092] a t =[ΔF t , ΔR t ]

[0093] Where, ΔF t is the adjustment amount of argon gas flow rate, ΔR t is the adjustment amount of the crucible speed.

[0094] Optionally, the reward function formula in step S5 is:

[0095] R(s t , a t )=-(α1·ΔT t +α1·ΔS t +α3·U t )

[0096] Among them, α1, α2, α3 are weight coefficients, ΔT t Indicates the temperature difference between the glass surface and the center, ΔS t Indicates stress deformation, U t Indicates uneven cooling.

[0097] It should be understood that the reward function is designed to optimize the following objectives:

[0098] 1. Reduce the temperature difference ΔT t : Reduce the temperature difference between the glass surface and the center.

[0099] 2. Reduce stress deformation ΔS t : Reduce the stress deformation of glass.

[0100] 3. Improve cooling uniformity ΔU t : Increase the uniformity of the cooling process.

[0101] Optionally, the reinforcement learning algorithm adopted in step S5 is specifically a deep deterministic policy gradient (DDPG) algorithm.

[0102] Optionally, the specific cycle process of the deep deterministic policy gradient algorithm is as follows:

[0103] For each state s in time step t , perform the following steps:

[0104] a. Get the current status s t :Using the temporal convolutional network model to predict future changes in physical parameters Combined with the current control parameter F t and R t , forming state s t ;

[0105] b. Select and execute action a t :Use the policy network to generate action a t =μ(s t |θ μ ) adjusting the argon gas flow rate and the crucible speed;

[0106] c. Environmental feedback: obtain actual temperature, pressure and stress sensor data, and update the environmental status (the adjustment of control parameters affects the changes in physical parameters of the glass tempering process);

[0107] d. Calculate the reward function R(s t , a t ): According to the new physical parameters obtained in steps ac, calculate the temperature difference ΔT between the glass surface and the center t , stress deformation ΔS t and cooling non-uniformity U t ;

[0108] e. State transfer: update state s t to t+1 , prepare the decision for the next time step;

[0109] f.Store experience: store the experience tuple (s t , a t ,R(s t , a t ), s t+1 ) is stored in the experience replay buffer;

[0110] g. Based on the decision made by the strategy network after feedback iteration, the control parameters are continuously optimized and updated until the temporal convolutional network model converges to obtain the optimal control parameters to improve the product quality of tempered glass.

[0111] Compared with the prior art, the intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning of the present invention has the following significant advantages:

[0112] 1. Accurate parameter control

[0113] The present invention uses a method combining graph neural network (GCN) and temporal convolutional network (TCN) to accurately extract the spatial and temporal characteristics of key parameters such as temperature, pressure, and stress during the production process. This high-precision parameter prediction and analysis capability significantly improves the adjustment accuracy of control parameters (such as argon gas flow rate, crucible speed, etc.), reduces the temperature difference between the glass surface and the center, reduces stress deformation, and thus improves the overall quality of tempered glass.

[0114] 2. Improved performance and response speed

[0115] By introducing the reinforcement learning algorithm, the cooling system can continuously interact with the production environment, learn and adjust the control strategy in real time. This enables the control system of the present invention to quickly respond to abnormal situations in the production process, realizes real-time optimization of the cooling process, and greatly improves production efficiency and product consistency.

[0116] 3. Optimize energy efficiency

[0117] The present invention can effectively improve energy utilization efficiency and reduce unnecessary energy consumption by optimizing the adjustment strategy of control parameters, which not only reduces production costs and enhances economic benefits, but also promotes environmentally friendly production and meets the requirements of sustainable development.

[0118] 4. Improved intelligence and automation

[0119] The present invention combines graph neural networks with reinforcement learning to achieve comprehensive automation and intelligent control of the production process, reducing the reliance on operator experience, reducing the uncertainty caused by human intervention, improving the repeatability and consistency of the production process, and ensuring the stability of product quality.

[0120] 5. Efficiently handle complex data relationships

[0121] By processing the spatial correlation of sensor data with graph neural networks and analyzing the time series characteristics with temporal convolutional networks, the present invention can fully understand and utilize the complex relationships between parameters such as temperature, pressure, and stress. This capability significantly improves the control system's ability to optimize the production process and overcomes the shortcomings of existing technologies in processing complex data relationships.

[0122] 6. Simplify data processing flow and improve model application efficiency

[0123] The present invention ensures the quality and reliability of the data input into the model through a systematic data collection, cleaning and classification processing method, and simplifies the data processing process. This not only reduces the complexity of data processing, but also improves the efficiency of model training and real-time application, ensuring the efficient operation of the control system.

[0124] 7. Reduce production costs and improve economic benefits

[0125] The present invention can optimize control parameters and improve energy efficiency, effectively reducing production costs. At the same time, high-quality tempered glass products enhance market competitiveness and improve the economic benefits of enterprises.

[0126] 8. Improve the stability and reliability of the model

[0127] The present invention improves the adaptability and robustness of the model in different production environments by combining graph neural networks with reinforcement learning, ensures the stable operation of the control system under various production conditions, and improves the reliability of the overall system.

[0128] In summary, the present invention realizes high-precision, real-time and intelligent control of the ultra-thin glass tempering process by introducing advanced graph neural networks and reinforcement learning technologies, overcomes the shortcomings of the existing technologies in control accuracy, real-time performance, energy utilization efficiency and data processing capabilities, significantly improves product quality and production efficiency, and has significant industrial application value and market competitiveness.

[0129] In addition, it should be noted that the present invention may be a method, a system, an apparatus and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0130] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0131] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0132] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, Python, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0133] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0134] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0135] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0136] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0137] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. An intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning, characterized in that: The following steps are involved: S1. Data collection and cleaning: various parameter data in the glass tempering process are collected through temperature sensors, pressure sensors and stress sensors installed on the production line, and the collected parameter data are cleaned, missing values, abnormal values ​​and repeated values ​​are processed to obtain cleaned parameter data. The collected parameter data include temperature, pressure and stress; S2. Data processing: The cleaned parameter data is divided into three levels: low, medium, and high according to the importance of the data dimension, and corresponding level labels are added to the cleaned parameter data. The cleaned parameter data is divided into a training set, a validation set, and a test set according to a preset ratio; S3. Build a graph neural network: Select a graph convolutional neural network as the architecture of the graph neural network, and input the collected parameter data as node features into the graph neural network for spatial feature capture to output high-dimensional features of the nodes, where each node represents the location of the sensor on the production line, and the edge represents the physical relationship between different nodes; S4. Constructing a time series convolutional network model: Based on the high-dimensional features of the output nodes, the constructed time series convolutional network model is used to capture the time series characteristics of various parameter data, and the temperature, pressure and stress in the glass tempering process are predicted in time to generate prediction results. The time series convolutional network model is trained through the training set, the hyperparameters of the time series convolutional network model are adjusted through the validation set, and the prediction effect of the time series convolutional network model is evaluated through the test set. S5. Reinforcement learning optimization control: Based on the prediction results of the time series convolutional network model, the reinforcement learning algorithm is used to optimize the control parameters of the cooling system in the glass tempering process and design a reward function to reduce the temperature difference between the glass surface and the center, reduce stress deformation, and improve cooling uniformity, thereby improving the quality of tempered glass. The state of the cooling system is the future changes in temperature, pressure, and stress of each node predicted by the time series convolutional network model, and the action of the cooling system is to adjust the control parameters in the cooling process.

2. According to claim 1, the intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning is characterized in that: Processing missing values, outliers and duplicate values ​​in step S1 includes supplementing missing values ​​through interpolation or context information, eliminating outliers through statistical analysis, and processing duplicate values ​​through comparison and deduplication.

3. The intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning according to claim 1 is characterized in that: In step S2, the cleaned parameter data is divided into a training set, a validation set and a test set according to a preset ratio of 70%:15%:15%.

4. The intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning according to claim 1 is characterized in that: The specific operation steps of selecting the graph convolutional neural network as the architecture of the graph neural network in step S3 are as follows: Define each sensor location as a node v i , the feature vector of each node is: x i =[T i ,P i ,S i ] Among them, T i Indicates the temperature sensor reading, P i Indicates the pressure sensor reading, S i Indicates the stress sensor reading; Construct an adjacency matrix A, where A ij =1 indicates that there is a connection between node i and node j, otherwise it is 0. In the graph convolutional neural network, the node features are updated by information interaction with neighboring nodes. The node features of the lth layer are represented by H (l) , the update rule is: in, Add self-loops to include the node’s own information, I is the unit matrix, yes The degree matrix of H (l) is the node feature matrix of the lth layer, w (l) is the learnable weight matrix, σ is the activation function; Use the Adam optimizer to update the model parameters of the graph convolutional neural network.

5. The intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning according to claim 1 is characterized in that: The step S4 specifically includes the following steps: The high-dimensional features of the nodes output by the graph convolutional neural network are used as the input of the time series convolutional network model to form a time series dataset: X TCN =[H (L) (t-n+1),H (L) (t-n+2),...,H (L) (t)] Among them, n is the length of the time window, t is the current time, H (L) (t) represents the node features at time t; The temporal convolutional network model consists of multiple stacked causal convolutional layers. The convolution kernel size of each layer is k, and the expansion factor is d. The formula is: Among them, O (l) is the output of layer l, Denotes the expansion factor as d l The convolution operation uses the mean square error to measure the difference between the predicted value and the true value: in, is the predicted output of the temporal convolutional network model, Y i is the true value, N is the number of samples; Use the Adam optimizer to update the model parameters of the temporal convolutional network model.

6. The intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning according to claim 1 is characterized in that: The state of the cooling system in step S5 is composed of the temperature, pressure, and stress changes of each node within a specified period of time in the future predicted by the temporal convolutional network model. The state of the cooling system is expressed as: Here, k is the time step of prediction.

7. The intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning according to claim 6 is characterized in that: The action of the cooling system in step S5 is to adjust the control parameters in the cooling process, the control parameters include the argon flow rate F and the crucible speed R, and the action of the cooling system is expressed as: a t =[ΔF t ,ΔR t ] Where, ΔF t is the adjustment amount of argon gas flow rate, ΔR t is the adjustment amount of the crucible speed.

8. The intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning according to claim 7 is characterized in that: The reward function formula in step S5 is: R(s t ,a t )=-(α1·ΔT t +α1·ΔS t +α3·U t ) Among them, α1, α2, α3 are weight coefficients, ΔT t Indicates the temperature difference between the glass surface and the center, ΔS t Indicates stress deformation, U t Indicates uneven cooling.

9. The intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning according to claim 8, characterized in that: The reinforcement learning algorithm used in step S5 is specifically a deep deterministic policy gradient algorithm.

10. The intelligent ultra-thin glass tempering control method based on graph neural network and reinforcement learning according to claim 9, characterized in that: The specific cycle process of the deep deterministic policy gradient algorithm is as follows: For each state s in time step t , perform the following steps: a. Get the current status s t :Using the temporal convolutional network model to predict future changes in physical parameters Combined with the current control parameter F t and R t , forming state s t ; b. Select and execute action a t :Use the policy network to generate action a t =μ(s t |θ μ ) adjusting the argon gas flow rate and the crucible speed; c. Environmental feedback: obtain actual temperature, pressure and stress sensor data and update environmental status; d. Calculate the reward function R(s t , a t ): According to the new physical parameters obtained in steps ac, calculate the temperature difference ΔT between the glass surface and the center t , stress deformation ΔS t and cooling non-uniformity U t ; e. State transfer: update state s t to t+1 , prepare the decision for the next time step; f.Store experience: store the experience tuple (s t , a t ,R(s t , a t ), s t+1 ) is stored in the experience replay buffer; g. Based on the decision made by the policy network after feedback iteration, the control parameters are continuously optimized and updated until the temporal convolutional network model converges to obtain the optimal control parameters.

Images