A Water Purifier Control Method Based on Deep Hybrid Q-Network

Abstract

This invention discloses a water purifier control method based on a deep hybrid Q-network, comprising the following steps: collecting and preprocessing the water purifier's operating data; establishing a deep hybrid Q-network model composed of LSTM, GCN, and DQN, and training it using the preprocessed water purifier operating data; inputting the water purifier's real-time operating data into the trained LSTM for feature extraction, outputting a temporal feature vector, and inputting the water purifier's real-time operating data and spatial information from the water purifier's built-in sensors into the trained GCN for feature extraction, outputting a spatial feature vector; inputting the two feature vectors into the trained DQN for processing, outputting a state-action feature matrix, and generating a control strategy for the water purifier based on this matrix. This invention addresses the problem that while existing AI-based water purifier control methods demonstrate high adaptability in handling the static and dynamic characteristics of water quality, they may not fully consider the impact of spatial dimensions on water quality.

Description

Technical Field

[0001] This invention relates to the field of water purifier technology, specifically a water purifier control method based on a deep mixing Q-network. Background Technology

[0002] In the field of water treatment technology, artificial intelligence-based water purifier control methods have made some progress. This method first establishes an initial water purifier control AI model composed of a first water treatment network and a second water treatment network. Second, it obtains static state vectors for each state point of the water purifier through the first water treatment network and changing state vectors through the second water treatment network, enabling a comprehensive and in-depth understanding of the changing trends and characteristics of water quality. Third, it updates the first and second water treatment networks based on the static and changing state vectors, generating the final target water purifier control AI model. This model not only accurately describes the static characteristics of water quality but also effectively captures and predicts the dynamic changes of water quality parameters over time. Finally, it generates corresponding water purifier control strategies using the static and changing state vectors output by the target water purifier control AI model, achieving precise control of the water purifier. However, although this method demonstrates high adaptability in handling the static and dynamic characteristics of water quality, it may not fully consider the impact of spatial dimensions on water quality, thus affecting the comprehensiveness and accuracy of water quality control. Summary of the Invention

[0003] To address the aforementioned shortcomings, this invention proposes a water purifier control method based on a deep hybrid Q-network. The aim is to solve the problem that while existing AI-based water purifier control methods demonstrate high adaptability in handling the static and dynamic characteristics of water quality, they may not fully consider the impact of spatial dimensions on water quality, thus affecting the comprehensiveness and accuracy of water quality control.

[0004] To achieve this objective, the present invention adopts the following technical solution:

[0005] A water purifier control method based on a deep mixing Q-network includes the following steps:

[0006] Step S1: Collect the operating data of the water purifier, including inlet water flow rate, outlet water flow rate, dissolved solids (TDS) content in the water, water temperature, turbidity, pH value, and inlet water pressure;

[0007] Step S2: Preprocess the operating data of the water purifier to obtain the preprocessed operating data of the water purifier;

[0008] Step S3: Establish a deep hybrid Q-network model, which includes a Long Short-Term Memory (LSTM) network, a Graph Convolutional Network (GCN) network, and a Deep Q-Network (DQN) network.

[0009] Step S4: Use the pre-processed water purifier's operating data to train the deep hybrid Q-network model, and obtain the trained deep hybrid Q-network model;

[0010] Step S5: Collect real-time operating data of the water purifier and spatial information from the built-in sensors of the water purifier;

[0011] Step S6: Input the real-time operation data of the water purifier into the LSTM in the trained deep hybrid Q network model for feature extraction and output the temporal feature vector; and input the real-time operation data of the water purifier and the spatial information of the built-in sensor of the water purifier into the GCN in the trained deep hybrid Q network model for feature extraction and output the spatial feature vector.

[0012] Step S7: Input the temporal feature vector and spatial feature vector into the DQN in the trained deep hybrid Q-network model for processing, and output the state-action feature matrix;

[0013] Step S8: Generate the control strategy for the water purifier based on the state-action feature matrix.

[0014] Preferably, step S2 specifically includes the following sub-steps:

[0015] Operating data of the water purifier {x1, x2, ..., x i The data of the water purifier, x, ..., x7, is normalized to obtain the normalized operating data x of the water purifier. norm Where i = 1,...,7, the specific calculation formula is as follows:

[0016]

[0017] Wherein, min(x) i ) represents the minimum value of the water purifier's operating data; max(x) i This indicates the maximum value of the water purifier's operating data;

[0018] And using bidirectional interpolation to fill in the operating data of the water purifier x i The missing values ​​in the data are identified, and combined with time step t and spatial information s, to obtain the pre-treated water purifier's operating data {x1', x2', ..., x...}. i ',...,x7',t,s}.

[0019] Preferably, in step S3, the LSTM includes an input gate l t Forgotten Gate tOutput gate o t and candidate memory units Among them, input gate l t Used to determine time series data a t The contribution to the hidden state can be expressed mathematically as follows:

[0020] l t =σ(W l a t +U l h l-1 +b l );

[0021] Forgotten Gate f t The mathematical expression used to determine how much hidden state information from previous times should be retained at the current time step is as follows:

[0022] f t =σ(W f a t +U f h t-1 +b f );

[0023] Output gate o t The mathematical expression used to determine the impact of the hidden state on the final output is as follows:

[0024] o t =σ(W O a t +U O h t-1 +b0);

[0025] Candidate memory units Used to combine time series data a t Update the memory cell state c with the hidden state of the previous time step. t The specific mathematical expression is as follows:

[0026]

[0027] Memory cell state c t The updated mathematical expression is as follows:

[0028]

[0029] Among them, a t This represents the input data at time step t, i.e., time series data, h. t-1 Let c represent the hidden state at time step t-1. t-1 W represents the state of the memory cell at time step t-1. l W represents the weight matrix from the input data to the input gate.f U represents the weight matrix from the input data to the forget gate. O W represents the weight matrix from the input data to the output gate. c U represents the weight matrix from the input data to the candidate memory units. l U represents the weight matrix from the hidden state to the input gate. f U represents the weight matrix from the hidden state to the forget gate. O U represents the weight matrix from the hidden state to the output gate. c b represents the weight matrix from the hidden state to the candidate memory unit. l b f b O and b c All of these represent bias terms, σ(x) represents the Sigmoid activation function, tanh(x) represents the hyperbolic tangent function, and ⊙ represents the sign of element-wise multiplication;

[0030] In step S6, the real-time operating data of the water purifier is input into the LSTM in the trained deep hybrid Q network model for feature extraction, and the time feature vector is output. This includes the following sub-steps:

[0031] Step S61: Organize the real-time operating data of the water purifier into time-series data a t Where t = 1, ..., T, and T represents the length of the time series;

[0032] Step S62: Place a t The input is processed by an LSTM to obtain the hidden state h at each time step. t , where h t The mathematical formula is as follows:

[0033] h t =o t ⊙tanh(c t );

[0034] Step S63: Retrieve the hidden state h of the last time step T As a time feature vector.

[0035] Preferably, in step S3, the GCN includes N graph convolutional layers;

[0036] In step S6, the real-time operating data of the water purifier and the spatial information from the built-in sensors of the water purifier are input into the GCN of the trained deep hybrid Q-network model for feature extraction, and the spatial feature vector is output. This specifically includes the following sub-steps:

[0037] Step S64: Define the nodes and edges in the spatial topology graph, where each node corresponds to a sensor;

[0038] Step S65: Construct a spatial topology graph based on the nodes and edges in the spatial topology graph, and calculate the adjacency matrix A corresponding to the spatial topology graph;

[0039] Step S66: Normalize the adjacency matrix A corresponding to the spatial topology graph to obtain the normalized adjacency matrix. Among them, the adjacency matrix The calculation formula is as follows:

[0040]

[0041] Where D represents the degree matrix;

[0042] Step S67: Combine the real-time operating data of the water purifier with the normalized adjacency matrix. Input nodes in the spatial topology graph and perform graph convolution operations on each graph convolutional layer to update the features of the corresponding nodes;

[0043] Step S68: Extract the feature matrix H from the output of the last convolutional layer. (N) As a spatial feature vector;

[0044] Among them, the feature matrix H output by the last graph convolutional layer (N) The specific calculation formula is as follows:

[0045]

[0046] Among them, H (N-1) B represents the feature matrix output by the (N-1)th graph convolutional layer; (N-1) Let represent the trainable weight matrix of the (N-1)th graph convolutional layer.

[0047] Preferably, in step S7, the specific calculation formula for the state-action feature matrix is ​​as follows:

[0048]

[0049] Where Q(p, α) represents the state-action feature matrix; p represents the current state, including the operating state of the water purifier; α represents the action, including the control operation of the water purifier; r represents the reward function, used to evaluate the effect of the action; γ represents the discount factor, used to weigh short-term gains against long-term benefits; This represents the maximum action value in the next state p'.

[0050] Preferably, step S8 specifically includes the following sub-step: selecting the action α with the maximum value based on the state-action feature matrix Q(p, α) and the current state p. * Among them, the action with the greatest specific value α * The calculation formula is as follows:

[0051]

[0052] Preferably, the method further includes the following steps: calculating the operating indicators of the water purifier and evaluating them to obtain evaluation results, wherein the operating indicators of the water purifier include water quality improvement rate, filter life extension rate and unit energy consumption reduction rate; and adjusting the parameters of the deep mixing Q-network model in real time based on the evaluation results.

[0053] The technical solutions provided in this application embodiment may include the following beneficial effects:

[0054] This solution establishes a deep hybrid Q-network model composed of a Long Short-Term Memory (LSTM) network, a Graph Convolutional Network (GCN), and a Deep Q-Network (DQN). Real-time operating data from the water purifier and spatial information from its built-in sensors are collected and processed by the deep hybrid Q-network model. The model outputs a state-action feature matrix, which is used to generate a control strategy for the water purifier, thus achieving control. Compared to water purifier control methods based on AI models, this solution's deep hybrid Q-network model, by fusing LSTM and GCN, not only captures the trend of water quality changes over time but also fully utilizes the spatial information from sensors to capture the spatial patterns of water quality changes, thereby improving the comprehensiveness and accuracy of water quality control. Attached Figure Description

[0055] Figure 1 This is a flowchart of a water purifier control method based on a deep hybrid Q-network. Detailed Implementation

[0056] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0057] A water purifier control method based on a deep mixing Q-network includes the following steps:

[0058] Step S1: Collect the operating data of the water purifier, including inlet water flow rate, outlet water flow rate, dissolved solids (TDS) content in the water, water temperature, turbidity, pH value, and inlet water pressure;

[0059] Step S2: Preprocess the operating data of the water purifier to obtain the preprocessed operating data of the water purifier;

[0060] Step S3: Establish a deep hybrid Q-network model, which includes a Long Short-Term Memory (LSTM) network, a Graph Convolutional Network (GCN) network, and a Deep Q-Network (DQN) network.

[0061] Step S4: Use the pre-processed water purifier's operating data to train the deep hybrid Q-network model, and obtain the trained deep hybrid Q-network model;

[0062] Step S5: Collect real-time operating data of the water purifier and spatial information from the built-in sensors of the water purifier;

[0063] Step S6: Input the real-time operation data of the water purifier into the LSTM in the trained deep hybrid Q network model for feature extraction and output the temporal feature vector; and input the real-time operation data of the water purifier and the spatial information of the built-in sensor of the water purifier into the GCN in the trained deep hybrid Q network model for feature extraction and output the spatial feature vector.

[0064] Step S7: Input the temporal feature vector and spatial feature vector into the DQN in the trained deep hybrid Q-network model for processing, and output the state-action feature matrix;

[0065] Step S8: Generate the control strategy for the water purifier based on the state-action feature matrix.

[0066] This solution proposes a water purifier control method based on a deep-mixing Q-network, such as... Figure 1As shown, the first step is to collect the water purifier's operating data, including inlet flow rate, outlet flow rate, dissolved solids (TDS) content, water temperature, turbidity, pH value, and inlet pressure. In this embodiment, the water purifier has multiple built-in sensors that collect operating data, providing a data foundation for subsequent model training. The second step is to preprocess the water purifier's operating data to obtain preprocessed operating data. In this embodiment, preprocessing the operating data helps improve its accuracy and stability. The third step is to establish a deep hybrid Q-network model, which includes a Long Short-Term Memory (LSTM) network, a Graph Convolutional Network (GCN), and a Deep Q-Network (DQN). In this embodiment, establishing a deep hybrid Q-network model facilitates the subsequent extraction of temporal features, spatial features, and state-action features. It should be further noted that the LSTM, GCN, and DQN are all existing neural networks. The fourth step is to train the deep hybrid Q-network model using the pre-processed operating data of the water purifier, resulting in a trained deep hybrid Q-network model. In this embodiment, training the deep hybrid Q-network model using the pre-processed operating data of the water purifier helps improve the decision-making accuracy of the deep hybrid Q-network model. The fifth step is to collect the real-time operating data of the water purifier and the spatial information of the water purifier's built-in sensors. In this embodiment, collecting the real-time operating data of the water purifier and the spatial information of the water purifier's built-in sensors helps provide a data foundation for subsequent extraction of temporal and spatial features. The sixth step is to input the real-time operating data of the water purifier into the LSTM in the trained deep hybrid Q-network model for feature extraction, outputting a temporal feature vector; and input the real-time operating data of the water purifier and the spatial information of the water purifier's built-in sensors into the GCN in the trained deep hybrid Q-network model for feature extraction, outputting a spatial feature vector. In this embodiment, the temporal feature vector extracted by the LSTM represents the trend of water quality changes over time, providing a basis for predicting future water quality conditions. The spatial feature vector extracted by the GCN represents the spatial correlation between different sensors and the pattern of water quality changes with space. The seventh step involves inputting the temporal and spatial feature vectors into the DQN of the trained deep hybrid Q-network model for processing, outputting a state-action feature matrix. In this embodiment, processing the output state-action feature matrix through DQN facilitates the generation of subsequent control strategies. The eighth step is to generate a control strategy for the water purifier based on the state-action feature matrix. In this embodiment, the state-action feature matrix provides an evaluation of each possible action in the current state, helping the deep hybrid Q-network model select the optimal action for the current state.

[0067] This solution establishes a deep hybrid Q-network model composed of a Long Short-Term Memory (LSTM) network, a Graph Convolutional Network (GCN), and a Deep Q-Network (DQN). Real-time operating data from the water purifier and spatial information from its built-in sensors are collected and processed by the deep hybrid Q-network model. The model outputs a state-action feature matrix, which is used to generate a control strategy for the water purifier, thus achieving control. Compared to water purifier control methods based on AI models, this solution's deep hybrid Q-network model, by fusing LSTM and GCN, not only captures the trend of water quality changes over time but also fully utilizes the spatial information from sensors to capture the spatial patterns of water quality changes, thereby improving the comprehensiveness and accuracy of water quality control.

[0068] Preferably, step S2 specifically includes the following sub-steps:

[0069] Operating data of the water purifier {x1, x2, ..., x i The data of the water purifier, x, ..., x7, is normalized to obtain the normalized operating data x of the water purifier. norm Where i = 1,...,7, the specific calculation formula is as follows:

[0070]

[0071] Wherein, min(x) i ) represents the minimum value of the water purifier's operating data; max(x) i This indicates the maximum value of the water purifier's operating data;

[0072] And using bidirectional interpolation to fill in the operating data of the water purifier x i The missing values ​​in the data are combined with the time step t and spatial information s to obtain the pre-treated water purifier's operating data {x1', x2', ..., x...}. i ',…,x7',t,s).

[0073] In this embodiment, the operating data x of the water purifier is used. i Normalization helps ensure the consistency of data distribution. Two-way interpolation is used to fill in the operational data x of the water purifier. i The missing values ​​in the data, combined with the time dimension t and the spatial dimension s, are used to obtain the pre-treated water purifier's operating data, which helps to ensure the continuity and integrity of the data.

[0074] Preferably, in step S3, the LSTM includes an input gate l t Forgotten Gate t Output gate o t and candidate memory units Among them, input gate l t Used to determine time series data at The contribution to the hidden state can be expressed mathematically as follows:

[0075] l t =σ(W l a t +U l h t-1 +b l );

[0076] Forgotten Gate f t The mathematical expression used to determine how much hidden state information from previous times should be retained at the current time step is as follows:

[0077] f t =σ(W f a t +U f h t-1 +b f );

[0078] Output gate o t The mathematical expression used to determine the impact of the hidden state on the final output is as follows:

[0079] o t =σ(W O a t +U O h t-1 +b O );

[0080] Candidate memory units Used to combine time series data a t Update the memory cell state c with the hidden state of the previous time step. t The specific mathematical expression is as follows:

[0081]

[0082] Memory cell state c t The updated mathematical expression is as follows:

[0083]

[0084] Among them, a t This represents the input data at time step t, i.e., time series data, h. t-1 Let c represent the hidden state at time step t-1. t-1 W represents the state of the memory cell at time step t-1. l W represents the weight matrix from the input data to the input gate. f W represents the weight matrix from the input data to the forget gate. O W represents the weight matrix from the input data to the output gate.c U represents the weight matrix from the input data to the candidate memory units. l U represents the weight matrix from the hidden state to the input gate. f U represents the weight matrix from the hidden state to the forget gate. O U represents the weight matrix from the hidden state to the output gate. c b represents the weight matrix from the hidden state to the candidate memory unit. l b f b O and b c All of these represent bias terms, σ(x) represents the Sigmoid activation function, tanh(x) represents the hyperbolic tangent function, and ⊙ represents the sign of element-wise multiplication;

[0085] In step S6, the real-time operating data of the water purifier is input into the LSTM in the trained deep hybrid Q network model for feature extraction, and the time feature vector is output. This includes the following sub-steps:

[0086] Step S61: Organize the real-time operating data of the water purifier into time-series data a t Where t = 1, ..., T, and T represents the length of the time series;

[0087] Step S62: Place a t The input is processed by an LSTM to obtain the hidden state h at each time step. t , where h t The mathematical formula is as follows:

[0088] h t =o t ⊙tanh(c t );

[0089] Step S63: Retrieve the hidden state h of the last time step T As a time feature vector.

[0090] In this embodiment, after processing the entire time series, the LSTM obtains the hidden state at each time step and takes the hidden state of the last time step as the time feature vector. This time feature vector contains the dynamic changes of the water purifier's operating parameters throughout the time series, providing crucial information for subsequent water quality prediction and control strategy generation. In other embodiments, the average value of the hidden states at all time steps can also be taken as the time feature vector.

[0091] Preferably, in step S3, the GCN includes N graph convolutional layers;

[0092] In step S6, the real-time operating data of the water purifier and the spatial information from the built-in sensors of the water purifier are input into the GCN of the trained deep hybrid Q-network model for feature extraction, and the spatial feature vector is output. This specifically includes the following sub-steps:

[0093] Step S64: Define the nodes and edges in the spatial topology graph, where each node corresponds to a sensor;

[0094] Step S65: Construct a spatial topology graph based on the nodes and edges in the spatial topology graph, and calculate the adjacency matrix A corresponding to the spatial topology graph;

[0095] Step S66: Normalize the adjacency matrix A corresponding to the spatial topology graph to obtain the normalized adjacency matrix. Among them, the adjacency matrix The calculation formula is as follows:

[0096]

[0097] Where D represents the degree matrix;

[0098] Step S67: Combine the real-time operating data of the water purifier with the normalized adjacency matrix. Input nodes in the spatial topology graph and perform graph convolution operations on each graph convolutional layer to update the features of the corresponding nodes;

[0099] Step S68: Extract the feature matrix H from the output of the last convolutional layer. (N) As a spatial feature vector;

[0100] Among them, the feature matrix H output by the last graph convolutional layer (N) The specific calculation formula is as follows:

[0101]

[0102] Among them, H (N-1) B represents the feature matrix output by the (N-1)th graph convolutional layer; (N-1) Let represent the trainable weight matrix of the (N-1)th graph convolutional layer.

[0103] In this embodiment, the real-time operating data of the water purifier and the normalized adjacency matrix are used. The input is GCN, and the output is a spatial feature vector. This spatial feature vector characterizes the spatial correlation between sensors. For example, the strong correlation between the influent flow rate and the effluent flow rate sensors may reflect the pump speed stability. Therefore, the spatial feature vector provides the basis for the subsequent generation of the state-action feature matrix.

[0104] Preferably, in step S7, the specific calculation formula for the state-action feature matrix is ​​as follows:

[0105]

[0106] Where Q(p, α) represents the state-action feature matrix; p represents the current state, including the operating state of the water purifier; α represents the action, including the control operation of the water purifier; r represents the reward function, used to evaluate the effect of the action; γ represents the discount factor, used to weigh short-term gains against long-term benefits; This represents the maximum action value in the next state p'.

[0107] In this embodiment, DQN evaluates the benefit of each action through a reward function, that is, the contribution of each action to water quality improvement, and combines a discount factor to weigh short-term benefits against long-term benefits, and finally outputs the maximum action value in the next state.

[0108] Preferably, step S8 specifically includes the following sub-steps: selecting the action α with the maximum value based on the state-action feature matrix Q(p, α) and the current state p. * Among them, the action with the greatest specific value α * The calculation formula is as follows:

[0109]

[0110] In this embodiment, by calculating and selecting the optimal action under the current state, the control strategy is ensured to accurately adapt to changes in the dynamic environment. In one embodiment, if the current water pollution is severe, increasing the pump speed or adjusting the filtration parameters is prioritized.

[0111] Preferably, the method further includes the following steps: calculating and evaluating the operating indicators of the water purifier to obtain evaluation results, wherein the operating indicators of the water purifier include water quality improvement rate, filter life extension rate, and unit energy consumption reduction rate; and adjusting the parameters of the deep mixing Q-network model in real time based on the evaluation results. In this embodiment, by adjusting the parameters of the deep mixing Q-network model in real time, continuous optimization of the deep mixing Q-network model can be achieved, thereby ensuring the water purification quality and efficient operation of the water purifier in complex environments.

[0112] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0113] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A water purifier control method based on a deep mixed Q network, characterized by: The method comprises the following steps: Step S1: collecting operation data of the water purifier, wherein the operation data of the water purifier comprises water inflow, water outflow, total dissolved solids (TDS) value, water temperature, turbidity, PH value and water inflow pressure; Step S2: preprocessing the operation data of the water purifier to obtain preprocessed operation data of the water purifier; Step S3: establishing a deep hybrid Q network model, wherein the deep hybrid Q network model comprises a long short-term memory (LSTM) network, a graph convolutional network (GCN) and a deep Q network (DQN); Step S4: training the deep hybrid Q network model using the preprocessed operation data of the water purifier to obtain a trained deep hybrid Q network model; Step S5: collecting real-time operation data of the water purifier and spatial information of a built-in sensor of the water purifier; Step S6: inputting the real-time operation data of the water purifier into the LSTM in the trained deep hybrid Q network model for feature extraction to output a time feature vector, and inputting the real-time operation data of the water purifier and the spatial information of the built-in sensor of the water purifier into the GCN in the trained deep hybrid Q network model for feature extraction to output a space feature vector; Step S7: inputting the time feature vector and the space feature vector into the DQN in the trained deep hybrid Q network model for processing to output a state-action feature matrix; Step S8: generating a control strategy of the water purifier based on the state-action feature matrix; In step S3, the GCN comprises N layers of graph convolutional layers. In step S6, the real-time operation data of the water purifier and the spatial information of the built-in sensor of the water purifier are inputted into the GCN in the trained deep hybrid Q network model for feature extraction to output a space feature vector, which comprises the following substeps: Step S64: defining nodes and edges in a spatial topology graph, wherein each node corresponds to a sensor; Step S65: constructing a spatial topology graph according to the nodes and edges in the spatial topology graph, and calculating an adjacency matrix corresponding to the spatial topology graph ; Step S66: Normalizing the adjacency matrix corresponding to the spatial topology graph The normalized adjacency matrix is obtained by performing normalization processing The calculation formula of the adjacency matrix is as follows: ； wherein denotes a degree matrix; Step S67: input the real-time running data of the water purifier and the normalized adjacency matrix The nodes in the input space topology graph are input, and graph convolution operations are performed through the graph convolution layers one by one to update the features of the corresponding nodes. Step S68: taking the feature matrix output by the last graph convolution layer as a spatial feature vector; wherein the feature matrix output by the last graph convolution layer The specific calculation formula is as follows: ； wherein, represents a feature matrix output by the Nth graph convolutional layer; represents a trainable weight matrix of the Nth graph convolutional layer; The method further comprises the following steps: calculating an operation index of the water purifier and evaluating the operation index to obtain an evaluation result, wherein the operation index of the water purifier comprises a water quality improvement rate, a filter core life extension rate and a unit energy consumption reduction rate; and adjusting parameters of the deep hybrid Q network model in real time based on the evaluation result.

2. The water purifier control method based on deep mixed Q network according to claim 1, characterized in that: In step S2, the method comprises the following substeps: Operation data of the water purifier Operation data of the water purifier after normalization processing , wherein, , and the specific calculation formula is as follows: ; wherein, represents a minimum value of the operation data of the water purifier; represents a maximum value of the operation data of the water purifier; and filling in missing values in the operation data of the water purifier using a bidirectional interpolation method , combined with time step t and spatial information s, to obtain preprocessed operation data of the water purifier . 3.The water purifier control method based on deep mixed Q network according to claim 1, characterized in that: In step S3, the LSTM comprises an input gate , a forget gate , an output gate and a candidate memory cell , wherein the input gate is used to decide the contribution of the time series data to the hidden state, and the specific mathematical expression is as follows: ； forget gate For deciding how much information of the previous hidden state to keep for the current time step, the following mathematical expression is used: ； Output gate For determining the influence of the hidden state on the final output, the following specific mathematical expression is used: ; Candidate memory cell For incorporating timing data Updating the memory cell state with the hidden state of the previous time step The specific mathematical representation is as follows: ； Memory cell state The updated mathematical expression is as follows: ； wherein, denotes the input data at the t-th time step, i.e., the time series data, denotes the hidden state at the t-1-th time step, denotes the memory cell state at the t-1-th time step, denotes the weight matrix of the input data to the input gate, denotes the weight matrix of the input data to the forget gate, denotes the weight matrix of the input data to the output gate, denotes the weight matrix of the input data to the candidate memory cell, denotes the weight matrix of the hidden state to the input gate, denotes the weight matrix of the hidden state to the forget gate, denotes the weight matrix of the hidden state to the output gate, denotes the weight matrix of the hidden state to the candidate memory cell, , , and all denote the bias term, denotes the Sigmoid activation function, denotes the hyperbolic tangent function, denotes the sign of the element-wise multiplication. In step S6, the real-time operation data of the water purifier is inputted into the LSTM in the trained deep hybrid Q network model for feature extraction to output a time feature vector, which comprises the following substeps: Step S61: Organize the real-time running data of the water purifier into time series data wherein, T represents the length of the time series. Step S62: obtaining the hidden state of the input LSTM at each time step The input LSTM is processed to obtain the hidden state at each time step wherein, The mathematical formula is as follows: ； Step S63: Take the hidden state of the last time step as the time feature vector.

4. The water purifier control method based on deep mixed Q network according to claim 1, characterized in that: In step S7, the state-action feature matrix is calculated according to the following formula: ； wherein, represents a state-action feature matrix; represents a current state, including the operating state of the water purifier; represents an action, including the control operation of the water purifier; represents a reward function, for evaluating the effect of the action; represents a discount factor, for weighing short-term gains against long-term benefits; represents the maximum action value in the next state .

5. The water purifier control method based on deep mixed Q network according to claim 4, characterized in that: In step S8, the method comprises the following substeps: According to the state-action feature matrix and the current state , an action with a specific maximum value is selected , wherein the calculation formula of the action with the specific maximum value is as follows: ​ 。

Images