Water purifier control method based on deep mixed Q network

Abstract

The invention discloses a water purifier control method based on a deep mixed Q network. The method comprises the following steps: collecting operation data of a water purifier and preprocessing the operation data; establishing a deep mixed Q network model composed of LSTM, GCN and DQN, and training the deep mixed Q network model by using the preprocessed operation data of the water purifier; inputting real-time operation data of the water purifier into the trained LSTM for feature extraction, outputting a time feature vector, inputting the real-time operation data of the water purifier and spatial information of a built-in sensor of the water purifier into the trained GCN for feature extraction, and outputting a spatial feature vector; and inputting the two feature vectors into the trained DQN for processing, outputting a state-action feature matrix, and generating a control strategy of the water purifier according to the state-action feature matrix. The invention solves the problem that the existing water purifier control method based on artificial intelligence does not fully consider the influence of the spatial dimension on the water quality although the existing water purifier control method shows very high adaptability in the aspects of static and dynamic characteristics of the treated water quality state.

Description

Technical Field

[0001] The present invention relates to the technical field of water purifiers, and specifically to a water purifier control method based on a deep hybrid Q-network. Background Art

[0002] In the existing water treatment technology field, certain progress has been made in the water purifier control method based on artificial intelligence. In this method, first, an initial water purifier control AI model composed of a first water quality treatment network and a second water quality treatment network is determined. Secondly, static state vectors of each water purifier state point are obtained through the first water quality treatment network, and change state vectors are obtained through the second water quality treatment network, enabling a comprehensive and in-depth understanding of the changing trends and characteristics of the water quality state. Thirdly, the first water quality treatment network and the second water quality treatment network are updated based on the static state vectors and change state vectors to generate a final target water purifier control AI model. This model can not only accurately describe the static characteristics of the water quality state but also effectively capture and predict the dynamic changes of water quality parameters over time. Finally, corresponding water purifier control strategies are generated through the static state vectors and change state vectors output by the decision-making of the target water purifier control AI model, enabling precise control of the water purifier. However, although this method shows high adaptability in dealing with the static and dynamic characteristics of the water quality state, it may not fully consider the influence of the spatial dimension on the water quality, thus affecting the comprehensiveness and accuracy of water quality control. Summary of the Invention

[0003] Aiming at the above defects, the present invention proposes a water purifier control method based on a deep hybrid Q-network, aiming to solve the problem that although the existing water purifier control method based on artificial intelligence shows high adaptability in dealing with the static and dynamic characteristics of the water quality state, it may not fully consider the influence of the spatial dimension on the water quality, thus affecting the comprehensiveness and accuracy of water quality control.

[0004] To achieve this purpose, the present invention adopts the following technical solutions:

[0005] A water purifier control method based on a deep hybrid Q-network includes the following steps:

[0006] Step S1: Collect the operation data of the water purifier, where the operation data of the water purifier includes the influent flow rate, effluent flow rate, total dissolved solids (TDS) value in the water, water temperature, turbidity, pH value, and influent pressure;

[0007] Step S2: Preprocess the operation data of the water purifier to obtain the preprocessed operation data of the water purifier;

[0008] Step S3: Establish a deep hybrid Q-network model, where the deep hybrid Q-network model includes a long short-term memory network (LSTM), a graph convolutional network (GCN), and a deep Q-network (DQN);

[0009] Step S4: Use the operation data of the preprocessed water purifier to train the deep hybrid Q-network model to obtain a trained deep hybrid Q-network model;

[0010] Step S5: Collect the real-time operation data of the water purifier and the spatial information of the built-in sensors of the water purifier;

[0011] Step S6: Input the real-time operation data of the water purifier into the LSTM in the trained deep hybrid Q-network model for feature extraction, and output a time feature vector; and input the real-time operation data of the water purifier and the spatial information of the built-in sensors of the water purifier into the GCN in the trained deep hybrid Q-network model for feature extraction, and output a spatial feature vector;

[0012] Step S7: Input the time feature vector and the spatial feature vector into the DQN in the trained deep hybrid Q-network model for processing, and output a state-action feature matrix;

[0013] Step S8: Generate a control strategy for the water purifier according to the state-action feature matrix.

[0014] Preferably, in step S2, it specifically includes the following sub-steps:

[0015] Normalize the operation data {x1, x2,..., x i ,..., x7} of the water purifier to obtain the normalized operation data x norm of the water purifier, where i = 1,..., 7, and the specific calculation formula is as follows:

[0016]

[0017] where min(x i ) represents the minimum value of the operation data of the water purifier; max(x i ) represents the maximum value of the operation data of the water purifier;

[0018] And use the bilinear interpolation method to fill in the missing values in the operation data x i of the water purifier, and combine the time step t and the spatial information s to obtain the preprocessed operation data {x1', x2',..., x i ',..., x7', t, s} of the water purifier.

[0019] Preferably, in step S3, the LSTM includes an input gate l t and a forget gate f t, output gate o t and candidate memory unit Among them, input gate l t is used to determine the contribution of sequential data a t to the hidden state. The specific mathematical expression is as follows:

[0020] l t = σ(W l a t + U l h l-1 + b l );

[0021] Forget gate f t is used to determine how much previous hidden state information to retain at the current time step. The specific mathematical expression is as follows:

[0022] f t = σ(W f a t + U f h t-1 + b f );

[0023] Output gate o t is used to determine the influence of the hidden state on the final output. The specific mathematical expression is as follows:

[0024] o t = σ(W O a t + U O h t-1 + b0);

[0025] Candidate memory unit is used to combine sequential data a t with the hidden state of the previous time step to update the memory unit state c t , and the specific mathematical expression is as follows:

[0026]

[0027] Memory unit state c t The mathematical expression for the update is as follows:

[0028]

[0029] Among them, a t represents the input data at the t-th time step, i.e., sequential data, h t-1 represents the hidden state at the (t - 1)-th time step, c t-1 represents the memory unit state at the (t - 1)-th time step, W l represents the weight matrix from the input data to the input gate, Wf The weight matrix representing the input data to the forget gate, U O The weight matrix representing the input data to the output gate, W c The weight matrix representing the input data to the candidate memory cell, U l The weight matrix representing the hidden state to the input gate, U f The weight matrix representing the hidden state to the forget gate, U O The weight matrix representing the hidden state to the output gate, U c The weight matrix representing the hidden state to the candidate memory cell, b l , b f , b O and b c All represent bias terms, σ(x) represents the Sigmoid activation function, tanh(x) represents the hyperbolic tangent function, and ⊙ represents the symbol for element-wise multiplication;

[0030] In step S6, the real-time operation data of the water purifier is input into the LSTM in the trained deep hybrid Q-network model for feature extraction, and a time feature vector is output, which specifically includes the following sub-steps:

[0031] Step S61: Organize the real-time operation data of the water purifier into time-series data a t , where t = 1,..., T, and T represents the length of the time series;

[0032] Step S62: Input a t into the LSTM for processing to obtain the hidden state h at each time step t , where the mathematical formula of h t is as follows:

[0033] h t = o t ⊙tanh(c t );

[0034] Step S63: Take the hidden state h at the last time step T as the time feature vector.

[0035] Preferably, in step S3, the GCN includes N graph convolutional layers;

[0036] In step S6, the real-time operation data of the water purifier and the spatial information of the built-in sensors of the water purifier are input into the GCN in the trained deep hybrid Q-network model for feature extraction, and a spatial feature vector is output, which specifically includes the following sub-steps:

[0037] Step S64: Define the nodes and edges in the spatial topology graph, where each node corresponds to a sensor;

[0038] Step S65: Construct a spatial topology graph based on the nodes and edges in the spatial topology graph, and calculate the adjacency matrix A corresponding to the spatial topology graph;

[0039] Step S66: Normalize the adjacency matrix A corresponding to the spatial topology graph to obtain the normalized adjacency matrix where the adjacency matrix has the following calculation formula:

[0040]

[0041] where D represents the degree matrix;

[0042] Step S67: Input the real-time operation data of the water purifier and the normalized adjacency matrix into the nodes in the spatial topology graph, and perform graph convolution operations through individual graph convolution layers to update the features of the corresponding nodes;

[0043] Step S68: Take the feature matrix H output by the last graph convolution layer (N) as the spatial feature vector;

[0044] where the feature matrix H output by the last graph convolution layer (N) has the following specific calculation formula:

[0045]

[0046] where H (N-1) represents the feature matrix output by the (N - 1)-th graph convolution layer; B (N-1) represents the trainable weight matrix of the (N - 1)-th graph convolution layer.

[0047] Preferably, in step S7, the specific calculation formula of the state-action feature matrix is as follows:

[0048]

[0049] where Q(p, α) represents the state-action feature matrix; p represents the current state, including the operation state of the water purifier; α represents the action, including the control operation of the water purifier; r represents the reward function for evaluating the action effect; γ represents the discount factor for weighing short-term benefits and long-term interests; represents the maximum action value in the next state p'.

[0050] Preferably, in step S8, it specifically includes the following sub-steps: Select the action α with the specific maximum value according to the state-action feature matrix Q(p, α) and the current state p * , where the action α with the specific maximum value * has the following calculation formula:

[0051]

[0052] Preferably, the method further includes the following steps: calculating the operation indexes of the water purifier, evaluating the same, and obtaining an evaluation result, wherein the operation indexes of the water purifier include a water quality improvement rate, a filter element life extension rate, and a unit energy consumption reduction rate; and adjusting the parameters of the deep hybrid Q-network model in real time based on the evaluation result.

[0053] The technical solution provided by the embodiment of the present application may include the following beneficial effects:

[0054] In this solution, a deep hybrid Q-network model composed of a long short-term memory network (LSTM), a graph convolutional network (GCN), and a deep Q-network (DQN) is established, real-time operation data of the water purifier and spatial information of sensors built in the water purifier are collected, and these data are input into the deep hybrid Q-network model for processing to output a state-action feature matrix, and a control strategy of the water purifier is generated accordingly, so as to realize the control of the water purifier. Compared with the water purifier control method based on the water purifier control AI model, in this solution, the deep hybrid Q-network model can not only capture the trend of water quality change over time by fusing LSTM and GCN, but also make full use of the spatial information of the sensors to capture the law of water quality change over space, thereby improving the comprehensiveness and accuracy of water quality control. Description of the Drawings

[0055] Figure 1 is a flowchart of the steps of a water purifier control method based on a deep hybrid Q-network. Detailed Embodiments

[0056] The following describes in detail the embodiments of the present invention. The examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

[0057] A water purifier control method based on a deep hybrid Q-network includes the following steps:

[0058] Step S1: Collecting the operation data of the water purifier, wherein the operation data of the water purifier includes the inlet flow rate, the outlet flow rate, the total dissolved solids (TDS) value in the water, the water temperature, the turbidity, the pH value, and the inlet pressure;

[0059] Step S2: Preprocessing the operation data of the water purifier to obtain the preprocessed operation data of the water purifier;

[0060] Step S3: Establish a deep hybrid Q-network model, where the deep hybrid Q-network model includes a long short-term memory network (LSTM), a graph convolutional network (GCN), and a deep Q-network (DQN);

[0061] Step S4: Use the operation data of the preprocessed water purifier to train the deep hybrid Q-network model to obtain a trained deep hybrid Q-network model;

[0062] Step S5: Collect the real-time operation data of the water purifier and the spatial information of the built-in sensors of the water purifier;

[0063] Step S6: Input the real-time operation data of the water purifier into the LSTM in the trained deep hybrid Q-network model for feature extraction, and output a time feature vector; and input the real-time operation data of the water purifier and the spatial information of the built-in sensors of the water purifier into the GCN in the trained deep hybrid Q-network model for feature extraction, and output a spatial feature vector;

[0064] Step S7: Input the time feature vector and the spatial feature vector into the DQN in the trained deep hybrid Q-network model for processing, and output a state-action feature matrix;

[0065] Step S8: Generate a control strategy for the water purifier according to the state-action feature matrix.

[0066] A water purifier control method based on a deep hybrid Q-network in this solution, as Figure 1As shown in the figure, the first step is to collect the operation data of the water purifier. Among them, the operation data of the water purifier includes the inlet water flow rate, the outlet water flow rate, the TDS value of the dissolved solids in the water, the water temperature, the turbidity, the pH value, and the inlet water pressure. In this embodiment, multiple sensors are built into the water purifier, and these sensors can collect the operation data of the water purifier, providing a data basis for subsequent model training. The second step is to preprocess the operation data of the water purifier to obtain the preprocessed operation data of the water purifier. In this embodiment, preprocessing the operation data of the water purifier is beneficial to improving the accuracy and stability of the operation data. The third step is to establish a deep hybrid Q-network model. Among them, the deep hybrid Q-network model includes a long short-term memory network (LSTM), a graph convolutional network (GCN), and a deep Q-network (DQN). In this embodiment, establishing a deep hybrid Q-network model is beneficial to subsequent extraction of time features, spatial features, and state-action features. Further explanation, the long short-term memory network (LSTM), the graph convolutional network (GCN), and the deep Q-network (DQN) are all existing neural networks. The fourth step is to use the preprocessed operation data of the water purifier to train the deep hybrid Q-network model to obtain the trained deep hybrid Q-network model. In this embodiment, training the deep hybrid Q-network model with the preprocessed operation data of the water purifier is beneficial to improving the decision-making accuracy of the deep hybrid Q-network model. The fifth step is to collect the real-time operation data of the water purifier and the spatial information of the sensors built into the water purifier. In this embodiment, collecting the real-time operation data of the water purifier and the spatial information of the sensors built into the water purifier is beneficial to providing a data basis for subsequent extraction of time features and spatial features. The sixth step is to input the real-time operation data of the water purifier into the LSTM in the trained deep hybrid Q-network model for feature extraction, and output a time feature vector; and input the real-time operation data of the water purifier and the spatial information of the sensors built into the water purifier into the GCN in the trained deep hybrid Q-network model for feature extraction, and output a spatial feature vector. In this embodiment, the time feature vector extracted by the LSTM characterizes the trend of water quality change over time, providing a basis for predicting the future water quality state. The spatial feature vector extracted by the GCN characterizes the spatial correlation between different sensors and the law of water quality change over space. The seventh step is to input the time feature vector and the spatial feature vector into the DQN in the trained deep hybrid Q-network model for processing, and output a state-action feature matrix. In this embodiment, processing and outputting a state-action feature matrix by the DQN is beneficial to the generation of subsequent control strategies. The eighth step is to generate a control strategy for the water purifier according to the state-action feature matrix. In this embodiment, the state-action feature matrix provides an evaluation of each possible action in the current state, helping the deep hybrid Q-network model select the optimal action in the current state.

[0067] In this solution, a deep hybrid Q-network model composed of a long short-term memory network (LSTM), a graph convolutional network (GCN), and a deep Q-network (DQN) is established. The real-time operation data of the water purifier and the spatial information of the built-in sensors of the water purifier are collected, and these data are input into the deep hybrid Q-network model for processing to output a state-action feature matrix, and a control strategy for the water purifier is generated accordingly, thereby realizing the control of the water purifier. Compared with the water purifier control method based on the AI model for water purifier control, in this solution, the deep hybrid Q-network model can not only capture the trend of water quality change over time by fusing LSTM and GCN, but also make full use of the spatial information of the sensors to capture the law of water quality change over space, thereby improving the comprehensiveness and accuracy of water quality control.

[0068] Preferably, in step S2, it specifically includes the following sub-steps:

[0069] Normalize the operation data {x1, x2,..., x i ,..., x7} of the water purifier to obtain the normalized operation data x norm of the water purifier, where i = 1,..., 7, and the specific calculation formula is as follows:

[0070]

[0071] Among them, min(x i ) represents the minimum value of the operation data of the water purifier; max(x i ) represents the maximum value of the operation data of the water purifier;

[0072] And use the bidirectional interpolation method to fill in the missing values in the operation data x i of the water purifier, and combine the time step t and the spatial information s to obtain the preprocessed operation data {x1', x2',..., x i ',..., x7', t, s) of the water purifier.

[0073] In this embodiment, normalizing the operation data x i of the water purifier is beneficial to ensuring the consistency of data distribution. By using the bidirectional interpolation method to fill in the missing values in the operation data x i of the water purifier and combining the time dimension t and the spatial dimension s to obtain the preprocessed operation data of the water purifier is beneficial to ensuring the continuity and integrity of the data.

[0074] Preferably, in step S3, the LSTM includes an input gate l t , a forget gate f t , an output gate o t , and a candidate memory unit Among them, the input gate l t is used to determine the time series data at Contribution to the hidden state, and the specific mathematical expression is as follows:

[0075] l t = σ(W l a t + U l h t-1 + b l );

[0076] Forget gate f t Used to determine how much of the previous hidden state information should be retained at the current time step, and the specific mathematical expression is as follows:

[0077] f t = σ(W f a t + U f h t-1 + b f );

[0078] Output gate o t Used to determine the influence of the hidden state on the final output, and the specific mathematical expression is as follows:

[0079] o t = σ(W O a t + U O h t-1 + b O );

[0080] Candidate memory cell Used to combine the sequential data a t with the hidden state of the previous time step to update the memory cell state c t , and the specific mathematical expression is as follows:

[0081]

[0082] Memory cell state c t The mathematical expression for the update is as follows:

[0083]

[0084] Among them, a t represents the input data at the t-th time step, i.e., sequential data, h t-1 represents the hidden state at the (t - 1)-th time step, c t-1 represents the memory cell state at the (t - 1)-th time step, W l represents the weight matrix from the input data to the input gate, W f represents the weight matrix from the input data to the forget gate, W O represents the weight matrix from the input data to the output gate, Wc The weight matrix representing the input data to the candidate memory cell, U l The weight matrix representing the hidden state to the input gate, U f The weight matrix representing the hidden state to the forget gate, U O The weight matrix representing the hidden state to the output gate, U c The weight matrix representing the hidden state to the candidate memory cell, b l and b f and b O and b c all represent bias terms, σ(x) represents the Sigmoid activation function, tanh(x) represents the hyperbolic tangent function, and ⊙ represents the symbol for element-wise multiplication;

[0085] In step S6, the real-time operation data of the water purifier is input into the LSTM in the trained deep hybrid Q-network model for feature extraction, and a time feature vector is output, which specifically includes the following sub-steps:

[0086] Step S61: Organize the real-time operation data of the water purifier into time-series data a t , where t = 1,..., T, and T represents the length of the time series;

[0087] Step S62: Input a t into the LSTM for processing to obtain the hidden state h t at each time step, where the mathematical formula for h t is as follows:

[0088] h t = o t ⊙tanh(c t );

[0089] Step S63: Take the hidden state h T at the last time step as the time feature vector.

[0090] In this embodiment, after the LSTM processes the entire time series, it obtains the hidden state at each time step and takes the hidden state at the last time step as the time feature vector. This time feature vector contains the dynamic change information of the operation parameters of the water purifier over the entire time series and can provide a key basis for subsequent water quality prediction and control strategy generation. In other embodiments, the average value of the hidden states at all time steps can also be taken as the time feature vector.

[0091] Preferably, in step S3, the GCN includes N graph convolutional layers;

[0092] In step S6, the real-time operation data of the water purifier and the spatial information of the built-in sensors of the water purifier are input into the GCN in the trained deep hybrid Q-network model for feature extraction, and a spatial feature vector is output, which specifically includes the following sub-steps:

[0093] Step S64: Define the nodes and edges in the spatial topology graph, where each node corresponds to a sensor;

[0094] Step S65: According to the nodes and edges in the spatial topology graph, construct the spatial topology graph and calculate the adjacency matrix A corresponding to the spatial topology graph;

[0095] Step S66: Perform normalization processing on the adjacency matrix A corresponding to the spatial topology graph to obtain the normalized adjacency matrix where the adjacency matrix has the following calculation formula:

[0096]

[0097] where D represents the degree matrix;

[0098] Step S67: Input the real-time operation data of the water purifier and the normalized adjacency matrix into the nodes in the spatial topology graph, and perform graph convolution operations through each graph convolution layer to update the features of the corresponding nodes;

[0099] Step S68: Take the feature matrix H output by the last graph convolution layer (N) as the spatial feature vector;

[0100] where the feature matrix H output by the last graph convolution layer (N) has the following specific calculation formula:

[0101]

[0102] where H (N-1) represents the feature matrix output by the (N - 1)-th graph convolution layer; B (N-1) represents the trainable weight matrix of the (N - 1)-th graph convolution layer.

[0103] In this embodiment, by inputting the real-time operation data of the water purifier and the normalized adjacency matrix into the GCN and outputting a spatial feature vector, this spatial feature vector characterizes the spatial correlation between sensors. For example, the strong correlation between the inlet flow sensor and the outlet flow sensor may reflect the pump speed stability. Therefore, the spatial feature vector provides a basis for the generation of the subsequent state-action feature matrix.

[0104] Preferably, in step S7, the specific calculation formula of the state-action feature matrix is as follows:

[0105]

[0106] Among them, Q(p, α) represents the state-action feature matrix; p represents the current state, including the operating state of the water purifier; α represents the action, including the control operation of the water purifier; r represents the reward function, which is used to evaluate the action effect; γ represents the discount factor, which is used to balance short-term benefits and long-term interests; represents the maximum action value in the next state p'.

[0107] In this embodiment, DQN evaluates the benefits of each action through the reward function, that is, the contribution of each action to water quality improvement, and combines the discount factor to balance short-term and long-term benefits, and finally outputs the maximum action value in the next state.

[0108] Preferably, in step S8, it specifically includes the following sub-steps: according to the state-action feature matrix Q(p, α) and the current state p, select the action α with the specific maximum value * where the action α with the specific maximum value * has the following calculation formula:

[0109]

[0110] In this embodiment, by calculating and selecting the optimal action in the current state, it is ensured that the control strategy can accurately adapt to the changes in the dynamic environment. In one embodiment, if the current water quality is severely polluted, priority is given to increasing the pump speed or adjusting the filtration parameters.

[0111] Preferably, it further includes the following steps: calculating the operating indicators of the water purifier and evaluating them to obtain an evaluation result, where the operating indicators of the water purifier include the water quality improvement rate, the filter element life extension rate, and the unit energy consumption reduction rate; based on the evaluation result, the parameters of the deep hybrid Q-network model are adjusted in real time. In this embodiment, by adjusting the parameters of the deep hybrid Q-network model in real time, continuous optimization of the deep hybrid Q-network model can be achieved, so as to ensure the water purification quality and efficient operation of the water purifier in a complex environment.

[0112] In addition, in each embodiment of the present invention, each functional unit can be integrated in a processing module, or each unit can exist physically alone, or two or more units can be integrated in one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0113] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A water purifier control method based on a deep hybrid Q-network, characterized in that: Including the following steps: Step S1: Collect the operation data of the water purifier. Among them, the operation data of the water purifier includes the inlet water flow rate, the outlet water flow rate, the TDS value of the dissolved solids in the water, the water temperature, the turbidity, the PH value, and the inlet water pressure; Step S2: Preprocess the operation data of the water purifier to obtain the preprocessed operation data of the water purifier; Step S3: Establish a deep hybrid Q-network model. Among them, the deep hybrid Q-network model includes a long short-term memory network LSTM, a graph convolutional network GCN, and a deep Q-network DQN; Step S4: Use the preprocessed operation data of the water purifier to train the deep hybrid Q-network model to obtain the trained deep hybrid Q-network model; Step S5: Collect the real-time operation data of the water purifier and the spatial information of the built-in sensors of the water purifier; Step S6: Input the real-time operation data of the water purifier into the LSTM in the trained deep hybrid Q-network model for feature extraction, and output a time feature vector; and input the real-time operation data of the water purifier and the spatial information of the built-in sensors of the water purifier into the GCN in the trained deep hybrid Q-network model for feature extraction, and output a spatial feature vector; Step S7: Input the time feature vector and the spatial feature vector into the DQN in the trained deep hybrid Q-network model for processing, and output a state-action feature matrix; Step S8: Generate a control strategy for the water purifier according to the state-action feature matrix.

2. The water purifier control method based on the deep hybrid Q-network according to claim 1, wherein: In step S2, it specifically includes the following sub-steps: Normalize the operating data {x1, x2, …, x i , …, x7} of the water purifier to obtain the operating data x norm of the water purifier after normalization, where i = 1, ..., 7, and the specific calculation formula is as follows: where min(x i ) represents the minimum value of the operating data of the water purifier; max(x i ) represents the maximum value of the operating data of the water purifier; And using the bidirectional interpolation method to fill in the missing values of the operating data x of the water purifier, and combining the time step t and the spatial information s to obtain the operating data {x1’, x2’, …, x i ’, …, x7’, t, s} of the preprocessed water purifier. i ​ 3. The water purifier control method based on the deep hybrid Q-network according to claim 1, wherein: In step S3, the LSTM includes an input gate l t , a forget gate f t , an output gate o t and a candidate memory cell Among them, the input gate l t is used to determine the contribution of the time-series data a t to the hidden state. The specific mathematical expression is as follows: l t = σ(W l a t + U l h t-1 + b l ); Forgotten gate f t It is used to determine how much of the previous hidden state information should be retained at the current time step. The specific mathematical expression is as follows: f t = σ(W f a t + U f h t-1 + b f ); Output gate o t It is used to determine the influence of the hidden state on the final output, and the specific mathematical expression is as follows: o t = σ(W O a t + U O h t-1 + b O ); Candidate memory unit For combining sequential data a t And the hidden state of the previous time step to update the memory unit state c t , and the specific mathematical expression is as follows: Memory cell state c t The updated mathematical expression is as follows: Among them, a t represents the input data at the t-th time step, i.e., the time series data, h t-1 represents the hidden state at the (t - 1)-th time step, c t-1 represents the memory cell state at the (t - 1)-th time step, W l represents the weight matrix from the input data to the input gate, W f represents the weight matrix from the input data to the forget gate, W O represents the weight matrix from the input data to the output gate, W c represents the weight matrix from the input data to the candidate memory cell, U l represents the weight matrix from the hidden state to the input gate, U f represents the weight matrix from the hidden state to the forget gate, U O represents the weight matrix from the hidden state to the output gate, U c represents the weight matrix from the hidden state to the candidate memory cell, b l , b f , b O and b c all represent bias terms, σ(x) represents the Sigmoid activation function, tanh(x) represents the hyperbolic tangent function, and ⊙ represents the symbol for element-wise multiplication; In step S6, input the real-time operation data of the water purifier into the LSTM in the trained deep hybrid Q-network model for feature extraction, and output a time feature vector, which specifically includes the following sub-steps: Step S61: Organize the real-time operation data of the water purifier into time-series data a t , where t = 1,..., T, and T represents the length of the time series; Step S62: Input a t into the LSTM for processing to obtain the hidden state h at each time step t , where the mathematical formula for h t is as follows: h t = o t ⊙tanh(c t ); Step S63: Take the hidden state h of the last time step T as the time feature vector.

4. The water purifier control method based on a deep hybrid Q-network according to claim 1, characterized in that: In step S3, the GCN includes N graph convolutional layers; In step S6, input the real-time operation data of the water purifier and the spatial information of the built-in sensors of the water purifier into the GCN in the trained deep hybrid Q-network model for feature extraction, and output a spatial feature vector, which specifically includes the following sub-steps: Step S64: Define the nodes and edges in the spatial topology graph. Among them, each node corresponds to a sensor; Step S65: Construct a spatial topology graph according to the nodes and edges in the spatial topology graph, and calculate the adjacency matrix A corresponding to the spatial topology graph; Step S66: Normalize the adjacency matrix A corresponding to the spatial topology graph to obtain the normalized adjacency matrix where the adjacency matrix has the following calculation formula: Among them, D represents the degree matrix; Step S67: Input the real-time operation data of the water purifier and the normalized adjacency matrix into the nodes in the spatial topology graph, and perform graph convolution operations through each graph convolution layer to update the features of the corresponding nodes; Step S68: Take the feature matrix H output by the last graph convolutional layer (N) as the spatial feature vector; Among them, the feature matrix H output by the last graph convolutional layer (N) The specific calculation formula is as follows: Among them, H (N-1) represents the feature matrix output by the (N - 1)-th graph convolutional layer; B (N-1) represents the trainable weight matrix of the (N - 1)-th graph convolutional layer.

5. The water purifier control method based on a deep hybrid Q-network according to claim 1, characterized in that: In step S7, the specific calculation formula of the state-action feature matrix is as follows: Among them, Q(p, α) represents the state-action feature matrix; p represents the current state, including the operating state of the water purifier; α represents the action, including the control operation of the water purifier; r represents the reward function, which is used to evaluate the action effect; γ represents the discount factor, which is used to balance short-term benefits and long-term interests; represents the maximum action value under the next state p'.

6. The water purifier control method based on a deep hybrid Q-network according to claim 5, wherein: In step S8, it specifically includes the following sub-steps: Based on the state-action feature matrix Q(p, α) and the current state p, select the action α with the specific maximum value * , where the action α with the specific maximum value * has the following calculation formula:

7. The water purifier control method based on the deep hybrid Q-network according to claim 1, characterized in that: It also includes the following steps: Calculate the operation indicators of the water purifier and evaluate them to obtain an evaluation result. Among them, the operation indicators of the water purifier include the water quality improvement rate, the filter element life extension rate, and the unit energy consumption reduction rate; Based on the evaluation result, adjust the parameters of the deep hybrid Q-network model in real time.

Images