A gas separation method based on hierarchical clustering and reinforcement learning

Abstract

The application discloses a gas separation method based on hierarchical clustering and reinforcement learning, comprising the following steps: S1, collecting gas data and preprocessing; S2, performing layer-by-layer clustering, constructing a clustering unit and a sample boundary index; S3, constructing a hierarchical state space, generating a state node and recording a parameter interval and a transition relationship; S4, establishing a reinforcement learning environment, generating a state-action-reward triple; S5, training a strategy network in parallel on a GPU, and updating parameters by traversing nodes; S6, acquiring gas data in real time, calling a strategy network to schedule separation and recording feedback. The application combines hierarchical clustering and reinforcement learning, realizes intelligent hierarchical modeling and dynamic optimization control of complex gas working conditions, and significantly improves gas separation efficiency, reduces energy consumption and enhances system stability.

Description

Technical Field

[0001] This invention relates to the field of reinforcement learning technology, and in particular to a gas separation method based on hierarchical clustering and reinforcement learning. Background Technology

[0002] Existing gas separation technologies often rely on traditional physical separation methods or experience-based parameter control strategies, such as compressed air adsorption, cryogenic distillation, or membrane separation. These methods achieve gas mixture separation by manually setting operating parameters or using fixed control strategies. However, due to the complexity of gas components, variable operating conditions, and significant dynamic characteristics of the equipment, traditional methods often struggle to maintain high and stable separation performance under all operating conditions, easily leading to increased energy consumption, decreased separation efficiency, or unstable equipment operation.

[0003] In recent years, some studies have attempted to introduce machine learning or intelligent control into gas separation processes to achieve automated parameter optimization. For example, neural networks are used to predict separation efficiency and adjust operating parameters, or reinforcement learning is used to optimize a single operating strategy under finite conditions. These methods have improved separation performance to some extent, but they have several significant shortcomings: First, existing methods often represent operating conditions too simply, failing to fully characterize the complex, high-dimensional, and variable gas properties, making it difficult for the strategy to generalize. Second, control strategy training is often performed directly in the original high-dimensional feature space, resulting in high state dimensionality, slow convergence speed, and a tendency to get trapped in local optima, and insufficient adaptability to new operating conditions. Third, existing methods lack hierarchical structures or hierarchical constraints, making it impossible to efficiently generate and update state nodes at different operating condition granularities, thus limiting the flexibility and scalability of the strategy network.

[0004] Therefore, how to provide a gas separation method based on hierarchical clustering and reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a gas separation method based on hierarchical clustering and reinforcement learning. This invention fully utilizes the hierarchical clustering algorithm to perform hierarchical modeling of complex gas conditions, and combines it with a reinforcement learning policy network to dynamically optimize and control the adjustable parameters of each state node. The invention details the complete implementation process from data acquisition, preprocessing, hierarchical clustering, hierarchical state space construction to policy training and online process scheduling, and has the advantages of high separation efficiency, low energy consumption, strong adaptability and intelligent control.

[0006] A gas separation method based on hierarchical clustering and reinforcement learning according to an embodiment of the present invention includes the following steps: S1. Collect gas data through sensor network and vision inspection equipment, and perform preprocessing; S2. Based on the preprocessed gas data, perform layer-by-layer clustering calculations to construct a hierarchical clustering structure consisting of parent-layer clustering units and child-layer clustering units, and establish a sample boundary range and sample index table for each clustering unit. S3. Construct a hierarchical state space according to the hierarchical clustering structure, convert each clustering unit into a state node, record the adjustable parameter range and transition relationship for each state node, and form a set of states with hierarchical constraints. S4. Establish a reinforcement learning training environment based on the state set, take the adjustable parameters corresponding to each state node as the action set, take the separation efficiency and energy consumption as the reward value, and generate a triplet sequence of state, action and reward. S5. On the GPU, a policy gradient-based reinforcement learning algorithm is used to perform parallel training on the policy network. The state nodes are traversed according to the hierarchical clustering structure to complete policy sampling and parameter updates, and the policy network parameter set is obtained. S6. Real-time gas data is acquired through the Internet of Things control system, the strategy network parameter set is called to generate process scheduling instructions, and gas separation is completed using distributed control equipment and feedback sequences are recorded.

[0007] Optionally, the gas data is acquired through a sensor network and visual inspection equipment deployed within the gas separation device. The sensor network includes gas component sensors, pressure sensors, temperature sensors, and flow sensors, which measure the gas component concentration, operating pressure, temperature distribution, and gas flow rate of each channel within the device in real time. The visual inspection equipment acquires images of gas flow status or interface features using a high-resolution camera, with the acquisition frequency synchronized with the operating cycle of the separation device. The preprocessing includes performing missing value interpolation, noise filtering, data normalization, timestamp alignment, and feature extraction on the sensor data and visual image signals to generate a standardized feature vector set that can be used for layer-by-layer clustering and state mapping, while also removing and smoothing outliers.

[0008] Optionally, S2 specifically includes: S21. Divide the standardized feature vector set into several coarse-grained clusters according to the initial clustering strategy, calculate the center vector and sample distribution within each cluster, and generate the initial clustering table of the parent layer clustering unit. S22. Perform fine-grained grouping on the feature vectors within each parent layer clustering unit, calculate the similarity between samples based on Euclidean distance or cosine similarity, and generate sub-layer clustering units through recursive partitioning. At the same time, establish a sample index table and sample boundary range record for each sub-layer clustering unit. S23. Establish a hierarchical relationship between the parent and child layers, clarify the parent layer clustering unit number and boundary constraints corresponding to each child layer clustering unit, so that the hierarchical clustering structure forms a traversable multi-layer node tree, and provides hierarchical constraint information for subsequent state mapping. S24. Perform boundary optimization operation on each cluster unit. By calculating the range of maximum and minimum feature values ​​within the cluster, removing outliers, and smoothing the boundaries, ensure that the boundaries of the cluster units closely cover the samples they contain. At the same time, update the sample index table to reflect the optimized boundary distribution. S25. Store the optimized parent and child clustering units in a data structure in sequence, and generate a state mapping table that can be used for reinforcement learning training, recording the center vector, boundary range, sample index and hierarchical relationship information of each clustering unit.

[0009] Optionally, the initial clustering strategy specifically includes: The preprocessed standardized feature vector set is subjected to feature weight normalization processing to map different types of sensor data and visual features to the same dimension range. The similarity matrix between samples is calculated based on the weighted eigenvectors, and the matrix elements are taken as the similarity measure of Euclidean distance or cosine similarity between samples; A bottom-up agglomerative hierarchical clustering method is adopted, treating each sample as a separate cluster. The average link distance between clusters is calculated iteratively, and the closest clusters are selected for merging until all samples are aggregated into several coarse-grained parent clusters. At the same time, the parent-child relationship and the sample index within the cluster are recorded for each merging operation. During the clustering process, a dynamic distance threshold and a cluster capacity limit are set, and the cluster merging strategy is automatically adjusted to avoid clusters being too large or too sparse. Finally, an initial clustering table is generated for each parent layer clustering unit. The initial clustering table includes a center vector, boundary range, and sample index table, which provides a basis for fine-grained sub-layer clustering.

[0010] Optionally, the process of establishing the hierarchical relationship specifically includes: Construct a similarity matrix by arranging all standardized feature vectors within the parent clustering unit according to the feature dimension. The matrix elements are the Euclidean distances between the vectors. Initial sub-clusters are then divided by setting a dynamic threshold. For each initial sub-cluster, recursive aggregation and splitting operations are performed. The average distance and maximum distance ratio within the cluster are used as the splitting criteria. Clusters that are too large or too sparse are further divided or merged. At the same time, the center vector and boundary range of each sub-cluster are updated. Generate a sample index table for each sub-cluster, recording the unique identifier of the sample to which it belongs and its relative position in the parent clustering unit; A hierarchical mapping relationship is established between parent and child layers, associating each child cluster with its corresponding parent cluster to form a complete traversable multi-layer tree structure, ensuring that the boundaries, centers, and indexes of each clustering unit are stored uniformly in the data structure.

[0011] Optionally, S3 specifically includes: S31. The parent and child clustering units are traversed sequentially, and a corresponding state node is generated for each clustering unit. The state node records the central feature vector, boundary range, and sample index of the clustering unit to form a state information table. S32. Define an adjustable parameter range for each state node, map the adsorption pressure, temperature, flow rate, and switching cycle in the gas separation device to the state node, and determine the minimum, maximum, and deviation of each parameter based on the feature vector distribution of the cluster unit to form a parameter constraint table. S33. Establish the transition relationship between state nodes based on the hierarchical clustering structure, generate a traversable hierarchical state space according to the hierarchical paths from parent to child, parallel nodes in the same layer, and child to parent, and record the condition constraints corresponding to each transition path in the state information table to ensure that the hierarchical constraints are followed during the strategy sampling process. S34. Store the state information table, parameter constraint table, and transition relationship in a hierarchical state space to form a multi-layered state set that can be directly used for reinforcement learning training, and support hierarchical traversal, parallel querying, and dynamic updating.

[0012] Optionally, S4 specifically includes: S41. Load each state node in the hierarchical state space into the reinforcement learning training environment, generate a trainable state vector based on the central feature vector, boundary range and hierarchical constraint information recorded in the state node, and associate each state node with a set of adjustable parameters in the reinforcement learning training environment to form an action space. S42. Define the reward function for reinforcement learning, using efficiency and energy consumption indicators as the main evaluation parameters. Calculate the reward value after each action is performed using a weighted combination or multi-objective evaluation method, and record the reward value in real time in the reinforcement learning training environment to feed back to the policy network. S43. In the reinforcement learning training environment, perform action sampling for each state node in sequence, simulate the next state reached after the action is executed according to the state information table, and form a complete triplet sequence of the current state, action and corresponding reward value, and store it in the training dataset in sequence to ensure that the training data covers all state nodes. S44. Organize the triplet sequence into a unified data structure that can be called by reinforcement learning algorithms, supporting hierarchical clustering structure traversal, parallel reading and dynamic updating, so that the policy network can obtain global state information, action constraints and reward feedback in real time during training.

[0013] Optionally, S5 specifically includes: S51. Load the hierarchical state space and the corresponding action set into the GPU parallel computing unit, initialize the policy network input for each state node, and organize the state node traversal sequence according to the hierarchical clustering structure from parent layer to child layer. S52. Parallel policy sampling is performed on each state node on the GPU. The policy network outputs the action distribution based on the current state vector. Action selection is generated by sampling according to the action distribution. At the same time, the immediate reward value after the action is executed and the feature vector of the next state node are calculated to form a complete training data batch. S53. A reinforcement learning algorithm based on policy gradients performs parallel gradient calculations on batches of training data, updates the policy network parameters synchronously based on the reward value, and accumulates gradients layer by layer according to the hierarchical clustering structure to ensure that the policy network obtains a globally optimized parameter set after traversing the complete hierarchical state space. S54. Store the updated policy network parameter set in GPU shared memory and generate a callable policy network weight file.

[0014] Optionally, the parallel strategy sampling process specifically includes: The feature vectors of the state nodes are loaded into the GPU tensor computation unit in batches, and each thread performs parallel computation on one state node. The policy network receives the input state vector in each thread, generates the action distribution probability vector through forward propagation, and selects actions by random sampling of the action distribution. At the same time, it records the current state, sampled actions, and the adjustable parameter values ​​corresponding to the actions within the same thread. After the action is executed, the immediate reward value returned by the simulation environment and the feature vector of the next state are written to the training data buffer. The GPU uses shared memory and atomic operations to synchronously update the training batch index, ensuring that all parallel threads complete the generation of triplet of state, action and reward in the same batch, which can be directly used for policy gradient calculation, realizing efficient parallel training and data acquisition in large-scale hierarchical state space.

[0015] The beneficial effects of this invention are: First, by performing hierarchical modeling of complex gas conditions through layer-by-layer clustering, the feature vectors, boundary ranges, and sample indices of the parent and child clustering units are structured into an traversable hierarchical state space. This enables the system to accurately represent high-dimensional and variable gas states, providing clear and interpretable state inputs for subsequent strategy optimization, thereby significantly improving the accuracy and generalization ability of state representation.

[0016] Secondly, by using a reinforcement learning strategy network to dynamically optimize the adjustable parameters of each state node in a hierarchical state space, and by establishing a triple sequence of state, action and reward and training it in parallel, intelligent control of key operating parameters such as adsorption pressure, temperature, flow rate and switching cycle is achieved. This enables the gas separation process to adapt to changes in different operating conditions, significantly improves separation efficiency and reduces energy consumption, while reducing human intervention and achieving intelligent and automated control.

[0017] Finally, during online operation, this invention combines an IoT control system to acquire gas data in real time and call the trained strategy network to generate process scheduling instructions, enabling the entire gas separation device to maintain a stable and efficient operating state under different operating conditions. This not only ensures the quality and yield of gas separation but also enhances the adaptability and robustness of the system, achieving a safe, reliable, and efficient gas separation process. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a gas separation method based on hierarchical clustering and reinforcement learning proposed in this invention; Figure 2 This is a flowchart illustrating the layer-by-layer clustering and hierarchical state space construction process of a gas separation method based on hierarchical clustering and reinforcement learning proposed in this invention. Figure 3 This is a flowchart illustrating the reinforcement learning training and real-time control process of a gas separation method based on hierarchical clustering and reinforcement learning proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figure 1-3 A gas separation method based on hierarchical clustering and reinforcement learning includes the following steps: S1. Collect gas data through sensor network and vision inspection equipment, and perform preprocessing; S2. Based on the preprocessed gas data, perform layer-by-layer clustering calculations to construct a hierarchical clustering structure consisting of parent-layer clustering units and child-layer clustering units, and establish a sample boundary range and sample index table for each clustering unit. S3. Construct a hierarchical state space according to the hierarchical clustering structure, convert each clustering unit into a state node, record the adjustable parameter range and transition relationship for each state node, and form a set of states with hierarchical constraints. S4. Establish a reinforcement learning training environment based on the state set, take the adjustable parameters corresponding to each state node as the action set, take the separation efficiency and energy consumption as the reward value, and generate a triplet sequence of state, action and reward. S5. On the GPU, a policy gradient-based reinforcement learning algorithm is used to perform parallel training on the policy network. The state nodes are traversed according to the hierarchical clustering structure to complete policy sampling and parameter updates, and the policy network parameter set is obtained. S6. Real-time gas data is acquired through the Internet of Things control system, the strategy network parameter set is called to generate process scheduling instructions, and gas separation is completed using distributed control equipment and feedback sequences are recorded.

[0021] In this embodiment, the gas data is collected through a sensor network and visual inspection equipment deployed within the gas separation device. The sensor network includes gas component sensors, pressure sensors, temperature sensors, and flow sensors, which measure the gas component concentration, operating pressure, temperature distribution, and gas flow rate of each channel within the device in real time. The visual inspection equipment acquires images of gas flow state or interface features using a high-resolution camera, with the acquisition frequency synchronized with the operating cycle of the separation device. The preprocessing includes performing missing value interpolation, noise filtering, data normalization, timestamp alignment, and feature extraction on the sensor data and visual image signals to generate a standardized feature vector set that can be used for layer-by-layer clustering and state mapping. At the same time, outliers are removed and smoothed.

[0022] In this embodiment, S2 specifically includes: S21. Divide the standardized feature vector set into several coarse-grained clusters according to the initial clustering strategy, calculate the center vector and sample distribution within each cluster, and generate the initial clustering table of the parent layer clustering unit. S22. Perform fine-grained grouping on the feature vectors within each parent layer clustering unit, calculate the similarity between samples based on Euclidean distance or cosine similarity, and generate sub-layer clustering units through recursive partitioning. At the same time, establish a sample index table and sample boundary range record for each sub-layer clustering unit. S23. Establish a hierarchical relationship between the parent and child layers, clarify the parent layer clustering unit number and boundary constraints corresponding to each child layer clustering unit, so that the hierarchical clustering structure forms a traversable multi-layer node tree, and provides hierarchical constraint information for subsequent state mapping. S24. Perform boundary optimization operation on each cluster unit. By calculating the range of maximum and minimum feature values ​​within the cluster, removing outliers, and smoothing the boundaries, ensure that the boundaries of the cluster units closely cover the samples they contain. At the same time, update the sample index table to reflect the optimized boundary distribution. S25. Store the optimized parent and child clustering units in a data structure in sequence, and generate a state mapping table that can be used for reinforcement learning training, recording the center vector, boundary range, sample index and hierarchical relationship information of each clustering unit.

[0023] In this embodiment, the initial clustering strategy specifically includes: The preprocessed standardized feature vector set is subjected to feature weight normalization processing to map different types of sensor data and visual features to the same dimension range. The similarity matrix between samples is calculated based on the weighted eigenvectors, and the matrix elements are taken as the similarity measure of Euclidean distance or cosine similarity between samples; A bottom-up agglomerative hierarchical clustering method is adopted, treating each sample as a separate cluster. The average link distance between clusters is calculated iteratively, and the closest clusters are selected for merging until all samples are aggregated into several coarse-grained parent clusters. At the same time, the parent-child relationship and the sample index within the cluster are recorded for each merging operation. During the clustering process, a dynamic distance threshold and a cluster capacity limit are set, and the cluster merging strategy is automatically adjusted to avoid clusters being too large or too sparse. Finally, an initial clustering table is generated for each parent layer clustering unit. The initial clustering table includes a center vector, boundary range, and sample index table, which provides a basis for fine-grained sub-layer clustering.

[0024] In this embodiment, the process of establishing the hierarchical relationship specifically includes: Construct a similarity matrix by arranging all standardized feature vectors within the parent clustering unit according to the feature dimension. The matrix elements are the Euclidean distances between the vectors. Initial sub-clusters are then divided by setting a dynamic threshold. For each initial sub-cluster, recursive aggregation and splitting operations are performed. The average distance and maximum distance ratio within the cluster are used as the splitting criteria. Clusters that are too large or too sparse are further divided or merged. At the same time, the center vector and boundary range of each sub-cluster are updated. Generate a sample index table for each sub-cluster, recording the unique identifier of the sample to which it belongs and its relative position in the parent clustering unit; A hierarchical mapping relationship is established between parent and child layers, associating each child cluster with its corresponding parent cluster to form a complete traversable multi-layer tree structure, ensuring that the boundaries, centers, and indexes of each clustering unit are stored uniformly in the data structure.

[0025] In this embodiment, S3 specifically includes: S31. The parent and child clustering units are traversed sequentially, and a corresponding state node is generated for each clustering unit. The state node records the central feature vector, boundary range, and sample index of the clustering unit to form a state information table. S32. Define adjustable parameter ranges for each state node, map the adsorption pressure, temperature, flow rate, and switching cycle in the gas separation device to the state node, and determine the minimum, maximum, and deviation values ​​of each parameter based on the feature vector distribution of the cluster unit to form a parameter constraint table, specifically including: Extract the corresponding gas features based on the central feature vector and boundary range of the cluster unit to which the state node belongs, namely: component concentration, operating pressure, temperature distribution and gas flow rate; Each gas characteristic is mapped to an adjustable operating parameter of the gas separation device, and the minimum, maximum and recommended step size of each parameter are determined by historical operating data; Discretization is performed on each parameter, dividing the continuous parameter space into several sampleable discrete intervals, and a parameter constraint table is generated to record the state conditions and device operation restrictions corresponding to each discrete value. Correct and smooth outliers and boundary values ​​to ensure that parameter ranges remain hierarchically consistent across different state nodes; The parameter range of each state node is stored together with its corresponding cluster center, boundary information and sample index to form a complete parameter mapping table; S33. Establish the transition relationship between state nodes based on the hierarchical clustering structure, generate a traversable hierarchical state space according to the hierarchical paths from parent to child, parallel nodes in the same layer, and child to parent, and record the condition constraints corresponding to each transition path in the state information table to ensure that the hierarchical constraints are followed during the strategy sampling process. S34. Store the state information table, parameter constraint table, and transition relationship in a hierarchical state space to form a multi-layered state set that can be directly used for reinforcement learning training, and support hierarchical traversal, parallel querying, and dynamic updating.

[0026] In this embodiment, S4 specifically includes: S41. Load each state node in the hierarchical state space into the reinforcement learning training environment, generate a trainable state vector based on the central feature vector, boundary range and hierarchical constraint information recorded in the state node, and associate each state node with a set of adjustable parameters in the reinforcement learning training environment to form an action space. S42. Define the reward function for reinforcement learning, using efficiency and energy consumption indicators as the main evaluation parameters. Calculate the reward value after each action is performed using a weighted combination or multi-objective evaluation method, and record the reward value in real time in the reinforcement learning training environment to feed back to the policy network. S43. In the reinforcement learning training environment, perform action sampling sequentially for each state node, simulate the next state reached after the action is executed according to the state information table, and form a complete triplet sequence of the current state, action, and corresponding reward value, and store it sequentially in the training dataset to ensure that the training data covers all state nodes. Specifically, this includes: Generate a set of discrete or continuous actions from the adjustable parameter range of the state node, and filter out action candidates that meet the hierarchical constraints according to the parameter constraint table. In the training environment, each action is applied to simulate the gas separation process. The instantaneous reward value is obtained by calculating the separation efficiency and energy consumption index under the action. The current state vector, the executed action and the reward value are recorded as a triple. After sampling all actions of a state node, the next state node is determined based on the state information table and the action sampling is repeated. The parent and child clustering units are traversed in turn to form a complete triplet sequence. Abnormal or unreachable states during the sampling process are removed and marked to ensure the integrity and effectiveness of the training dataset; S44. Organize the triplet sequence into a unified data structure that can be called by reinforcement learning algorithms, supporting hierarchical clustering structure traversal, parallel reading and dynamic updating, so that the policy network can obtain global state information, action constraints and reward feedback in real time during training.

[0027] In this embodiment, S5 specifically includes: S51. Load the hierarchical state space and the corresponding action set into the GPU parallel computing unit, initialize the policy network input for each state node, and organize the state node traversal sequence according to the hierarchical clustering structure from parent layer to child layer. S52. Parallel policy sampling is performed on each state node on the GPU. The policy network outputs the action distribution based on the current state vector. Action selection is generated by sampling according to the action distribution. At the same time, the immediate reward value after the action is executed and the feature vector of the next state node are calculated to form a complete training data batch. S53. A reinforcement learning algorithm based on policy gradients performs parallel gradient calculations on batches of training data, updates the policy network parameters synchronously based on the reward value, and accumulates gradients layer by layer according to the hierarchical clustering structure to ensure that the policy network obtains a globally optimized parameter set after traversing the complete hierarchical state space. S54. Store the updated policy network parameter set in GPU shared memory and generate a callable policy network weight file.

[0028] In this embodiment, the parallel strategy sampling process specifically includes: The feature vectors of the state nodes are loaded into the GPU tensor computation unit in batches, and each thread performs parallel computation on one state node. The policy network receives the input state vector in each thread, generates the action distribution probability vector through forward propagation, and selects actions by random sampling of the action distribution. At the same time, it records the current state, sampled actions, and the adjustable parameter values ​​corresponding to the actions within the same thread. After the action is executed, the immediate reward value returned by the simulation environment and the feature vector of the next state are written to the training data buffer. The GPU uses shared memory and atomic operations to synchronously update the training batch index, ensuring that all parallel threads complete the generation of triplet of state, action and reward in the same batch, which can be directly used for policy gradient calculation, realizing efficient parallel training and data acquisition in large-scale hierarchical state space.

[0029] Example 1: To verify the feasibility of this invention in practice, it was applied to the intelligent control process of a multi-component gas separation device. In this scenario, traditional gas separation methods rely on experience to set operating parameters, facing problems such as large fluctuations in gas composition, unstable separation efficiency, high energy consumption, and heavy manual adjustment burden. By employing this invention, a sensor network and high-resolution visual inspection equipment are deployed to collect data such as gas component concentration, pressure, temperature, and flow rate in real time, while simultaneously acquiring image information of the gas flow state. All collected raw data undergoes missing value interpolation, noise filtering, normalization, and feature extraction to form a standardized feature vector set.

[0030] Based on this, layer-by-layer clustering is performed on the preprocessed feature vectors to construct a hierarchical clustering structure composed of parent and child clustering units. Each clustering unit records the sample boundary range and sample index, forming a complete hierarchical state space through the parent-child layer relationship. Subsequently, adjustable parameter ranges are defined for each state node, including adsorption pressure between 0.8 and 1.6 MPa, temperature between 280 and 320 Kelvin, and flow rate between 50 and 150 standard cubic meters per hour. Transition relationships between state nodes are established, enabling the reinforcement learning policy network to complete policy sampling and parameter updates under hierarchical constraints. The policy gradient reinforcement learning algorithm is used for parallel training on a GPU. Historical operating data and simulation environment are used to iteratively optimize the policy network. After training, the policy network can generate process scheduling instructions based on real-time gas data, automatically adjust the parameters of each channel, and execute gas separation operations.

[0031] To visually demonstrate the application's effectiveness, the runtime data is summarized in the table below: Table 1 Data on the Intelligent Control Effect of Gas Separation Unit

[0032] During application, the device can respond quickly to different mixed gas compositions and operating conditions. For example, when processing a mixed gas containing nitrogen, oxygen, carbon dioxide, and a small amount of rare gases, the separation efficiency increased from an average of 85% to over 95% after the optimization strategy of this invention, the gas purity increased from 90% to 98%, energy consumption decreased by about 12%, and the gas processing capacity remained at 120 standard cubic meters per hour. Simultaneously, the device maintained stable operation under fluctuating conditions, with deviations controlled within ±1.5%. In long-term continuous operation tests, the strategy network dynamically adjusted operating parameters based on real-time data, avoiding the efficiency decline caused by changes in operating conditions in traditional methods, further verifying the feasibility and superiority of the method.

[0033] As can be seen from the above embodiments, the present invention effectively solves the problems of complex operating conditions and difficulty in adaptive parameter optimization of traditional gas separation methods in practical applications, and achieves the comprehensive advantages of high separation efficiency, high gas purity and low energy consumption, while ensuring the stable operation of the system under dynamic operating conditions.

[0034] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A gas separation method based on hierarchical clustering and reinforcement learning, characterized in that, Includes the following steps: S1. Collect gas data through sensor network and vision inspection equipment, and perform preprocessing; S2. Based on the preprocessed gas data, perform layer-by-layer clustering calculations to construct a hierarchical clustering structure consisting of parent-layer clustering units and child-layer clustering units, and establish a sample boundary range and sample index table for each clustering unit. S3. Construct a hierarchical state space according to the hierarchical clustering structure, convert each clustering unit into a state node, record the adjustable parameter range and transition relationship for each state node, and form a set of states with hierarchical constraints. S4. Establish a reinforcement learning training environment based on the state set, take the adjustable parameters corresponding to each state node as the action set, take the separation efficiency and energy consumption as the reward value, and generate a triplet sequence of state, action and reward. S5. On the GPU, a policy gradient-based reinforcement learning algorithm is used to perform parallel training on the policy network. The state nodes are traversed according to the hierarchical clustering structure to complete policy sampling and parameter updates, and the policy network parameter set is obtained. S6. Real-time gas data is acquired through the Internet of Things control system, the strategy network parameter set is called to generate process scheduling instructions, and gas separation is completed using distributed control equipment and feedback sequences are recorded.

2. The gas separation method based on hierarchical clustering and reinforcement learning according to claim 1, characterized in that, The gas data is collected through a sensor network and visual inspection equipment deployed within the gas separation device. The sensor network includes gas component sensors, pressure sensors, temperature sensors, and flow sensors, which measure the gas component concentration, operating pressure, temperature distribution, and gas flow rate in each channel of the device in real time. The visual inspection equipment acquires images of gas flow state or interface features through a high-resolution camera, with the acquisition frequency synchronized with the operating cycle of the separation device. The preprocessing includes performing missing value interpolation, noise filtering, data normalization, timestamp alignment, and feature extraction on the sensor data and visual image signals to generate a standardized feature vector set that can be used for layer-by-layer clustering and state mapping. At the same time, outliers are removed and smoothed.

3. The gas separation method based on hierarchical clustering and reinforcement learning according to claim 1, characterized in that, S2 specifically includes: S21. Divide the standardized feature vector set into several coarse-grained clusters according to the initial clustering strategy, calculate the center vector and sample distribution within each cluster, and generate the initial clustering table of the parent layer clustering unit. S22. Perform fine-grained grouping on the feature vectors within each parent layer clustering unit, calculate the similarity between samples based on Euclidean distance or cosine similarity, and generate sub-layer clustering units through recursive partitioning. At the same time, establish a sample index table and sample boundary range record for each sub-layer clustering unit. S23. Establish a hierarchical relationship between the parent and child layers, clarify the parent layer clustering unit number and boundary constraints corresponding to each child layer clustering unit, so that the hierarchical clustering structure forms a traversable multi-layer node tree, and provides hierarchical constraint information for subsequent state mapping. S24. Perform boundary optimization operation on each cluster unit. By calculating the range of maximum and minimum feature values ​​within the cluster, removing outliers, and smoothing the boundaries, ensure that the boundaries of the cluster units closely cover the samples they contain. At the same time, update the sample index table to reflect the optimized boundary distribution. S25. Store the optimized parent and child clustering units in a data structure in sequence, and generate a state mapping table that can be used for reinforcement learning training, recording the center vector, boundary range, sample index and hierarchical relationship information of each clustering unit.

4. The gas separation method based on hierarchical clustering and reinforcement learning according to claim 3, characterized in that, The initial clustering strategy specifically includes: The preprocessed standardized feature vector set is subjected to feature weight normalization processing to map different types of sensor data and visual features to the same dimension range. The similarity matrix between samples is calculated based on the weighted eigenvectors, and the matrix elements are taken as the similarity measure of Euclidean distance or cosine similarity between samples; A bottom-up agglomerative hierarchical clustering method is adopted, treating each sample as a separate cluster. The average link distance between clusters is calculated iteratively, and the closest clusters are selected for merging until all samples are aggregated into several coarse-grained parent clusters. At the same time, the parent-child relationship and the sample index within the cluster are recorded for each merging operation. During the clustering process, a dynamic distance threshold and a cluster capacity limit are set, and the cluster merging strategy is automatically adjusted to avoid clusters being too large or too sparse. Finally, an initial clustering table is generated for each parent layer clustering unit. The initial clustering table includes a center vector, boundary range, and sample index table, which provides a basis for fine-grained sub-layer clustering.

5. The gas separation method based on hierarchical clustering and reinforcement learning according to claim 3, characterized in that, The process of establishing the hierarchical relationship specifically includes: Construct a similarity matrix by arranging all standardized feature vectors within the parent clustering unit according to the feature dimension. The matrix elements are the Euclidean distances between the vectors. Initial sub-clusters are then divided by setting a dynamic threshold. For each initial sub-cluster, recursive aggregation and splitting operations are performed. The average distance and maximum distance ratio within the cluster are used as the splitting criteria. Clusters that are too large or too sparse are further divided or merged. At the same time, the center vector and boundary range of each sub-cluster are updated. Generate a sample index table for each sub-cluster, recording the unique identifier of the sample to which it belongs and its relative position in the parent clustering unit; A hierarchical mapping relationship is established between parent and child layers, associating each child cluster with its corresponding parent cluster to form a complete traversable multi-layer tree structure, ensuring that the boundaries, centers, and indexes of each clustering unit are stored uniformly in the data structure.

6. The gas separation method based on hierarchical clustering and reinforcement learning according to claim 1, characterized in that, S3 specifically includes: S31. The parent and child clustering units are traversed sequentially, and a corresponding state node is generated for each clustering unit. The state node records the central feature vector, boundary range, and sample index of the clustering unit to form a state information table. S32. Define an adjustable parameter range for each state node, map the adsorption pressure, temperature, flow rate, and switching cycle in the gas separation device to the state node, and determine the minimum, maximum, and deviation of each parameter based on the feature vector distribution of the cluster unit to form a parameter constraint table. S33. Establish the transition relationship between state nodes based on the hierarchical clustering structure, generate a traversable hierarchical state space according to the hierarchical paths from parent to child, parallel nodes in the same layer, and child to parent, and record the condition constraints corresponding to each transition path in the state information table to ensure that the hierarchical constraints are followed during the strategy sampling process. S34. Store the state information table, parameter constraint table, and transition relationship in a hierarchical state space to form a multi-layered state set that can be directly used for reinforcement learning training, and support hierarchical traversal, parallel querying, and dynamic updating.

7. The gas separation method based on hierarchical clustering and reinforcement learning according to claim 1, characterized in that, S4 specifically includes: S41. Load each state node in the hierarchical state space into the reinforcement learning training environment, generate a trainable state vector based on the central feature vector, boundary range and hierarchical constraint information recorded in the state node, and associate each state node with a set of adjustable parameters in the reinforcement learning training environment to form an action space. S42. Define the reward function for reinforcement learning, using efficiency and energy consumption indicators as the main evaluation parameters. Calculate the reward value after each action is performed using a weighted combination or multi-objective evaluation method, and record the reward value in real time in the reinforcement learning training environment to feed back to the policy network. S43. In the reinforcement learning training environment, perform action sampling for each state node in sequence, simulate the next state reached after the action is executed according to the state information table, and form a complete triplet sequence of the current state, action and corresponding reward value, and store it in the training dataset in sequence to ensure that the training data covers all state nodes. S44. Organize the triplet sequence into a unified data structure that can be called by reinforcement learning algorithms, supporting hierarchical clustering structure traversal, parallel reading and dynamic updating, so that the policy network can obtain global state information, action constraints and reward feedback in real time during training.

8. The gas separation method based on hierarchical clustering and reinforcement learning according to claim 1, characterized in that, S5 specifically includes: S51. Load the hierarchical state space and the corresponding action set into the GPU parallel computing unit, initialize the policy network input for each state node, and organize the state node traversal sequence according to the hierarchical clustering structure from parent layer to child layer. S52. Parallel policy sampling is performed on each state node on the GPU. The policy network outputs the action distribution based on the current state vector. Action selection is generated by sampling according to the action distribution. At the same time, the immediate reward value after the action is executed and the feature vector of the next state node are calculated to form a complete training data batch. S53. A reinforcement learning algorithm based on policy gradients performs parallel gradient calculations on batches of training data, updates the policy network parameters synchronously based on the reward value, and accumulates gradients layer by layer according to the hierarchical clustering structure to ensure that the policy network obtains a globally optimized parameter set after traversing the complete hierarchical state space. S54. Store the updated policy network parameter set in GPU shared memory and generate a callable policy network weight file.

9. The gas separation method based on hierarchical clustering and reinforcement learning according to claim 1, characterized in that, The parallel strategy sampling process specifically includes: The feature vectors of the state nodes are loaded into the GPU tensor computation unit in batches, and each thread performs parallel computation on one state node. The policy network receives the input state vector in each thread, generates the action distribution probability vector through forward propagation, and selects actions by random sampling of the action distribution. At the same time, it records the current state, sampled actions, and the adjustable parameter values ​​corresponding to the actions within the same thread. After the action is executed, the immediate reward value returned by the simulation environment and the feature vector of the next state are written to the training data buffer. The GPU uses shared memory and atomic operations to synchronously update the training batch index, ensuring that all parallel threads complete the generation of triplet of state, action and reward in the same batch, which can be directly used for policy gradient calculation, realizing efficient parallel training and data acquisition in large-scale hierarchical state space.

Images