Course learning and reinforcement learning driven dynamic optimal self-consistent clustering method

Abstract

The invention provides a course learning and reinforcement learning driven dynamic optimal self-consistent clustering method, and belongs to the technical field of big data processing, and the method comprises the following steps: carrying out the tensor form of original data, and then carrying out the normalization processing to obtain standardized data; information interaction between samples is realized through a GNN graph neural network or a Transform framework; in order to divide the samples into fewest categories and ensure that the samples in each category have consistency, a dynamic division mechanism is set for reinforcement learning; the multi-stage training strategy based on course learning comprises the following steps: firstly, designing course learning, and finally, dynamically adjusting courses. According to the algorithm, clustering division can be adjusted in real time according to dynamic changes of a data set, retraining or manual intervention is not needed, and the algorithm is particularly suitable for dynamic scenes such as user behavior analysis and traffic flow prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of big data processing, and particularly to a dynamic optimal self-consistent clustering method driven by course learning and reinforcement learning. Background Art

[0002] With the wide application of data-driven decision-making in all walks of life, the efficient processing of dynamic data sets has become a core challenge in today's information society. As an important data analysis tool, clustering algorithms are widely used in fields such as pattern recognition, data mining, and machine learning. Its main goal is to divide data into several groups (clusters) such that the similarity of data points within the same cluster is as high as possible, and the similarity between different clusters is as low as possible. However, traditional clustering methods are mainly designed for static data sets, and the current state of the art of existing clustering algorithms is summarized as follows:

[0003] Classical clustering algorithms: such as K-Means, Hierarchical Clustering, and DBSCAN, usually based on predefined distance metrics or density estimates, are suitable for data sets with fixed scale and feature distribution.

[0004] Algorithm characteristics: These algorithms usually require predefining the number of clusters or density parameters and aggregating data points based on global features. However, the sensitivity of the algorithms to input parameters and the processing ability for high-dimensional data limit their application scope.

[0005] Static limitation: Traditional clustering methods are mainly for static data sets, that is, it is assumed that all data points are available when clustering, and the distribution and characteristics of the data do not change during the clustering process. In practical applications, many data sets are dynamic, such as e-commerce user behavior data, network traffic logs, or medical sensor data. These data are continuously updated over time, and traditional clustering methods are difficult to adapt to the incremental update and real-time response requirements of dynamic data.

[0006] With the widespread application of data-driven decision-making in various industries, how to efficiently process dynamically changing data sets has become a key challenge. Among them, data partitioning based on data characteristics is a classic research problem. However, existing clustering methods mainly focus on static data partitioning and have the following defects: Static nature: Most existing algorithms are designed for static data and are difficult to handle the situation where data is dynamically updated over time. Fixed number of clusters: Existing methods usually require the number of clusters to be preset before clustering. This manually designed approach lacks flexibility and is prone to inefficient use or loss of resources. For example, in an e-commerce platform, too few clusters may lead to incorrect grouping of users with different interests, thereby reducing the accuracy of recommendations; while too many clusters may cause waste and redundancy of computing resources. Self-consistency problem: Existing clustering methods cluster based on similarity, but fail to effectively solve the conflict problem within categories, which can easily lead to serious consequences in critical fields such as healthcare. For example, during the prescribing process, if the properties of two drugs conflict, it will cause serious consequences to the patient's health or life if they are in the same prescription. Efficiency limitation: Existing clustering methods based on reinforcement learning are often limited to relatively small data scales and are difficult to scale up to larger data scales, lacking practicality. Moreover, these methods lack real-time response capabilities. Therefore, there is a need to design a dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning. Summary of the Invention

[0007] The purpose of the present invention is to provide a dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning to solve the technical problems existing in the existing clustering methods mentioned in the background art.

[0008] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0009] A dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning, the method comprising the following steps:

[0010] Step 1: Tensorize the original data into a tensor form and then perform normalization processing to obtain standardized data;

[0011] Step 2: Implement information interaction between samples through a GNN graph neural network or a Transformer framework;

[0012] Step 3: To divide the samples into the fewest categories and ensure the consistency of samples within each category, set up a dynamic partitioning mechanism for reinforcement learning;

[0013] Step 4: Based on several stage training strategies of curriculum learning, first design the curriculum learning, and finally perform dynamic curriculum adjustment.

[0014] Furthermore, the dynamic partitioning mechanism in step 3 includes partitioning into existing categories, creating new categories, designing a reward mechanism, and optimizing through reinforcement learning.

[0015] Furthermore, the specific process of creating new categories is as follows: Set up a dynamic masking mechanism and a dynamic partitioning process.

[0016] Dynamic masking mechanism: To handle the problem of uncertain dynamic category numbers, a dynamic masking mechanism is designed to unlock the creation of new categories. The formula for the masking mechanism is as follows:

[0017] Y = softmax{y i , y2, y new , MASK, MASK, MASK, MASK}

[0018] where y new represents the prediction score for creating a new category. The mask MASK is only unlocked when y new is selected, which is used to dynamically add new categories, enabling the model to flexibly adapt to dynamic data and avoid over-partitioning;

[0019] Dynamic partitioning process: For the current sample, the model first determines whether it can be classified into existing categories. If so, it is partitioned into one of the existing categories such as category one or two. If not, a new category is created and the sample is assigned to the new category. Through the dynamic partitioning mechanism, the model achieves clustering optimization with the minimum number of categories.

[0020] Furthermore, the specific process of designing the reward mechanism is as follows:

[0021] The reward mechanism is the core of the reinforcement learning module, which is used to evaluate the quality of the current partitioning state. The goal is to achieve self-consistency among samples with the minimum number of categories. The reward function is designed as follows:

[0022]

[0023] where N represents the total number of samples, S represents the number of categories used,

[0024] Internal consistency reward: If the conflicts among samples in the same category are less than the set value, a higher reward is given. Category number penalty: When the number of partitioned categories is more than the set value, a penalty is imposed to encourage the model to reduce the number of categories.

[0025] Furthermore, the specific process of the dynamic partitioning process is as follows:

[0026] Gradient update: Using the deep reinforcement learning framework, the reward value is converted into a gradient signal to optimize the model parameters. During the update process, the feature distribution of the samples and the current partitioning state are dynamically considered, so as to achieve the optimal partitioning at each step;

[0027] Dynamic class assignment. The model can flexibly decide whether to assign to an existing class or create a new class based on sample characteristics and historical classification status, effectively avoiding the problems of over-clustering or under-clustering.

[0028] Convergence and robustness. Through several rounds of training, the model gradually converges to the optimal strategy. To enhance the adaptability to abnormal data and distribution changes, a random perturbation mechanism is introduced to further improve the robustness of the model.

[0029] Furthermore, the specific process of the curriculum learning design in step 4 is as follows:

[0030] Gradual increase in the number of samples: In the initial stage, in the early stage of training, fewer samples than the set value are used for training, enabling the model to focus on mastering basic knowledge and correctly completing the classification. In the advanced stage: As the model's performance in the early curriculum tends to stabilize, the number of training samples is gradually increased. This step-by-step approach can effectively improve the generalization ability of the model and avoid a sharp increase in the learning difficulty caused by an excessive amount of training data. In the final stage, in the high-order curriculum, although the number of samples is larger and the complexity is higher, since the model already has basic knowledge and reasoning ability, it can more efficiently handle complex tasks.

[0031] Gradual complication of self-consistency conditions: Self-consistency is a complex logical reasoning problem. In curriculum learning, the learning difficulty of the model is gradually increased by simplifying the conditions. In the initial stage: Only a single self-consistency factor is considered. If a certain factor A and -A in sample 1 and sample 2 conflict, then it is determined that the two are not self-consistent and cannot be classified into the same category. In the advanced stage: More judgment conditions are gradually introduced. When considering factors A, B, and C in sample 1 and sample 2 simultaneously, the model needs to comprehensively judge the self-consistency of all factors. If some factors are irrelevant but do not conflict, then the two are still classified into the same category. In the final stage: The model needs to handle more complex factor reasoning problems to complete higher-level self-consistency detection.

[0032] Furthermore, the specific process of dynamic curriculum adjustment is as follows:

[0033] To enhance the adaptability of the model, a dynamic curriculum learning mechanism is designed to adjust the curriculum in real time: When the sample nature or self-consistency conditions change, the curriculum design can be dynamically updated. Adaptive training: The model can quickly adapt to new data characteristics and self-consistency rules, thereby achieving efficient learning under different data distributions and task conditions. Through curriculum learning, in the process of gradually increasing the task difficulty in training, the model can more robustly learn self-consistency reasoning ability and the optimal classification strategy, effectively solving the learning bottleneck problem caused by a large number of samples and high complexity.

[0034] Furthermore, the system for implementing this method includes a data representation and preprocessing module, an information interaction network, a clustering decision network, a reinforcement learning module, and a curriculum learning module. The data representation and preprocessing module is connected to the clustering decision network via the information interaction network, and the clustering decision network is connected to the curriculum learning module via the reinforcement learning module;

[0035] The data representation and preprocessing module is used to clean, normalize, and extract features from the input data, generate an input format suitable for model processing, and the data is fed into the model in sequence to achieve real-time dynamic data processing;

[0036] The information interaction network is a neural network based on information propagation, which updates the data representation using the important information of other nodes, effectively captures the interaction of relationships between data, and thus makes more accurate judgments;

[0037] The clustering decision network is used to tell the neural network the intermediate state of clustering and execute preset actions, such as dividing into existing categories and forming new categories, so as to achieve the purpose of clustering division;

[0038] The reinforcement learning module is used to define a reward mechanism, guide the model to optimize the clustering algorithm through trial and error, and update the parameters of the neural network, so as to make the model smarter;

[0039] The curriculum learning module is used to utilize the concept of curriculum learning during the process of updating the model parameters, train the model in segments, enable the model to gradually transition from simple tasks to complex tasks, drive the model to learn useful knowledge faster, and also enable the model to handle more complex tasks, ensuring that the model has stronger scalability.

[0040] Due to the adoption of the above technical solutions, the present invention has the following beneficial effects:

[0041] (1) The algorithm of the present invention can adjust the clustering division in real time according to the dynamic changes of the data set without retraining or manual intervention, and is particularly suitable for dynamic scenarios such as user behavior analysis and traffic flow prediction.

[0042] (2) Clustering optimization: With the help of the reinforcement learning mechanism, dynamically adjust the number of clusters, which can not only reduce redundancy and save resources, but also ensure that the internal data of each cluster is highly consistent and avoid conflicts.

[0043] (3) High efficiency: Support incremental learning and real-time processing, greatly reducing the computational complexity of the algorithm, especially in solving the problem of difficult processing of large-scale dynamic data in reinforcement learning.

[0044] (4) Versatility: Applicable to a variety of specific application scenarios:

[0045] Personalized Recommendation System: It analyzes users' purchase history, browsing records, and behavioral data in real time, dynamically assigns users to appropriate interest groups, thereby improving the accuracy of recommended products and the purchase conversion rate of users.

[0046] Base Station Optimization: By dynamically monitoring the geographical distribution and traffic demand of users, it optimizes the coverage range and signal strength configuration of base stations, reducing communication interruptions and resource waste.

[0047] Disaster Emergency: It analyzes the material needs and personnel distribution in the disaster-stricken area in real time, quickly allocates relief supplies to the places where they are most needed, and improves the disaster relief efficiency.

[0048] Medical and Health Data Management: Dynamically monitors patients' health conditions and the distribution of medical resources, and optimizes resource allocation and treatment plans.

[0049] Financial Risk Management: It evaluates market fluctuations and customer behaviors in real time, groups customers to adjust risk control strategies, and reduces financial losses. Description of the Drawings

[0050] Figure 1 is the flowchart of the method of the present invention;

[0051] Figure 2 is the sample division diagram of the present invention. Detailed Implementation Manner

[0052] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the following preferred embodiments are given with reference to the accompanying drawings for further detailed description of the present invention. However, it should be noted that many details listed in the specification are only for enabling the reader to have a thorough understanding of one or more aspects of the present invention, and these aspects of the present invention can be implemented even without these specific details.

[0053] As Figure 1 shown, a dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning. This method is composed of an end-to-end neural network. By inputting the samples to be divided, our method can directly output the division results, and make the output results strictly maintain the following two characteristics: 1) The number of clusters required is the least; 2) All samples in a single cluster are self-consistent. The method includes the following steps:

[0054] Step 1: Data Representation

[0055] Original data tensorization: The main purpose of this step is to convert the original data such as pictures and texts into tensor form for subsequent training of the neural network.

[0056] For picture data: Convolutional neural network (CNN) can be used to extract picture features and generate corresponding tensor representations from them.

[0057] For text data: Pretrained models in natural language processing (such as BERT, GPT, etc.) can be used to extract text features and convert them into semantic vector or tensor representations. Specifically, for a piece of text, first perform tokenization on it, then convert the words into vector form through an embedding layer, and use a deep model to encode the context features of it, and finally obtain a high-dimensional tensor representation.

[0058] Of course, traditional feature extraction methods (such as TF-IDF, bag-of-words model, etc.) can also be used to represent text data in tensor form. In addition, other types of data (such as time series, structured data) can also be converted into tensor representations through specific preprocessing and feature extraction methods to ensure that various types of data can adapt to the input requirements of the neural network.

[0059] Normalize data: In order to ensure the balanced contribution of each feature to the clustering algorithm, it is necessary to standardize the data. There are multiple options for this step, and an appropriate standardization method can be selected according to the effect. For example:

[0060] 1-Minimum-maximum normalization: Scale the data to the interval [0,1]:

[0061]

[0062] where x min and x max are the minimum and maximum values of the tensorized sample representation x i respectively.

[0063] 2-Z-Score standardization: Convert the data into a distribution with a mean of 0 and a standard deviation of 1:

[0064]

[0065] where μ is the feature mean and σ is the feature standard deviation.

[0066] Through the above steps, the data is cleaned and unified in format, providing high-quality input for the dynamic optimal consistency clustering algorithm, thereby improving the robustness and accuracy of the algorithm.

[0067] Step 2: Information interaction

[0068] The information propagation mechanism can update the representation of the original sample by integrating the information of other samples, so that the updated sample representation contains richer context information. In this paper, the information interaction between samples is mainly realized through frameworks such as GNN (graph neural network) or Transformer.

[0069] In the case of having sample correlation data (such as the road map between cities): GNN is a more suitable choice because it efficiently models the relationships between nodes using the graph structure. Specifically, the sample representation is updated according to the following rules:

[0070]

[0071] where A is the normalized graph adjacency matrix, X (l) is the sample representation at the l-th layer, W (l) is the trainable weight matrix, and σ() is the activation function. Through multi-layer iteration, the samples can gradually fuse the information of neighboring nodes to form a more comprehensive representation.

[0072] In the case of lacking graph structure data:

[0073] The Transformer framework provides a flexible solution. Even if there is no explicit correlation data between samples, information interaction can still be achieved between any two samples through the attention mechanism. Its update rule is:

[0074]

[0075] where d k prevents the problem of dimensional growth caused by sample representations. The attention mechanism can dynamically calculate the similarity between samples and fuse information based on this, so that each sample can integrate the feature information of other samples.

[0076] Regardless of whether the GNN or the Transformer framework is adopted, the information interaction mechanism can enable each sample to obtain information from other samples, which is crucial for the clustering process because it can improve the quality of sample representations and the clustering effect.

[0077] Step 3: Partition learning

[0078] In the previous module, the samples obtained the feature information of other samples through the information propagation mechanism, thus forming rich representations. On this basis, the decision-making process of clustering is performed. This method combines Point Networks and reinforcement learning to complete the clustering task through a dynamic decision-making mechanism.

[0079] Application of Point Networks

[0080] The clustering task is essentially a global learning task, and its key steps include:

[0081] Global max pooling: Apply the max pooling operation to the features of each point to map the point cloud features into a global description:

[0082] g(x) = MaxPooling({f(x1), f(x2),..., f(x n})

[0083] Among them, f(x1) is the intermediate feature of the sample, and g(x) is the global feature description of the sample.

[0084] Classification of the global representation: Input the global feature into a multi-layer perceptron (MLP) for classification prediction:

[0085]

[0086] The clustering task is a global learning task. The first step is to map the point cloud features to a global description by applying the global max pooling operation to the features of each point. The second step is to input the global representation into a classification task. Different from previous clustering methods, this method does not use the classification step. Instead, it uses the idea of reinforcement learning to make the decision model more intelligent.

[0087] In the reinforcement learning method, specific actions need to be executed according to the global description. This method designs two main actions, namely: dividing into existing categories and forming new categories. To ensure that this method uses the fewest categories for division, a dynamic division mechanism is used. The process Figure 2 is shown as follows: For a current sample, first judge whether it can be divided into the original categories 1 and 2. If it can, then it is divided into these two categories, and then the next sample is predicted. If it cannot be divided into the original categories, then a new category is formed. In this case, a new category needs to be unlocked and added to the division of the next sample, as Figure 2 shown.

[0088] This operation is difficult to perform in a general classification network because the number of classification categories is inconsistent. A dynamic mask mechanism is designed to achieve this, and the formula is as follows:

[0089] Y = softmax{y i , y2, y new , MASK, MASK, MASK, MASK}

[0090] This mask can only be executed when the current prediction is y new to unlock a new category. This dynamic mask mechanism helps the model adapt to dynamic data and uses the fewest categories for division.

[0091] Penalty score: In the reinforcement model, the penalty score is important and can guide the model to learn intelligence in the set direction. In the method, two conditions need to be met: 1) achieving the division with the fewest number of groups, and 2) each group being self-consistent and conflict-free. After classifying all samples, an intermediate clustering result can be obtained. Based on this clustering result, a reward score can be calculated according to the following formula:

[0092]

[0093] where N represents the number of all samples, and S represents the number of used classes. Reinforcement learning learns for a higher reward score. First, it enables the model to learn self-consistent knowledge, and then gradually reduces the number of classes to correctly obtain a higher score. By setting such a reward function, the two preset goals can be achieved.

[0094] Step 4: Reinforcement learning

[0095] The core of the clustering task is to divide the samples into the fewest classes and ensure the consistency of the samples within each class. A dynamic division mechanism is designed and achieved through two main actions: dividing into existing classes and creating new classes;

[0096] Creating new classes:

[0097] (1) Dynamic masking mechanism

[0098] To handle the problem of uncertain dynamic class numbers, a dynamic masking mechanism is designed to unlock the creation of new classes. The formula of the masking mechanism is as follows:

[0099] Y = softmax{y i ,y2,y new ,MASK,MASK,MASK,MASK}

[0100] where y new represents the prediction score for creating a new class, and the mask (MASK) is only unlocked when selecting y new to dynamically add new classes. This mechanism enables the model to flexibly adapt to dynamic data and avoid over-division.

[0101] (2) Dynamic division process

[0102] For the current sample, the model first determines whether it can be classified into an existing class. If so, it is divided into one of the existing classes such as class 1, 2, etc.; if not, a new class is created and the sample is assigned to the new class. Through the dynamic division mechanism, the model realizes the clustering optimization with the minimum number of classes. The overall process schematic diagram is as Figure 2 shown:

[0103] (3) Design of Reward Mechanism

[0104] The reward mechanism is the core of the reinforcement learning module and is used to evaluate the quality of the current partitioning state. Its goal is to achieve self-consistency among samples with the fewest number of categories. The reward function is designed as follows:

[0105]

[0106] where N represents the total number of samples and S represents the number of categories used.

[0107] Internal Consistency Reward: If there are fewer conflicts among samples in the same category, a higher reward is given.

[0108] Category Quantity Penalty: Excessive numbers of partitioned categories will be penalized to encourage the model to reduce the number of categories.

[0109] (4) Optimization Process of Reinforcement Learning

[0110] Gradient updates utilize deep reinforcement learning frameworks (such as DQN, Policy Gradient) to convert the reward value into a gradient signal and optimize the model parameters. During the update process, the feature distribution of the samples and the current partitioning state are dynamically considered, thereby achieving the optimal partitioning at each step.

[0111] Dynamic Category Allocation: The model can flexibly decide whether to allocate to an existing category or create a new category based on the sample characteristics and historical partitioning state. This flexibility effectively avoids the problems of over-clustering or under-clustering.

[0112] Convergence and Robustness: Through multiple rounds of training, the model gradually converges to the optimal policy. To enhance the adaptability to abnormal data and distribution changes, a random perturbation mechanism is introduced to further improve the robustness of the model.

[0113] Step 5: Curriculum Learning

[0114] The above method can achieve partitioning samples into the fewest self-consistent categories. However, in cases where the number of samples is large or self-consistency judgment is difficult, the partitioning effect of the model may decline. This is because the model needs to first learn self-consistent knowledge and then learn how to complete the partitioning with the fewest number of categories. When the number of samples is excessive, the search space grows exponentially, increasing the difficulty of model learning. To solve this problem, a multi-stage training strategy based on curriculum learning is proposed.

[0115] Two Core Considerations in Curriculum Setting

[0116] The design of curriculum learning is based on the following two points:

[0117] (1) Gradual increase in the number of samples, initial stage: In the early stage of training, a small number of samples are used for training, enabling the model to focus on mastering basic knowledge and correctly completing the classification. Advanced stage: As the model's performance in the early courses stabilizes, the number of training samples is gradually increased. This step-by-step approach can effectively improve the model's generalization ability and avoid a sharp increase in the learning difficulty caused by an excessive amount of training data. Final stage: In the advanced courses, although the number of samples is larger and the complexity is higher, the model can more efficiently handle complex tasks because it already has the basic knowledge and reasoning ability.

[0118] (2) Gradual complication of the self-consistency condition

[0119] Self-consistency is a complex logical reasoning problem. In curriculum learning, the learning difficulty of the model is gradually increased by simplifying the conditions: Initial stage: Only consider a single self-consistency factor. For example, if there is a conflict between a certain factor A and -A in sample 1 and sample 2, it is determined that the two are not self-consistent and cannot be classified into the same category. Advanced stage: Gradually introduce more judgment conditions. For example, when considering factors A, B, and C in sample 1 and sample 2 simultaneously, the model needs to comprehensively judge the self-consistency of all factors. If some factors are not related (such as A and B) but do not conflict, the two can still be classified into the same category. Final stage: The model needs to handle more complex multi-factor reasoning problems to complete higher-level self-consistency detection.

[0120] (3) Dynamic curriculum adjustment

[0121] To enhance the model's adaptability, a dynamic curriculum learning mechanism is designed: Real-time adjustment of the curriculum: When the nature of the samples or the self-consistency conditions change, the curriculum design can be dynamically updated. Adaptive training: The model can quickly adapt to new data characteristics and self-consistency rules, thus achieving efficient learning under different data distributions and task conditions. Through curriculum learning, the model can more robustly learn self-consistency reasoning ability and optimal classification strategies during the training process of gradually increasing the task difficulty, effectively solving the learning bottleneck problem caused by a large number of samples and high complexity.

[0122] The system structure of the implementation method includes the following modules:

[0123] Data representation and preprocessing module: Clean, normalize, and extract features from the input data to generate an input format suitable for model processing, and the data is sent into the model in sequence to achieve real-time dynamic data processing.

[0124] Information interaction network: A neural network based on information propagation that updates the data representation using important information from other nodes, effectively capturing the interaction of relationships between data to make more accurate judgments.

[0125] Clustering Decision Network: Essentially a classification neural network, it tells the neural network the intermediate state of clustering and executes preset actions, such as classifying into existing categories and forming new categories, so as to achieve the purpose of clustering division.

[0126] Reinforcement Learning Module: Defines a reward mechanism to guide the model to optimize the clustering algorithm through trial and error. This process essentially updates the parameters of the neural network, making our model smarter.

[0127] Curriculum Learning Module: During the process of updating the model parameters, we utilize the concept of curriculum learning to train the model in segments. Let the model gradually transition from simple tasks to complex tasks, driving the model to learn useful knowledge faster and enabling the model to handle more complex tasks, ensuring that our model has stronger scalability.

[0128] This method has the following advantages:

[0129] Dynamic Adaptability: Data can be input into the model in a time series manner. Through the already divided data results, the cluster division is adjusted in real time to adapt to the dynamically changing data environment.

[0130] Clustering Optimization: Through reinforcement learning, the number of clusters is minimized while ensuring a high degree of consistency of the data within the clusters, avoiding internal conflicts, which is a core point of the invention.

[0131] High Efficiency: Utilizing curriculum learning, the model transitions from simple tasks to difficult tasks, thus supporting real-time data processing and incremental learning, significantly improving the efficiency of the algorithm and the ability to process large-scale data.

[0132] Versatility: Applicable to a variety of application scenarios, such as personalized recommendation, which can efficiently and dynamically cluster the user's interest points and adapt to the real-time changes of user behavior. For example, in an e-commerce platform, the algorithm can dynamically adjust the classification of recommended products and continuously optimize the recommendation results according to user feedback, thus achieving a highly personalized recommendation experience. In base station optimization, it can be used to dynamically cluster the location data of mobile users to analyze the user's behavior patterns. For example, by dynamically adjusting the base station coverage range or handover strategy, the resource allocation can be made more efficient, reducing congestion and resource waste, and improving the overall performance of the network.

[0133] Matters not covered in this invention are well-known technologies.

[0134] The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning, characterized in that: The method includes the following steps: Step 1: Tensorize the original data into tensor form and then perform normalization to obtain standardized data; Step 2: Implement information interaction between samples through a GNN (Graph Neural Network) or a Transformer framework; Step 3: To divide the samples into the fewest categories and ensure the consistency of samples within each category, a dynamic partitioning mechanism is set up for reinforcement learning; Step 4: Based on the training strategies of several stages of curriculum learning, first design the curriculum learning, and finally perform dynamic curriculum adjustment.

2. The dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning according to claim 1, wherein: The dynamic partitioning mechanism in Step 3 includes partitioning into existing categories, creating new categories, designing a reward mechanism, and optimizing reinforcement learning.

3. The dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning according to claim 2, characterized in that: The specific process of creating new categories is as follows: Set up a dynamic masking mechanism and a dynamic partitioning process: Dynamic masking mechanism: To handle the problem of uncertain dynamic category numbers, a dynamic masking mechanism is designed to unlock the creation of new categories. The formula of the masking mechanism is as follows: Y = softmax{y i , y2, y new , MASK, MASK, MASK, MASK,} Among them, y new represents the prediction score for creating a new category. The mask MASK is only unlocked when y new is selected and is used to dynamically add new categories, enabling the model to flexibly adapt to dynamic data and avoid over-partitioning; Dynamic partitioning process: For the current sample, the model first determines whether it can be classified into existing categories. If it can, it is classified into one of the existing categories such as Category 1 or Category 2. If it cannot be classified, a new category is created and the sample is assigned to the new category. Through the dynamic partitioning mechanism, the model achieves clustering optimization with the fewest category numbers.

4. The dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning according to claim 3, characterized in that: The specific process of designing the reward mechanism is as follows: The reward mechanism is the core of the reinforcement learning module, used to evaluate the quality of the current partitioning state. The goal is to achieve self-consistency among samples with the fewest category numbers. The reward function is designed as follows: where N represents the total number of samples and S represents the number of categories used. Internal consistency reward: If the conflicts among samples in the same category are less than the set value, a higher reward is given. Category number penalty: When the number of classified categories is more than the set value, a penalty will be imposed, encouraging the model to reduce the number of categories.

5. The dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning according to claim 3, characterized in that: The specific process of the dynamic partitioning process is as follows: Gradient update: Using the deep reinforcement learning framework, convert the reward value into a gradient signal to optimize the model parameters. During the update process, dynamically consider the feature distribution of samples and the current partitioning state, so as to achieve the optimal partitioning at each step; Dynamic category assignment: The model can flexibly decide whether to assign to existing categories or create new categories according to the sample characteristics and historical partitioning states, effectively avoiding the problems of over-clustering or under-clustering; Convergence and robustness: Through several rounds of training, the model gradually converges to the optimal strategy. To enhance the adaptability to abnormal data and distribution changes, a random perturbation mechanism is introduced to improve the robustness of the model.

6. The dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning according to claim 1, wherein: The specific process of designing the curriculum learning in Step 4 is as follows: Gradual increase in the number of samples: In the initial stage, in the early stage of training, use fewer samples than the set value for training, so that the model focuses on mastering basic knowledge and correctly completing the partitioning. Advanced stage: As the model's performance in the early curriculum tends to be stable, gradually increase the number of training samples. The gradual approach can effectively improve the generalization ability of the model and avoid a sharp increase in learning difficulty due to too much training data volume. Final stage: In the high-order curriculum, although the number of samples is larger and the complexity is higher, since the model already has basic knowledge and reasoning ability, it can handle complex tasks more efficiently; Gradual complication of self-consistency conditions: Self-consistency is a complex logical reasoning problem. In course learning, the learning difficulty of the model is gradually increased by simplifying the conditions. Initial stage: Only a single self-consistency factor is considered. If a certain factor A and -A in sample 1 and sample 2 conflict, it is determined that the two are not self-consistent and cannot be classified into the same category. Advanced stage: More judgment conditions are gradually introduced. Sample 1 and sample 2 consider factors A, B, and C at the same time. The model needs to comprehensively judge the self-consistency of all factors. If some factors are not related but do not conflict, the two are still classified into the same category. Final stage: The model needs to handle more complex factor reasoning problems to complete higher-level self-consistency detection.

7. The dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning according to claim 1, characterized in that: The specific process of dynamic course adjustment is as follows: To enhance the adaptability of the model, a dynamic course learning mechanism is designed to adjust the course in real time: When the sample nature or self-consistency conditions change, the course design can be dynamically updated. Adaptive training: The model can quickly adapt to new data characteristics and self-consistency rules, so as to achieve efficient learning under different data distributions and task conditions. Through course learning, the model can more robustly learn self-consistency reasoning ability and optimal partitioning strategy in the training process of gradually increasing task difficulty, effectively solving the learning bottleneck problem caused by a large number of samples and high complexity.

8. The dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning according to claim 1, characterized in that: The system for implementing this method includes a data representation and preprocessing module, an information interaction network, a clustering decision network, a reinforcement learning module, and a course learning module. The data representation and preprocessing module is connected to the clustering decision network through the information interaction network, and the clustering decision network is connected to the course learning module through the reinforcement learning module; The data representation and preprocessing module is used to clean, normalize, and extract features from the input data, generate an input format suitable for model processing, and the data is fed into the model in sequence to achieve real-time dynamic data processing; The information interaction network is used for a neural network based on information propagation, updating the data representation using important information from other nodes, effectively capturing the interaction of relationships between data, and thus making more accurate judgments; The clustering decision network is used to tell the neural network the intermediate state of clustering and execute preset actions, such as classifying into existing categories and forming new categories, so as to achieve the purpose of clustering division; The reinforcement learning module is used to define a reward mechanism, guide the model to optimize the clustering algorithm through trial and error, and update the parameters of the neural network, so as to make the model smarter; The course learning module is used to utilize the concept of course learning during the process of updating the model parameters, train the model in segments, let the model start from simple tasks and gradually transition to complex tasks, drive the model to learn useful knowledge faster, and also enable the model to handle more complex tasks, ensuring that the model has stronger scalability.

Images