Course learning and reinforcement learning driven dynamic optimal self-consistent clustering method

Abstract

The application provides a course learning and reinforcement learning driven dynamic optimal self-consistent clustering method, and belongs to the technical field of big data processing.The method comprises the following steps: original data is zentified in a tensor form, and then normalized processing is performed to obtain standardized data; information interaction between samples is realized through a GNN graph neural network or a Transformer framework; a dynamic division mechanism is set for reinforcement learning to divide the samples into the least categories and ensure that the samples in each category are consistent; and a training strategy based on several stages of course learning is used, wherein course learning is designed first, and dynamic course adjustment is performed last.The algorithm can adjust clustering division in real time according to the dynamic change of a data set, does not need to be retrained or manually intervened, and is particularly suitable for dynamic scenes such as user behavior analysis and traffic flow prediction.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, and in particular to a dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning. Background Technology

[0002] With the widespread application of data-driven decision-making across various industries, the efficient processing of dynamic datasets has become a core challenge in today's information society. Clustering algorithms, as an important data analysis tool, are widely used in pattern recognition, data mining, and machine learning. Their main goal is to divide data into several groups (clusters), maximizing the similarity of data points within the same cluster and minimizing the similarity between different clusters. However, traditional clustering methods are primarily designed for static datasets. The current state of clustering algorithm technology can be summarized as follows:

[0003] Classic clustering algorithms, such as K-Means, Hierarchical Clustering, and Density Clustering (DBSCAN), are typically based on predefined distance metrics or density estimates and are suitable for datasets of fixed size and feature distribution.

[0004] Algorithm characteristics: These algorithms typically require pre-defining the number or density parameters of clusters and aggregating data points based on global features. However, the sensitivity of these algorithms to input parameters and their ability to process high-dimensional data limit their application scope.

[0005] Staticity Limitation: Traditional clustering methods primarily target static datasets, assuming all data points are available at the time of clustering and that the data distribution and characteristics remain unchanged during the clustering process. In practical applications, many datasets are dynamic, such as e-commerce user behavior data, network traffic logs, or medical sensor data. This data is constantly updated over time, making it difficult for traditional clustering methods to adapt to the incremental updates and real-time response requirements of dynamic data.

[0006] With the widespread application of data-driven decision-making across various industries, efficiently handling dynamically changing datasets has become a key challenge, and data partitioning based on data characteristics is a classic research problem. However, existing clustering methods mainly focus on static data partitioning, which has the following drawbacks: Staticity: Existing algorithms are mostly designed for static data and struggle to handle situations where data updates dynamically over time. Fixed Cluster Number: Existing methods typically require pre-setting the number of clusters before clustering, a manual design approach that lacks flexibility and can easily lead to inefficient resource utilization or loss. For example, in e-commerce platforms, too few clusters may result in users with different interests being incorrectly grouped, thus reducing the accuracy of recommendations; while too many clusters may lead to wasted and redundant computing resources. Self-Consistency Problem: Existing clustering methods cluster based on similarity, but fail to effectively resolve conflicts within categories, which can easily lead to serious consequences in critical fields such as healthcare. For instance, in the process of prescribing medication, if two drugs have conflicting properties and are included in the same prescription, it could have serious consequences for the patient's health or life. Efficiency limitations: Existing reinforcement learning-based clustering methods are often limited to small datasets, making it difficult to scale to larger datasets and lacking practicality. Furthermore, these methods lack real-time responsiveness. Therefore, there is a need to design dynamic, optimal, self-consistent clustering methods driven by curriculum learning and reinforcement learning. Summary of the Invention

[0007] The purpose of this invention is to provide a dynamic optimal self-consistent clustering method driven by course learning and reinforcement learning, thereby solving the technical problems of existing clustering methods mentioned in the background art.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning, the method comprising the following steps:

[0010] Step 1: Tensor the original data into tensor form, and then normalize it to obtain standardized data;

[0011] Step 2: Implement information interaction between samples using a GNN (Graph Neural Network) or the Transformer framework;

[0012] Step 3: To divide the samples into the fewest possible categories and ensure consistency among samples within each category, a dynamic partitioning mechanism is set up for reinforcement learning;

[0013] Step 4: Based on a multi-stage training strategy for course learning, firstly, the course learning is designed, and finally, the course is dynamically adjusted.

[0014] Furthermore, the dynamic partitioning mechanism in step 3 includes partitioning into existing categories, creating new categories, designing reward mechanisms, and optimizing through reinforcement learning.

[0015] Furthermore, the specific process for creating new categories involves setting up a dynamic masking mechanism and a dynamic partitioning process.

[0016] Dynamic masking mechanism: To handle the problem of an uncertain number of dynamic categories, a dynamic masking mechanism is designed to unlock the creation of new categories. The formula for the masking mechanism is as follows:

[0017] Y = softmax{y i y2,y new MASK, MASK, MASK, MASK

[0018] Among them, y new This indicates the predicted score for creating a new category; the mask MASK is only used when y is selected. new It is unlocked at certain times to dynamically add new categories, enabling the model to flexibly adapt to dynamic data and avoid over-segmentation;

[0019] Dynamic partitioning process: For the current sample, the model first determines whether it can be classified into an existing category. If it can, it is classified into one of the existing categories, such as category one or category two. If it cannot be classified, a new category is created and the sample is assigned to the new category. Through the dynamic partitioning mechanism, the model achieves clustering optimization with the minimum number of categories.

[0020] Furthermore, the specific process of designing the reward mechanism is as follows:

[0021] The reward mechanism is the core of the reinforcement learning module, used to evaluate the quality of the current partition state. The goal is to achieve consistency between samples using the minimum number of classes. The reward function is designed as follows:

[0022]

[0023] Where N represents the total number of samples, and S represents the number of categories used.

[0024] Internal consistency reward: If there are fewer conflicts between samples in the same category than a set value, a higher reward is given. Category number penalty: If the number of categories exceeds a set value, a penalty will be imposed to encourage the model to reduce the number of categories.

[0025] Furthermore, the specific process of dynamically dividing the process is as follows:

[0026] Gradient update utilizes a deep reinforcement learning framework to transform reward values ​​into gradient signals, optimize model parameters, and dynamically consider the feature distribution of samples and the current partitioning state during the update process, thereby achieving the optimal partitioning at each step.

[0027] Dynamic category assignment allows the model to flexibly decide whether to assign to an existing category or create a new category based on sample characteristics and historical partitioning status, effectively avoiding over-clustering or under-clustering problems.

[0028] Convergence and robustness: Through several rounds of training, the model gradually converges to the optimal policy. To enhance its adaptability to outlier data and distribution changes, a random perturbation mechanism is introduced to further improve the model's robustness.

[0029] Furthermore, the specific process of designing the course learning in step 4 is as follows:

[0030] Gradual increase in the number of samples: In the initial stage, in the early stages of training, fewer than the set number of samples are used for training, allowing the model to focus on mastering basic knowledge and correctly completing the partitioning. In the advanced stage, as the model's performance in the early courses tends to stabilize, the number of training samples is gradually increased. This step-by-step approach can effectively improve the model's generalization ability and avoid a sharp increase in learning difficulty due to excessive training data. In the final stage, in advanced courses, although the number of samples is larger and the complexity is higher, the model can handle complex tasks more efficiently because it already has basic knowledge and reasoning ability.

[0031] The gradual increase in the complexity of self-consistency conditions: Self-consistency is a complex logical reasoning problem. In the course, the learning difficulty of the model is gradually increased by simplifying the conditions. Initial stage: Only a single self-consistency factor is considered. If a factor A and -A in sample 1 and sample 2 conflict, then the two are judged to be inconsistent and cannot be classified into the same category. Advanced stage: More judgment conditions are gradually introduced. Factors A, B, and C are considered for both sample 1 and sample 2. The model needs to comprehensively judge the self-consistency of all factors. If some factors are unrelated but do not conflict, then the two are still classified into the same category. Final stage: The model needs to deal with more complex factor reasoning problems to complete a higher level of self-consistency detection.

[0032] Furthermore, the specific process for dynamic curriculum adjustment is as follows:

[0033] To enhance the model's adaptability, a dynamic curriculum learning mechanism is designed to adjust the curriculum in real time: when the properties of the samples or the self-consistency conditions change, the curriculum design can be dynamically updated for adaptive training: the model can quickly adapt to new data characteristics and self-consistency rules, thereby achieving efficient learning under different data distributions and task conditions. Through curriculum learning, the model can more robustly learn self-consistent reasoning ability and optimal partitioning strategy during the training process of gradually increasing task difficulty, effectively solving the learning bottleneck problem caused by a large number of samples and high complexity.

[0034] Furthermore, the system for implementing this method includes a data representation and preprocessing module, an information interaction network, a clustering decision network, a reinforcement learning module, and a curriculum learning module. The data representation and preprocessing module is connected to the clustering decision network via the information interaction network, and the clustering decision network is connected to the curriculum learning module via the reinforcement learning module.

[0035] The data representation and preprocessing module is used to clean, normalize, and extract features from the input data to generate an input format suitable for model processing. The data is fed into the model in time sequence, thereby enabling real-time dynamic data processing.

[0036] Information exchange networks are used in neural networks based on information propagation to update data representations using important information from other nodes, effectively capturing the interaction of relationships between data, thereby making more accurate judgments.

[0037] Clustering decision networks are used to inform neural networks of the intermediate states of clustering, enabling them to perform pre-defined actions, such as assigning clusters to existing categories or forming new categories, thereby achieving the purpose of clustering.

[0038] The reinforcement learning module is used to define the reward mechanism, guide the model to optimize the clustering algorithm through trial and error, update the parameters of the neural network, and thus make the model smarter.

[0039] The course learning module is used to train the model in stages during the process of updating model parameters. It utilizes the concept of course learning to gradually transition the model from simple tasks to complex tasks, driving the model to learn useful knowledge more quickly and enabling it to handle more complex tasks, thus ensuring that the model has stronger scalability.

[0040] The present invention, by adopting the above-described technical solution, has the following beneficial effects:

[0041] (1) The algorithm of this invention can adjust the clustering division in real time according to the dynamic changes of the dataset without retraining or manual intervention, and is particularly suitable for dynamic scenarios such as user behavior analysis and traffic flow prediction.

[0042] (2) Clustering optimization: By using reinforcement learning mechanism, the number of clusters can be dynamically adjusted, which can reduce redundancy, save resources, and ensure that the internal data of each cluster is highly consistent and avoid conflicts.

[0043] (3) High efficiency: It supports incremental learning and real-time processing, which greatly reduces the computational complexity of the algorithm, especially in solving the problem of reinforcement learning's difficulty in processing large-scale dynamic data.

[0044] (4) Versatility: Applicable to a variety of specific application scenarios:

[0045] Personalized recommendation system: Analyzes users' purchase history, browsing records, and behavioral data in real time, dynamically assigning users to suitable interest groups, thereby improving the accuracy of recommended products and user purchase conversion rates.

[0046] Base station optimization: By dynamically monitoring the geographical distribution and traffic demand of users, the coverage and signal strength configuration of base stations are optimized to reduce communication interruptions and resource waste.

[0047] Disaster emergency response: Real-time analysis of material needs and personnel distribution in disaster-stricken areas enables rapid allocation of relief supplies to the most needed locations, improving disaster relief efficiency.

[0048] Healthcare data management: dynamically monitor patients' health status and the distribution of medical resources, and optimize resource allocation and treatment plans.

[0049] Financial risk management: Real-time assessment of market volatility and customer behavior, grouping customers to adjust risk control strategies and reduce financial losses. Attached Figure Description

[0050] Figure 1 This is a flowchart of the method of the present invention;

[0051] Figure 2 This is the sample division diagram of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments. However, it should be noted that many details listed in the specification are merely to provide the reader with a thorough understanding of one or more aspects of the present invention, and these aspects of the invention can be implemented even without these specific details.

[0053] like Figure 1 As shown, this is a dynamic optimal self-consistent clustering method driven by course learning and reinforcement learning. This method utilizes an end-to-end neural network. Given samples to be partitioned as input, our method directly outputs the partitioning results, strictly maintaining the following two characteristics: 1) the number of clusters required is minimized; 2) all samples within a single cluster remain self-consistent. The method includes the following steps:

[0054] Step 1: Data Representation

[0055] Tensorization of raw data: The main purpose of this step is to convert raw data such as images and text into tensor form for subsequent training of neural networks.

[0056] For image data: Convolutional Neural Networks (CNNs) can be used to extract image features and generate corresponding tensor representations from them.

[0057] For textual data: Pre-trained models in natural language processing (such as BERT, GPT, etc.) can be used to extract textual features and transform them into semantic vectors or tensor representations. Specifically, for a piece of text, it is first tokenized, then the words are converted into vector form through an embedding layer, and the contextual features are encoded using a deep model to finally obtain a high-dimensional tensor representation.

[0058] Of course, traditional feature extraction methods (such as TF-IDF, bag-of-words model, etc.) can also be used to represent textual data as tensors. In addition, other types of data (such as time series and structured data) can also be converted into tensor representations through specific preprocessing and feature extraction methods to ensure that all types of data can be adapted to the input requirements of neural networks.

[0059] Normalizing data: To ensure a balanced contribution of each feature to the clustering algorithm, the data needs to be standardized. There are several options for this step; choose the appropriate standardization method based on the desired effect. For example:

[0060] 1-Min-Max Normalization: Scaling the data to the interval [0,1]:

[0061]

[0062] Where, x min and x max The sample representations x are respectively tensor quantization samples. i The minimum and maximum values.

[0063] 2-Z-Score standardization: Transforms the data into a distribution with a mean of 0 and a standard deviation of 1.

[0064]

[0065] Where μ is the characteristic mean and σ is the characteristic standard deviation.

[0066] Through the above steps, the data is cleaned and standardized, providing high-quality input for the dynamic optimal consistency clustering algorithm, thereby improving the algorithm's robustness and accuracy.

[0067] Step 2: Information Interaction

[0068] Information propagation mechanisms can update the representation of the original sample by integrating information from other samples, resulting in a more comprehensive and contextually richer representation. This paper primarily utilizes frameworks such as GNNs (Graph Neural Networks) or Transformers to implement information exchange between samples.

[0069] When sample association data is available (such as road maps between cities): GNN is a more suitable choice because it efficiently models the relationships between nodes using a graph structure. Specifically, the sample representation is updated according to the following rules:

[0070]

[0071] Where A is the normalized graph adjacency matrix, X (l) This is the representation of the l-th layer sample, W (l) It is a trainable weight matrix, and σ() is the activation function. Through multiple iterations, the samples can gradually fuse information from neighboring nodes to form a more comprehensive representation.

[0072] In the absence of graph-structured data:

[0073] The Transformer framework provides a flexible solution, allowing information exchange between any two samples through an attention mechanism even without explicit data correlation. Its update rule is as follows:

[0074]

[0075] Where d k To prevent the dimensionality growth problem caused by sample representation, the attention mechanism can dynamically calculate the similarity between samples and fuse information in a weighted manner, so that each sample can integrate the feature information of other samples.

[0076] Regardless of whether GNN or Transformer framework is used, the information exchange mechanism enables each sample to obtain information from other samples, which is crucial for the clustering process because it can improve the quality of sample representation and improve clustering results.

[0077] Step 3: Divide Learning

[0078] In the previous module, samples acquired feature information from other samples through an information propagation mechanism, thus forming a rich representation. Based on this, a clustering decision-making process is executed. This method combines Point Networks and reinforcement learning, using a dynamic decision-making mechanism to complete the clustering task.

[0079] Applications of Point Networks

[0080] Clustering is essentially a global learning task, and its key steps include:

[0081] Global max pooling: Applying max pooling to the features of each point maps the point cloud features to a global description.

[0082] g(x)=MaxPooling({f(x1), f(x2),..., f(x n})

[0083] Where f(x1) is the intermediate feature of the sample, and g(x) is the global feature description of the sample.

[0084] Global representation classification: Input global features into a multilayer perceptron (MLP) for classification prediction.

[0085]

[0086] Clustering is a global learning task. The first step is to apply global max pooling to the features of each point to map the point cloud features to a global description. The second step is to input the global representation into a classification task. Unlike previous clustering methods, this method does not use the classification step. It uses the idea of ​​reinforcement learning to make the decision model more intelligent.

[0087] In reinforcement learning methods, specific actions need to be performed based on the global description. This method designs two main actions: classifying into existing categories and forming new categories. To ensure that the method uses the fewest possible categories for classification, a dynamic classification mechanism is used, with the following process... Figure 2 As shown, the process is as follows: For the current sample, first determine whether it can be classified into the existing categories 1 and 2. If it can, classify it into these two categories, and then predict the next sample. If it cannot be classified into an existing category, a new category is formed. In this case, it is necessary to unlock a new category to add to the classification of the next sample, such as... Figure 2 As shown.

[0088] This operation is difficult to perform in general classification networks because the number of categories is inconsistent. Therefore, a dynamic mask mechanism is designed to achieve this, as shown in the following formula:

[0089] Y = softmax{y i y2,y new MASK, MASK, MASK, MASK

[0090] This mask is only valid when the current prediction is y. new This dynamic masking mechanism allows the model to adapt to dynamic data and make partitions using a minimal number of categories.

[0091] Penalty score: In reinforcement learning models, the penalty score is crucial, guiding the model to learn intelligence in a predetermined direction. The method requires two conditions to be met: 1) achieving the partitioning with the minimum number of groups, and 2) each group being self-consistent and conflict-free. After classifying all samples, an intermediate clustering result is obtained. Based on this clustering result, a reward score can be calculated using the following formula:

[0092]

[0093] Where N represents the total number of books in all samples, and S represents the number of classes used. Reinforcement learning aims to learn for higher reward scores. First, the model learns self-consistent knowledge, and then the number of classes is gradually reduced to correctly obtain a higher score. By setting such a reward function, two preset objectives can be achieved.

[0094] Step 4: Reinforcement Learning

[0095] The core of clustering is to divide samples into the fewest possible categories while ensuring consistency within each category. A dynamic partitioning mechanism is designed, achieved through two main actions: partitioning to existing categories and creating new categories.

[0096] Create a new category:

[0097] (1) Dynamic masking mechanism

[0098] To address the issue of an uncertain number of dynamic categories, a dynamic masking mechanism was designed to unlock the creation of new categories. The formula for the masking mechanism is as follows:

[0099] Y = softmax{y i y2,y new MASK, MASK, MASK, MASK

[0100] Among them, y new This indicates the predicted score for creating a new category; the mask (MASK) is only applied when y is selected. new It is unlocked at certain times to dynamically add new categories. This mechanism allows the model to flexibly adapt to dynamic data and avoid over-segmentation.

[0101] (2) Dynamic partitioning process

[0102] For the current sample, the model first determines whether it can be assigned to an existing category. If it can, it is assigned to one of the existing categories, such as category 1, category 2, etc.; if it cannot be assigned, a new category is created, and the sample is assigned to the new category. Through this dynamic partitioning mechanism, the model achieves clustering optimization with the minimum number of categories. The overall process diagram is shown below. Figure 2 As shown:

[0103] (3) Design of reward mechanism

[0104] The reward mechanism is the core of the reinforcement learning module, used to evaluate the quality of the current split state. Its goal is to achieve consistency between samples using the minimum number of classes. The reward function is designed as follows:

[0105]

[0106] Where N represents the total number of samples and S represents the number of categories used.

[0107] Internal consistency reward: A higher reward is given if there are few conflicts between samples in the same category.

[0108] Category count penalty: Models will be penalized for having too many categories, thus encouraging them to reduce the number of categories.

[0109] (4) Optimization process of reinforcement learning

[0110] Gradient updates utilize deep reinforcement learning frameworks (such as DQN and Policy Gradient) to transform reward values ​​into gradient signals, optimizing model parameters. During the update process, the feature distribution of samples and the current partitioning state are dynamically considered, thereby achieving optimal partitioning at each step.

[0111] Dynamic class assignment allows the model to flexibly decide whether to assign samples to existing classes or create new ones based on sample characteristics and historical partitioning status. This flexibility effectively avoids over-clustering or under-clustering problems.

[0112] Convergence and robustness: Through multiple rounds of training, the model gradually converges to the optimal policy. To enhance its adaptability to outlier data and distribution changes, a random perturbation mechanism is introduced to further improve the model's robustness.

[0113] Step 5: Course Learning

[0114] The above method can classify samples into the fewest self-consistent categories, but its effectiveness may decline when the number of samples is large or when self-consistency is difficult to determine. This is because the model needs to learn self-consistency knowledge first, and then learn how to complete the classification with the fewest categories. When the number of samples is too large, the search space grows exponentially, increasing the difficulty of model learning. To address this issue, a multi-stage training strategy based on curriculum learning is proposed.

[0115] Two core considerations in curriculum design

[0116] The course design is based on the following two points:

[0117] (1) Gradual increase in the number of samples: Initial stage: In the early stages of training, a small number of samples are used to allow the model to focus on mastering basic knowledge and correctly performing the partitioning. Advanced stage: As the model's performance in the early courses stabilizes, the number of training samples is gradually increased. This step-by-step approach can effectively improve the model's generalization ability and avoid a sharp increase in learning difficulty due to excessive training data. Final stage: In advanced courses, although the number of samples is larger and the complexity is higher, the model can handle complex tasks more efficiently because it already possesses basic knowledge and reasoning ability.

[0118] (2) The gradual increase in the complexity of self-consistency conditions

[0119] Self-consistency is a complex logical reasoning problem. In this course, the learning difficulty of the model is gradually increased by simplifying the conditions: Initial stage: Only a single self-consistency factor is considered. For example, if factor A and -A in sample 1 and sample 2 conflict, then they are determined to be inconsistent and cannot be classified into the same category. Advanced stage: More judgment conditions are gradually introduced. For example, factors A, B, and C are considered simultaneously for samples 1 and 2; the model needs to comprehensively judge the self-consistency of all factors. If some factors are unrelated (such as A and B) but do not conflict, then they can still be classified into the same category. Final stage: The model needs to handle more complex multi-factor reasoning problems to achieve a higher level of self-consistency detection.

[0120] (3) Dynamic curriculum adjustment

[0121] To enhance the model's adaptability, a dynamic curriculum learning mechanism was designed: Real-time curriculum adjustment: The curriculum design can be dynamically updated when sample properties or self-consistency conditions change. Adaptive training: The model can quickly adapt to new data characteristics and self-consistency rules, thus achieving efficient learning under different data distributions and task conditions. Through curriculum learning, the model can more robustly learn self-consistent reasoning ability and optimal partitioning strategies during training that gradually increases task difficulty, effectively solving the learning bottleneck problem caused by a large number of samples and high complexity.

[0122] The system architecture for implementing this method includes the following modules:

[0123] Data representation and preprocessing module: Cleans, normalizes and extracts features from input data to generate an input format suitable for model processing. The data is fed into the model in time sequence, thereby enabling real-time dynamic data processing.

[0124] Information interaction network: A neural network based on information propagation that uses important information from other nodes to update data representation, effectively capturing the interaction of relationships between data, thereby making more accurate judgments.

[0125] Clustering decision network: Essentially a classification neural network, it tells the neural network the intermediate state of clustering and executes preset actions, such as classifying into existing categories and forming new categories, thereby achieving the purpose of clustering.

[0126] Reinforcement learning module: Defines a reward mechanism to guide the model to optimize the clustering algorithm through trial and error. This process essentially updates the parameters of the neural network, making our model smarter.

[0127] Course Learning Module: During the model parameter update process, we utilize the concept of course learning to train the model in stages. This allows the model to gradually transition from simple tasks to complex ones, driving it to learn useful knowledge more quickly and enabling it to handle more complex tasks, thus ensuring our model has stronger scalability.

[0128] This method has the following advantages:

[0129] Dynamic adaptability: Data can be input into the model in a time sequence, and the cluster partitioning can be adjusted in real time based on the partitioned data results to adapt to the dynamically changing data environment.

[0130] Cluster optimization: Minimizing the number of clusters through reinforcement learning while ensuring high consistency of data within the clusters and avoiding internal conflicts is a core aspect of invention and innovation.

[0131] Efficiency: By leveraging course learning, the model can progress from simple tasks to difficult tasks, thereby supporting real-time data processing and incremental learning, significantly improving the efficiency of the algorithm and its ability to process large-scale data.

[0132] Versatility: Applicable to various application scenarios, such as personalized recommendations. It can efficiently and dynamically cluster user interests, adapting to real-time changes in user behavior. For example, in e-commerce platforms, the algorithm can dynamically adjust the classification of recommended products and continuously optimize the recommendation results based on user feedback, thereby achieving a highly personalized recommendation experience. In base station optimization, it can be used to dynamically cluster mobile user location data to analyze user behavior patterns. For example, by dynamically adjusting base station coverage or handover strategies, resource allocation becomes more efficient, reducing congestion and resource waste, and improving overall network performance.

[0133] Matters not covered in this invention are common knowledge.

[0134] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A dynamic optimal self-consistent clustering method driven by curriculum learning and reinforcement learning, characterized in that: The method comprises the following steps: Step 1: The original data tensor is converted into a tensor form, and then normalized to obtain standardized data, the original data including original picture and text data; Step 2: Information interaction between samples is realized through a GNN graph neural network or a Transformer framework; Step 3: The samples are divided into the least categories, and the dynamic division mechanism is set to realize reinforcement learning, ensuring the consistency of samples in each category; Step 4: Based on the training strategy of several stages of curriculum learning, first, the design of curriculum learning, and finally, dynamic curriculum adjustment; The dynamic division mechanism in step 3 includes division into existing categories, creation of new categories, design of reward mechanism and reinforcement learning optimization; The specific process of creating a new category is to set a dynamic mask mechanism and a dynamic division process: Dynamic mask mechanism: In order to deal with the problem of uncertain number of dynamic categories, a dynamic mask mechanism is designed to unlock the creation of new categories, and the formula of the mask mechanism is as follows: Y = softmax {y i , y2, y new , MASK, MASK, MASK, MASK} Wherein, y new Indicates the prediction score of creating a new category, and the mask MASK is only unlocked when y new Is selected, used for dynamically increasing new categories, so that the model can flexibly adapt to dynamic data and avoid over-partitioning; Dynamic division process: For the current sample, the model first judges whether it can be classified into an existing category, if it can, it is divided into one of the existing categories, if it cannot, a new category is created, and the sample is assigned to the new category. Through the dynamic division mechanism, the model realizes the clustering optimization of the minimum number of categories. 2.The course learning and reinforcement learning driven dynamic optimal self-consistent clustering method according to claim 1, characterized in that: The specific process of the design of the reward mechanism is as follows: The reward mechanism is the core of the reinforcement learning module, which is used to evaluate the pros and cons of the current division state, and the goal is to achieve self-consistency between samples with the least number of categories. The reward function is designed as follows: Where N represents the total number of samples, S represents the number of categories used, Internal consistency reward: if the conflict between samples in the same category is less than the set value, a higher reward is given, and the category number penalty: if the number of categories is too large, it will be punished, encouraging the model to reduce the number of categories.

3. The curriculum learning and reinforcement learning driven dynamic optimal self-consistent clustering method of claim 1, wherein: The specific process of the dynamic division process is as follows: Gradient update: using a deep reinforcement learning framework, the reward value is converted into a gradient signal to optimize the model parameters. During the update process, the feature distribution of the sample and the current division state are dynamically considered, so that the optimal division is realized at each step; Dynamic category allocation: the model can flexibly decide whether to allocate to an existing category or create a new category according to the sample characteristics and historical division state, effectively avoiding the problems of over-clustering or under-clustering; Convergence and robustness: through several rounds of training, the model gradually converges to the optimal strategy. In order to enhance the adaptability to abnormal data and distribution changes, a random disturbance mechanism is introduced to improve the robustness of the model.

4. The curriculum learning and reinforcement learning driven dynamic optimal self-consistent clustering method of claim 1, wherein: The specific process of the design of curriculum learning in step 4 is as follows: Step-by-step increase in sample size: In the initial stage, use fewer samples than the set value for training in the early stage of training, so that the model focuses on mastering the basics and completing the division correctly. In the advanced stage, gradually increase the number of training samples as the model's performance in the early courses tends to stabilize. This gradual approach can effectively improve the model's generalization ability and avoid the steep increase in learning difficulty caused by excessive training data. In the final stage, in the high-level courses, although the sample size is larger and the complexity is higher, the model has already acquired basic knowledge and reasoning ability, so it can handle complex tasks more efficiently. Step-by-step complexity of self-consistency conditions: Self-consistency is a complex logical reasoning problem. In course learning, the learning difficulty of the model is gradually increased by simplifying the conditions. In the initial stage, only a single self-consistency factor is considered. If a factor A and -A in sample 1 and sample 2 conflict, it is determined that the two cannot be divided into the same category. In the advanced stage, more judgment conditions are gradually introduced. Sample 1 and sample 2 consider factors A, B, and C. The model needs to judge the self-consistency of all factors. If some factors are not related but not conflicting, the two can still be divided into the same category. In the final stage, the model needs to handle more complex factor reasoning problems to complete higher-level self-consistency detection.

5. The curriculum learning and reinforcement learning driven dynamic optimal self-consistent clustering method of claim 1, wherein: The specific process of dynamic course adjustment is as follows: To enhance the adaptability of the model, a dynamic course learning mechanism is designed to adjust the course in real time: when the sample properties or self-consistency conditions change, the course design can be dynamically updated. The model can quickly adapt to new data characteristics and self-consistency rules, enabling efficient learning under different data distributions and task conditions. Through course learning, the model can more stably learn self-consistency reasoning ability and optimal division strategy in the process of gradually increasing task difficulty, effectively solving the learning bottleneck problem caused by large sample size and high complexity.

6. The curriculum learning and reinforcement learning driven dynamic optimal self-consistent clustering method of claim 1, wherein: The system implementing this method includes a data representation and preprocessing module, an information interaction network, a clustering decision network, a reinforcement learning module, and a course learning module. The data representation and preprocessing module is connected to the clustering decision network through the information interaction network. The clustering decision network is connected to the course learning module through the reinforcement learning module. The data representation and preprocessing module is used to clean, normalize, and extract features from input data, generating an input format suitable for model processing. The data is sent to the model in real time, enabling real-time dynamic data processing. The information interaction network is a neural network based on information propagation, which updates the data representation using important information from other nodes, effectively capturing the interaction between data relationships to make more accurate judgments. The clustering decision network is used to inform the neural network of the intermediate state of clustering and execute pre-set actions, including division into existing categories and formation of new categories, to achieve the purpose of clustering division. The reinforcement learning module is used to define a reward mechanism to guide the model to optimize the clustering algorithm through trial and error, updating the parameters of the neural network to make the model smarter. The course learning module is used in the process of updating the model parameters, uses the concept of course learning, trains the model in sections, makes the model start from a simple task and gradually transits to a complex task, drives the model to learn useful knowledge faster, and also enables the model to process more complex tasks, and ensures that the model has stronger expansion capability.

Images