Large-model multi-dimensional data filtering method and system based on adaptive reinforcement learning

Abstract

The invention discloses a large-model multi-dimensional data filtering method and system based on adaptive reinforcement learning. The method comprises the following steps: splitting a constructed composite data set into a plurality of single-dimensional data sets with different dimensions; pre-training each corresponding single-dimension screening model by using the single-dimension data set to generate a high-quality training set; using each pre-trained single-dimension screening model to respectively evaluate the single-dimension data set of the corresponding dimension, and generating each single-dimension score; performing linear weighted summation on each single-dimensional score to generate a comprehensive score; generating a screening data set through a screening strategy according to the comprehensive score; training a downstream model through a high-quality training set or a screening data set, and quantifying the performance gain obtained in the training process into an excitation signal; and updating the single-dimension screening model and the weight vector by using the excitation signal. According to the method, the problems that a filtering target of a data screening method is inconsistent with a final optimization target of downstream task performance, the evaluation dimension is single, and result fusion is rigid are solved.

Description

Technical Field

[0001] This invention relates to the fields of data processing and machine learning, and in particular to a method and system for filtering multi-dimensional data of large models based on adaptive reinforcement learning. Background Technology

[0002] With the widespread application of Large Language Models (LLMs) across various industries, the requirements for the scale and quality of training data are constantly increasing. Data augmentation techniques, as an effective solution, significantly improve the adaptability and coverage of models by generating diverse and large-scale training data. These techniques effectively alleviate the bottleneck of data scarcity in specific domains through various operations such as synonym rewriting, semantic transformation, and contextual expansion of existing data, and have demonstrated significant value in practical applications. However, data augmentation is often accompanied by problems such as noise introduction and semantic distortion, affecting data quality and thus limiting the performance improvement of downstream models. Therefore, how to effectively filter and optimize augmented data has become a crucial step in ensuring the performance of downstream tasks.

[0003] Data filtering is an important method for optimizing data augmentation effects. It improves the efficiency and effectiveness of model training by automatically identifying and selecting high-quality subsets of data within the augmented dataset. This process enhances the overall quality of the augmented corpus, ensuring improved performance for downstream tasks. Existing data filtering methods are mainly divided into four categories: rule-based filtering, manual detection, external model evaluation, and reinforcement learning-based adaptive filtering methods. However, these methods still have some significant shortcomings in practical applications.

[0004] First, the filtering objectives of existing methods are inconsistent with the ultimate optimization objectives of downstream tasks. Second, the evaluation dimensions are usually limited to a single aspect, failing to comprehensively grasp the multi-layered quality of the data. Furthermore, the result fusion mechanisms of existing methods are too rigid, making it difficult to adaptively adjust the importance of each dimension based on specific data. Summary of the Invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides a method and system for filtering large-scale multi-dimensional data based on adaptive reinforcement learning, which can solve the problems of inconsistency between the filtering target and the final optimization target of downstream task performance, single evaluation dimension, and rigid result fusion of existing data filtering methods.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] The first aspect is a large-scale, multi-dimensional data filtering method based on adaptive reinforcement learning, which includes the following steps:

[0008] S1. Construct a composite dataset; split the composite dataset into multiple single-dimensional datasets of different dimensions;

[0009] S2. Pre-train the corresponding single-dimensional screening models using the single-dimensional dataset to generate a high-quality training set;

[0010] S3. Use the pre-trained single-dimensional screening models to evaluate the single-dimensional datasets of the corresponding dimensions and generate scores for each single dimension; use the weight vector to linearly weight and sum the scores of each single dimension to generate a comprehensive score; generate a screening dataset based on the comprehensive score and the screening strategy.

[0011] S4. Train the downstream model using a high-quality training set or a selected dataset, and quantify the performance gain obtained during training into an activation signal; use the activation signal to update the single-dimensional selected model and weight vector.

[0012] Preferably, S1 includes:

[0013] S11. Collect expert-annotated data to generate an expert dataset; use a question-answering model to automatically generate a generated dataset in the form of question-answer triples based on the expert dataset; integrate the generated dataset and the expert dataset to generate a composite dataset;

[0014] S12. Divide the composite dataset into three single-dimensional datasets based on three dimensions: reasonableness of question content, consistency of question answers, and consistency of content answers; and randomize the single-dimensional datasets.

[0015] Preferably, S2 includes:

[0016] The model function scores each sample in the single-dimensional dataset, generating a selection probability value for the sample; based on the selection probability value, a binary selection decision is generated through Bernoulli sampling to determine whether a sample is selected; based on the selection results, the selected samples are extracted to construct a high-quality training set.

[0017] Preferably, S3 includes:

[0018] Based on the comprehensive score, a binary choice decision is generated by Bernoulli sampling to determine the selected state of the sample; the top 70% of the samples in the binary choice decision are extracted to form the screening dataset.

[0019] Preferably, S4 includes:

[0020] S41. Fine-tune the downstream model using a high-quality training set or a filtered dataset, and use a loss function to maximize the probability of accurately answering the question; after fine-tuning, calculate the performance gain of the downstream model.

[0021] S42. Convert the performance gain of the downstream model into an excitation signal;

[0022] S43. Pass the excitation signal to the single-dimensional screening model for training or adaptive weight learning.

[0023] As a preferred option, the negative log-likelihood loss function is used in S41. The downstream model parameters are optimized and defined as follows:

[0024]

[0025] in, For batch size, These represent the question, the content, and the answer, respectively. Indicates that in a given Under these conditions, the downstream model parameters are At that time, the answer is generated. The probability of.

[0026] Preferably, the performance difference of the downstream model before and after fine-tuning is converted into a scalar excitation signal. Used to guide data filtering strategies, defined as follows:

[0027] in, For the initial model performance, To fine-tune the model performance, For high-quality training sets or to select datasets, reward_fn is the reward function.

[0028] Preferably, S43 includes:

[0029] The REINFORCE algorithm is used to optimize the single-dimensional screening model. The objective function is defined as follows:

[0030]

[0031] Gradient calculation uses:

[0032]

[0033] Where E represents expectation, Expectation motivator Indicates the filtering strategy. Where S is the batch size, S represents the selected subset of samples, and D is the training dataset;

[0034] The weight vector is updated using a reinforcement learning mechanism, which is defined as follows:

[0035] .

[0036] in, Indicates weight, express gradient, It's the learning rate. It signifies a reward.

[0037] Secondly, a large-scale, multi-dimensional data filtering system based on adaptive reinforcement learning includes:

[0038] The data preparation module is used to construct composite datasets and split them into multiple single-dimensional datasets with different dimensions.

[0039] The single-dimensional screening model training module is used to pre-train each single-dimensional screening model using a composite dataset.

[0040] The adaptive weight training module uses pre-trained single-dimensional screening models to evaluate the corresponding single-dimensional datasets and generates the screening dataset based on the evaluation results.

[0041] The activation calculation module trains downstream models by filtering datasets and quantifies the performance gains obtained during training into activation signals; it then uses these activation signals to update the single-dimensional model selection and weight vector.

[0042] The large-scale multi-dimensional data filtering system based on adaptive reinforcement learning is used to implement the large-scale multi-dimensional data filtering method based on adaptive reinforcement learning as described in the first aspect.

[0043] Compared with the prior art, the beneficial effects of the present invention are reflected in:

[0044] 1. Unlike traditional technologies where the filtering target is disconnected from the downstream task, relies on static rules, or is not directly linked to task performance, this invention adopts a training mechanism based on adaptive reinforcement learning. It uses the improvement of downstream model performance as an incentive signal to directly align the optimization target of the filtering model with the performance of the downstream task. This allows the filtering process to adaptively serve the final task objective. Only a small amount of labeled data is needed to make the performance of the downstream model approach the training effect of the full dataset, which significantly improves training efficiency and filtering effectiveness.

[0045] 2. Unlike traditional technologies that rely on a single evaluation dimension, fixed weights for multi-dimensional fusion, and difficulty in comprehensively measuring data quality, this invention employs a multi-dimensional quality assessment and adaptive scoring fusion strategy. It conducts joint evaluation from three dimensions: the rationality of the question and content, the consistency between the question and the answer, and the consistency between the content and the answer. Through an adaptive weight module, it adaptively fuses the scores of each dimension, achieving a comprehensive and fine-grained evaluation of the intrinsic quality characteristics of the data. This enhances the adaptability, flexibility, and robustness of the filtering framework to different data distributions and task scenarios. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention. Detailed Implementation

[0048] To make the technical means, inventive features, objectives, and effects of the invention readily understandable, the invention is further described below with reference to specific illustrations. However, the invention is not limited to the embodiments described below.

[0049] It should be noted that the structures, proportions, sizes, etc., illustrated in the accompanying drawings of this specification are only used to complement the content disclosed in the specification for those skilled in the art to understand and read, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0050] Example 1:

[0051] like Figure 1 The method for filtering multi-dimensional data from large models based on adaptive reinforcement learning, as shown, includes the following steps:

[0052] S1. Construct a composite dataset; split the composite dataset into multiple single-dimensional datasets of different dimensions;

[0053] This step focuses on building a high-quality training dataset and ensuring the diversity and structure of data sources through systematic data integration and dimensionality partitioning. Specifically, it includes the following sub-steps:

[0054] S11. Collect expert-annotated data to generate an expert dataset; use a question-answering model to automatically generate a generated dataset in the form of question-answer triples based on the expert dataset; integrate the generated dataset and the expert dataset to generate a composite dataset;

[0055] This step constructs a composite data source through collaborative data augmentation and professional annotation. A lightweight question-answering model, fine-tuned based on expert data, automatically generates a semantically rich dataset. This generated dataset is then integrated with the expert dataset. This strategy significantly expands the semantic coverage of the training samples, effectively alleviating the cost pressure of manual annotation while maintaining the rigor of the specialized domain. Specifically, it includes:

[0056] Dataset generation: Synthetic question-answer triples are generated after fine-tuning the model using expert-annotated data. ,in To indicate a problem, Represents content, This represents the answer. The generated dataset is constructed from question-answer triples. The definition is as follows:

[0057]

[0058] Expert dataset: The basic expert dataset selected for this invention As shown in Table 1:

[0059] Table 1: Selected expert datasets

[0060]

[0061] expert dataset With generating datasets The complete composite dataset D is obtained by integration, defined as follows:

[0062]

[0063] S12. Divide the composite dataset into three single-dimensional datasets based on three dimensions: Question Content Reasonableness (QC), Question Answer Consistency (QA), and Content Answer Consistency (CA); and randomize the single-dimensional datasets.

[0064] This step splits the composite dataset into multiple single-dimensional datasets based on different evaluation dimensions. This structured transformation allows the features of each dimension to be represented independently, providing clear learning objectives for subsequent single-dimensional selection models. Specifically, this includes:

[0065] The composite dataset D is split into single-dimensional subsets based on three dimensions: Question-Content Appropriateness (QC), Question-Response Consistency (QA), and Content-Response Consistency (CA). The definition is as follows:

[0066]

[0067] Specifically, it includes:

[0068] The dataset is split into a question-content plausibility dimension. The pairs of questions and content are extracted from the dataset.

[0069]

[0070] The dataset is split into question-answer consistency dimensions. Question-answer pairings are extracted from the dataset.

[0071]

[0072] The dataset is split into a content-response consistency dimension. Pairs of content and responses are extracted from the dataset.

[0073]

[0074] Preprocess the single-dimensional dataset. Prepare normalized input for subsequent model training from the single-dimensional dataset obtained through the above splitting. (Right now , , The dataset of the target dimension is selected and randomized using random recombination techniques (such as the Fisher-Yates shuffle algorithm) to avoid quality assessment bias caused by the correlation of sample sequences.

[0075] S2. Use a single-dimensional filtering model to filter the corresponding single-dimensional dataset and generate a high-quality training set.

[0076] Adaptive single-dimensional data filtering model training. This step utilizes improved model performance to train a single-dimensional filtering model. The specific steps are as follows: This step optimizes filtering performance through reinforcement learning, ensuring that model filtering is directly related to improved question-answering performance.

[0077] First, the selection probability of a sample is calculated by the embedding model and used as a confidence index. Then, a random binary decision is made based on the confidence score to determine whether a sample is selected. After selection, a high-quality training dataset is formed. Next, the performance of the sample in the downstream model is used to calculate the incentive value to guide model optimization. Finally, reinforcement learning is used to update the data to maximize the expected incentive. This closed-loop process allows the model to dynamically adapt to data characteristics, improving selection accuracy and system reliability. Specifically, it includes:

[0078] S21. Data Scoring. This step uses a single dimension to screen the model for evaluating the quality of the data. Specifically, it involves using a model function. (e.g., BERT model) for each sample Scoring is performed, and the selection probability value of the sample is calculated, that is, the probability of being selected. The specific process is as follows:

[0079]

[0080] S22. Data Screening. This step is used to decide whether a sample should be selected for a high-quality dataset. Specifically, it involves Bernoulli sampling (…). Generate binary choice decisions Determine the selected sample status:

[0081]

[0082] S23. Constructing the Selected Dataset. This step involves building a high-quality training set based on the selection results. Specifically, the selected samples are extracted according to the aforementioned selection strategy to form a high-quality training set. .

[0083] S3. Adaptive Weight Learning Based on Reinforcement Learning. This stage aims to achieve uniformity and balance in data filtering by coordinating the weights of multiple single-dimensional filtering models. Specifically, it includes the following sub-steps:

[0084] S31, Single-Dimensional Scoring. In this step, the model for each dimension is independently evaluated on the dataset to provide a single-dimensional quality score. Specifically, this includes:

[0085] In the three dimensions of Question-Content Reasonableness (QC), Question-Response Consistency (QA), and Content-Response Consistency (CA), the corresponding screening models are used to evaluate the datasets for each dimension. The specific steps are as follows:

[0086] S311, Dataset Acquisition. Select the single-dimensional dataset corresponding to S12 based on the different training dimensions. , , .

[0087] S312, Dimensional Score Calculation. A pre-trained single-dimensional screening model is used to score each dimension of the dataset, providing foundational data for the subsequent weighted calculation process. Specifically, the single-dimensional screening model trained in S2 is used to score the three dimensions: Question and Content Appropriateness (QC), Question and Answer Consistency (QA), and Content and Answer Consistency (CA). The single-dimensional score calculation is defined as follows:

[0088]

[0089] Where k represents a specialized index, This represents a single-dimensional dataset corresponding to a specific index, with a pre-trained single-dimensional filtering model for each dimension. Provide specialized scoring .

[0090] S32. Weighted Screening. In this step, the invention adaptively adjusts the weights to integrate the scores of each dimension into a comprehensive quality index, ensuring that high-quality data is selected. Single-dimensional scores are linearly weighted and summed to form a comprehensive score, which then drives the screening decision. A probabilistic model is used to determine the data selection or rejection status, and the screening quality is guided by the importance of each dimension as indicated by the weights. Specifically, this includes:

[0091] S321. Comprehensive score calculation. Using weight vectors. , dimensional scoring Perform linear weighted summation to obtain the comprehensive score. The specific steps are as follows:

[0092]

[0093] Each dimension is scored. Specific scores for each dimension composition.

[0094] S322, Data Filtering. Based on comprehensive scoring. A selection and screening strategy is used to choose high-quality samples to form the final training set. This is achieved through Bernoulli sampling (…). Generate binary choice decisions Determine the selected sample status:

[0095]

[0096] S323. Construct a filtered dataset. Based on the filtering results... Constructing a high-quality training set. In this process, the present invention extracts... The first 70% of the samples form a high-quality sample set. .

[0097] S4. Adaptive Excitation Calculation. This part employs an adaptive excitation calculation method to improve downstream model performance and directly correlate it with excitation. First, the downstream model is trained using the dataset. Then, the performance gain obtained during training is quantified into an excitation signal to guide data and weight adjustments. Specifically, it includes the following sub-steps:

[0098] S41. Downstream Model Training. This step aims to fine-tune the downstream model using the dataset. The dataset is passed from other modules and uses a loss function to maximize the probability of accurately answering questions. After fine-tuning, the model's performance gain needs to be calculated.

[0099] S411, Model Fine-tuning. High-quality training sets from adaptive single-dimensional data passed from other modules are used to train the model. Or adaptive weight learning for selecting datasets The downstream model is then fine-tuned. A negative log-likelihood loss function is used during model training. Optimize model parameters The optimization objective is to maximize the probability of answering the question accurately. The process is defined as follows:

[0100]

[0101] in, For batch size, These represent the question, the content, and the answer, respectively. Indicates that in a given Under the condition that the model parameters are At that time, the answer is generated. The probability of.

[0102] S412. Performance Evaluation. After fine-tuning, evaluate the performance of the downstream model on the test set and calculate the performance gain to provide input for subsequent stimulus calculations.

[0103] S42, Activation Calculation. This step transforms the model's performance improvements into activation signals to optimize the screening and weight adjustment strategies. First, the performance difference before and after model fine-tuning is converted into a scalar activation signal. In this process, the activation signal is used not only to update the screening model but also to adjust the weights through reinforcement learning strategies.

[0104] S421, Excitation Calculation. This involves converting the performance difference of the model before and after fine-tuning into scalar excitation signals. This is used to guide data filtering strategies. It is defined as follows:

[0105] in, For the initial model performance, To fine-tune the model performance, High-quality training set after single-dimensional filtering Or filter the dataset reward_fn is the reward function.

[0106] S43, Model and Weight Update. The activation signal... Passed to the adaptive single-dimensional filtering model for training or adaptive weight learning, for subsequent updates.

[0107] S431. Single-dimensional screening model update: The Reinforce algorithm of reinforcement learning is used to update the screening model parameters.

[0108] Specifically, the REINFORCE algorithm is used to optimize the screening model, and the objective function is defined as follows:

[0109]

[0110] The gradient is calculated using:

[0111]

[0112] Where E represents expectation, Expectation motivator Indicates the filtering strategy. Here, S represents the batch size, S represents the selected subset of samples, and D represents the training dataset.

[0113] S432, Adaptive Weight Update. In this stage, the performance of the selected samples in the downstream task is evaluated, and incentive values ​​are generated to inspire model optimization. Incentives As a scaling factor in gradient ascent, it guides the adjustment of the weight vector to favor combinations of dimensions that improve performance. The process is defined as follows:

[0114] Update the weights. Optimize the screening strategy based on downstream performance feedback by adjusting the weights through reinforcement learning to improve strategy adaptability. During this process, the weights are normalized after adjustment to ensure model stability and initial effectiveness.

[0115] Weight adjustment. The weights are updated using a reinforcement learning mechanism to ensure adaptive optimization of the policy. The weight update mechanism is defined as follows:

[0116]

[0117] Learning rate and incentives Optimize the direction of the guiding weights.

[0118] Weight normalization. Normalization ensures the rationality and stability of the weights. This is achieved using the following formula:

[0119]

[0120] Optionally, to specifically illustrate the advantages of the present invention, this embodiment compares the present invention with mainstream methods on a dataset, and the results are shown in Table 2:

[0121] Table 2: Comparison results of the present invention with other methods

[0122]

[0123] Optionally, as can be seen from the table, the present invention demonstrates significant performance advantages on the three major datasets TriviaQA, NQ, and HotpotQA. Specifically, the F1 score on the TriviaQA dataset reaches 70.10 (approaching 99.6% of the performance on the full dataset), the EM score on the HotpotQA dataset is improved by 29.24%, and the F1 score of 73.23 on the NQ dataset surpasses all single-dimensional baseline methods.

[0124] Example 2:

[0125] A large-scale, multi-dimensional data filtering system based on adaptive reinforcement learning includes:

[0126] The data preparation module is used to construct composite datasets and split them into multiple single-dimensional datasets with different dimensions.

[0127] The single-dimensional screening model training module is used to pre-train each single-dimensional screening model using a composite dataset.

[0128] The adaptive weight training module uses pre-trained single-dimensional screening models to evaluate the corresponding single-dimensional datasets and generates the screening dataset based on the evaluation results.

[0129] The activation calculation module trains downstream models by filtering datasets and quantifies the performance gains obtained during training into activation signals; it then uses these activation signals to update the single-dimensional model selection and weight vector.

[0130] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments, including components, without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A method for filtering multi-dimensional data in large models based on adaptive reinforcement learning, characterized in that, Includes the following steps: S1. Construct a composite dataset; split the composite dataset into multiple single-dimensional datasets of different dimensions; S2. Pre-train the corresponding single-dimensional screening models using the single-dimensional dataset to generate a high-quality training set; S3. Use the pre-trained single-dimensional screening models to evaluate the single-dimensional datasets of the corresponding dimensions and generate scores for each single dimension. Use the weight vector to linearly weight and sum the scores of each single dimension to generate a comprehensive score. Based on the comprehensive score, a filtered dataset is generated using a filtering strategy. S4. Train the downstream model using a high-quality training set or a selected dataset, and quantify the performance gain obtained during training into an activation signal; use the activation signal to update the single-dimensional selected model and weight vector.

2. The method for filtering multi-dimensional data of large models based on adaptive reinforcement learning according to claim 1, characterized in that, S1 includes: S11. Collect expert annotation data to generate an expert dataset; A question-answering model is used to automatically generate a dataset in the form of question-answer triples based on an expert dataset; The generated dataset and the expert dataset are integrated to create a composite dataset; S12. Divide the composite dataset into three single-dimensional datasets based on three dimensions: reasonableness of question content, consistency of question answers, and consistency of content answers; and randomize the single-dimensional datasets.

3. The method for filtering multi-dimensional data of large models based on adaptive reinforcement learning according to claim 1, characterized in that, S2 include: The model function scores each sample in the single-dimensional dataset, generating a selection probability value for the sample; based on the selection probability value, a binary selection decision is generated through Bernoulli sampling to determine whether a sample is selected; based on the selection results, the selected samples are extracted to construct a high-quality training set.

4. The method for filtering multi-dimensional data of large models based on adaptive reinforcement learning according to claim 1, characterized in that, S3 include: Based on the comprehensive score, a binary choice decision is generated by Bernoulli sampling to determine the selected state of the sample; the top 70% of the samples in the binary choice decision are extracted to form the screening dataset.

5. The method for filtering multi-dimensional data of large models based on adaptive reinforcement learning according to claim 1, characterized in that, S4 include: S41. Fine-tune the downstream model using a high-quality training set or a filtered dataset, and use a loss function to maximize the probability of accurately answering the question; after fine-tuning, calculate the performance gain of the downstream model. S42. Convert the performance gain of the downstream model into an excitation signal; S43. Pass the excitation signal to the single-dimensional screening model for training or adaptive weight learning.

6. The method for filtering multi-dimensional data of large models based on adaptive reinforcement learning according to claim 1, characterized in that, S41 uses the negative log-likelihood loss function. The downstream model parameters are optimized and defined as follows: ； in, For batch size, These represent the question, the content, and the answer, respectively. Indicates that in a given Under these conditions, the downstream model parameters are At that time, the answer is generated. The probability of.

7. The method for filtering multi-dimensional data of large models based on adaptive reinforcement learning according to claim 1, characterized in that, The performance difference of the downstream model before and after fine-tuning is converted into a scalar excitation signal. Used to guide data filtering strategies, defined as follows: in, For the initial model performance, To fine-tune the model performance, For high-quality training sets or to select datasets, reward_fn is the reward function.

8. The method for filtering multi-dimensional data of large models based on adaptive reinforcement learning according to claim 1, characterized in that, S43 includes: The REINFORCE algorithm is used to optimize the single-dimensional screening model. The objective function is defined as follows: ； Gradient calculation uses: ； Where E represents expectation, Expectation motivator Indicates the filtering strategy. Where S is the batch size, S represents the selected subset of samples, and D is the training dataset; The weight vector is updated using a reinforcement learning mechanism, which is defined as follows: ； in, Indicates weight, express gradient, It's the learning rate. It signifies a reward.

9. A large-scale, multi-dimensional data filtering system based on adaptive reinforcement learning, characterized in that: include: The data preparation module is used to construct composite datasets and split them into multiple single-dimensional datasets with different dimensions. The single-dimensional screening model training module is used to pre-train each single-dimensional screening model using a composite dataset. The adaptive weight training module uses pre-trained single-dimensional screening models to evaluate the corresponding single-dimensional datasets and generates the screening dataset based on the evaluation results. The activation calculation module trains downstream models by filtering datasets and quantifies the performance gains obtained during training into activation signals; it then uses these activation signals to update the single-dimensional model selection and weight vector. The large-scale multi-dimensional data filtering system based on adaptive reinforcement learning is used to implement the large-scale multi-dimensional data filtering method based on adaptive reinforcement learning as described in claim 1.

Images