Data mining method and device based on large language model and reinforcement learning

Abstract

The invention provides a data mining method and device based on a large language model and reinforcement learning. The method comprises the following steps: receiving a data mining task demand input by a user; performing task decomposition on the data mining task demand based on a large language model to generate a task tree; controlling a large language model to select a current leaf node task from the task tree, and sending the current leaf node task to a reinforcement learning module; controlling a reinforcement learning module to execute the current leaf node task to obtain an execution result; calling a large language model to evaluate the execution result, judging whether the task is successful or not, taking the output of the current leaf node task as the input of the leaf node task selected next time when the task is successful, and selecting the leaf node task to be executed next time; and strategy adjustment is carried out when the task fails. According to the method, the data mining efficiency can be improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a data mining method and apparatus based on large-scale language models and reinforcement learning. Background Technology

[0002] In the power grid sector, data mining plays a crucial role. For example, by analyzing historical data, we can perform load forecasting, renewable energy generation forecasting, equipment fault diagnosis, and line loss analysis.

[0003] Currently, the implementation of these data mining tasks mainly relies on the following technical means: 1. Manual Modeling Process: This process is completed manually by domain experts. It typically includes: understanding business requirements, data preprocessing (cleaning, integration, transformation), feature engineering (feature extraction and selection), model selection, model training and hyperparameter tuning, and model evaluation and deployment. This process is highly dependent on the expert's experience, time-consuming, labor-intensive, and difficult to scale. 2. Automated Machine Learning (AutoML) Platforms: Platforms such as Google AutoML and H2O.ai can automatically perform feature selection, model selection, and hyperparameter tuning for a well-defined prediction task (such as classification or regression). For example, given a processed data table, AutoML can automatically find the best model to predict the values ​​in a certain column. However, these tools typically operate at the "atomic task" level, i.e., a well-defined modeling step. They cannot handle a complex chain of tasks requiring multiple nested data preprocessing, feature engineering, and model prediction. They solve the problem of "how to do a single step well," but not the problem of "how many steps should be done, and what are those steps?"

[0004] 3. Workflow orchestration tools: such as Apache Airflow, allow users to predefine multi-step data processing flows in the form of a directed acyclic graph (DAG). However, users need to pre-define the tasks of each step and their dependencies, and then the system executes them according to the plan.

[0005] 4. Large Language Model (LLM)-Assisted Programming: In recent years, large language models, represented by the GPT series, have been used to assist in generating code for data analysis. Users can describe their requirements using natural language, and the LLM generates corresponding Python or SQL code snippets, which are then manually integrated and executed by the user. However, in the LLM-assisted programming model, the LLM is responsible for generating code (planning), but it is unaware of the actual execution effect of the code. If the model training fails or the accuracy is insufficient, the LLM cannot obtain this feedback to correct its initial strategy. There is a lack of an effective closed-loop feedback mechanism between the planning layer (LLM) and the execution layer (model training). Patent text CN120106135A discloses a method, system, and electronic device for optimizing large language model instructions based on task decomposition. This method improves the understanding ability of large models by splitting graph reasoning tasks into multiple sub-tasks and designing independent instruction templates for each sub-task. However, this method still requires designing independent instruction templates for each sub-task, resulting in a complex process and low execution efficiency. Summary of the Invention

[0006] This invention provides a data mining method and apparatus based on large-scale language models and reinforcement learning, which can improve the efficiency of data mining.

[0007] A data mining method based on large language models and reinforcement learning includes: Receive data mining task requirements from user input; Based on a large language model, the data mining task requirements are decomposed to generate a task tree. The large language model is controlled to select the current leaf node task from the task tree and send the current leaf node task to the reinforcement learning module. The reinforcement learning module is controlled to execute the task of the current leaf node and obtain the execution result. The large language model is invoked to evaluate the execution result and determine whether the task is successful. If the task is successful, the output of the current leaf node task is used as the input of the next selected leaf node task, and the next leaf node task to be executed is selected. If the task fails, the strategy is adjusted.

[0008] Furthermore, the data mining task requirements are described using natural language.

[0009] Furthermore, based on a large-scale language model, the data mining task requirements are decomposed to generate a task tree, including: The data mining task requirements are analyzed based on the large language model, and knowledge related to the data mining task requirements is obtained from a preset knowledge base based on the analysis results. A root node is generated based on the data mining task requirements. Leaf node tasks are then generated by logically organizing the acquired knowledge from relevant domains, thus obtaining the task tree. Furthermore, the large language model is controlled to select the current leaf node task from the bottom up of the task tree, and the input and output of the current leaf node task are defined. The current leaf node task and its defined input and output are then sent to the reinforcement learning module.

[0010] Furthermore, controlling the reinforcement learning module to execute the task of the current leaf node includes: The control reinforcement learning module selects the appropriate model from the preset model tool library based on the current leaf node task; Obtain the corresponding model parameters from the preset knowledge base and load them into the model selected by the reinforcement learning module; The selected model and its corresponding model parameters are used as actions, and data related to the current leaf node task is obtained from the knowledge base. Based on the description of the current leaf node task and the data related to the current leaf node task, the reinforcement learning module generates the environment by observing the environment, executes the current leaf node task, and calculates the reward value. The model search and model parameter optimization are performed based on the reward value until the stopping condition is met.

[0011] Furthermore, model search and model parameter optimization are performed based on the reward value, including: The current state, the action performed, the reward obtained, and the next state observed after performing the current action are combined into an experience quadruple and stored in the experience replay buffer. The loss value is calculated by randomly sampling a preset number of empirical tuples from the empirical replay buffer. Backpropagation and gradient descent algorithms are used to update the reinforcement learning policy based on the loss value; The reinforcement learning module selects the next action based on the current state and the updated reinforcement learning strategy.

[0012] Furthermore, executing the task of the current leaf node also includes: calculating the corresponding model accuracy based on the output of the currently selected model; The stopping condition is that the accuracy of the current model is higher than a preset threshold, or the action is executed a preset number of times. The task is determined to be successful when the current model accuracy is higher than a preset threshold; the task is determined to be unsuccessful when the number of times the action is executed reaches a preset number and the accuracy of the selected models is lower than the preset threshold. The success or failure of the task, and the corresponding model output data, are fed back to the large language model as the execution result.

[0013] Furthermore, when a task fails, strategies are adjusted, including: The large language model generates adjustment instructions for the reinforcement learning module. The reinforcement learning module obtains a new data generation environment from a preset knowledge base according to the adjustment instructions, and then executes the current leaf node task, calculates the reward value, performs model search, and optimizes model parameters again.

[0014] Furthermore, when a task fails, strategies are adjusted, including: The task tree is restructured using the large language model.

[0015] A data mining device based on large language models and reinforcement learning includes: The receiving module is used to receive data mining task requirements input by the user. The scheduling module is used to decompose the data mining task requirements using a large language model, generate tasks, and control the large language model to select the current leaf node task from the task tree and send the current leaf node task to the reinforcement learning module. The execution module is used to control the reinforcement learning module to execute the task of the current leaf node and obtain the execution result; The adjustment module is used to call a large language model to evaluate the execution results and determine whether the task is successful. If the task is successful, the output of the current leaf node task is used as the input of the next selected leaf node task to select the next leaf node task to be executed; if the task fails, the strategy is adjusted.

[0016] Furthermore, the data mining task requirements are described using natural language.

[0017] Furthermore, the scheduling module utilizes a large language model to decompose the data mining task requirements and generate a task tree, including: The large-scale language model is used to parse the data mining task requirements, and knowledge related to the data mining task requirements is obtained from a preset knowledge base based on the parsing results. A root node is generated based on the data mining task requirements, and leaf node tasks are generated according to the logical relationships of the obtained knowledge in related fields to obtain the task tree.

[0018] Furthermore, the scheduling module controls the large language model to select the current leaf node task from the bottom up of the task tree, defines the input and output of the current leaf node task, and sends the current leaf node task and its defined input and output to the reinforcement learning module.

[0019] Furthermore, the execution module controls the reinforcement learning module to execute the task of the current leaf node, including: The control reinforcement learning module selects the appropriate model from the preset model tool library based on the current leaf node task; Obtain the corresponding model parameters from the preset knowledge base and load them into the model selected by the reinforcement learning module; The selected model and its corresponding model parameters are used as actions, and data related to the current leaf node task is obtained from the knowledge base. Based on the description of the current leaf node task and the data related to the current leaf node task, the reinforcement learning module generates the environment by observing the environment, executes the current leaf node task, and calculates the reward value. The model search and model parameter optimization are performed based on the reward value until the stopping condition is met.

[0020] Furthermore, the execution module controls the reinforcement learning module to perform model search and model parameter optimization based on the reward value, including: The current state, the action performed, the reward obtained, and the next state observed after performing the current action are combined into an experience tuple and stored in the experience replay buffer. The loss value is calculated by randomly sampling a preset number of empirical tuples from the empirical replay buffer. Backpropagation and gradient descent algorithms are used to update the reinforcement learning policy based on the loss value; The reinforcement learning module selects the next action based on the current state and the updated reinforcement learning strategy.

[0021] Furthermore, the execution module controls the reinforcement learning module to execute the current leaf node task, and also includes: calculating the corresponding model accuracy based on the output of the currently selected model; The stopping condition is that the accuracy of the current model is higher than a preset threshold, or the action is executed a preset number of times. The task is determined to be successful when the model accuracy of the current model is higher than a preset threshold; the task is determined to be unsuccessful when the number of times the action is executed reaches a preset number and the model accuracy of the selected models is lower than the preset threshold. The success or failure of the task, and the corresponding model output data, are fed back to the large language model as the execution result.

[0022] Furthermore, the adjustment module invokes the large language model to adjust the strategy when the task fails, including: The large language model generates adjustment instructions, which are then sent to the reinforcement learning module. The reinforcement learning module obtains a new data generation environment from a preset knowledge base based on the adjustment instructions, and then re-executes the current leaf node task, calculates the reward value, performs model search, and optimizes model parameters.

[0023] Furthermore, the adjustment module invokes a large language model to adjust the strategy when the task fails, including: The task tree is redesigned and modified using the large language model.

[0024] The data mining method and apparatus based on large-scale language models and reinforcement learning provided by this invention have at least the following beneficial effects: (1) It is proposed to use the semantic understanding and logical reasoning capabilities of large-scale language models to automatically decompose fuzzy, high-level business requirements into structured, executable multi-level task trees, and to dynamically adjust the planning based on execution feedback. A two-layer intelligent structure of large-scale language model and reinforcement learning is constructed. The upper-layer large-scale language model is responsible for macro planning (what to do and in what order), and the lower-layer reinforcement learning module is responsible for micro execution and optimization (how to do it specifically). The two are closely combined through a closed-loop feedback mechanism, which eliminates the need for complex modeling and improves data mining efficiency. (2) The reinforcement learning module models the execution process of the leaf nodes in the task tree as a reinforcement learning problem. Through exploration and learning, it adaptively searches for the optimal solution in the huge model library and hyperparameter space. Compared with traditional AutoML, it has stronger generalization and learning ability. (3) When any leaf node task fails or performs poorly, control is returned to the large language model for high-level decision-making, enabling workflow backtracking and reconstruction, which greatly enhances the robustness and problem-solving capabilities of the system. (4) Users do not need to be data mining experts. They only need to describe their final goal in natural language, and the system can automatically complete the entire process from data processing to model building, thus making data analysis capabilities more accessible to everyone. (5) Dynamic adaptation and backtracking capabilities enable the system to proactively seek alternative paths and avoid process interruptions when faced with complex realities such as poor data quality or unsuitable models. The combination of large-scale language models and reinforcement learning is more likely to discover innovative and efficient data processing links and model combinations than simply relying on expert experience or fixed processes; (6) The core framework of the method (large language model + reinforcement learning) is highly versatile and can be applied to any field that requires complex data mining, such as finance (e.g., credit risk control), healthcare (e.g., disease prediction), and manufacturing (e.g., quality control). Attached Figure Description

[0025] Figure 1 This is a flowchart of one embodiment of the data mining method based on large language models and reinforcement learning provided by the present invention.

[0026] Figure 2 A flowchart illustrating another embodiment of the data mining method based on large language models and reinforcement learning provided by the present invention.

[0027] Figure 3 This is a flowchart of one embodiment of the task decomposition method based on large language models and reinforcement learning provided by the present invention.

[0028] Figure 4 This is a flowchart illustrating one embodiment of the data mining method based on large language models and reinforcement learning provided by the present invention for performing a task.

[0029] Figure 5 This is a schematic diagram of one embodiment of the data mining device based on large language models and reinforcement learning provided by the present invention. Detailed Implementation

[0030] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0031] refer to Figure 1 and Figure 2 In some embodiments, a data mining method based on large language models and reinforcement learning is provided, including: S1. Receive data mining task requirements input by the user; S2. Based on a large-scale language model, the data mining task requirements are decomposed to generate a task tree; S3. Control the large language model to select the current leaf node task from the task tree and send the current leaf node task to the reinforcement learning module; S4. Control the reinforcement learning module to execute the task of the current leaf node and obtain the execution result; S5. Call the large language model to evaluate the execution result and determine whether the task is successful. If the task is successful, use the output of the current leaf node task as the input of the next selected leaf node task and select the next leaf node task to be executed. If the task fails, adjust the strategy.

[0032] Specifically, in step S1, the data mining task requirements input by the user can be described in natural language.

[0033] Further, refer to Figure 3 In step S2, the data mining task requirements are decomposed based on a large language model to generate a task tree, including: S21. Based on the large language model, the data mining task requirements are analyzed, and knowledge related to the data mining task requirements is obtained from a preset knowledge base according to the analysis results. S22. Generate a root node based on the data mining task requirements, and generate leaf node tasks according to the logical relationship of the obtained knowledge in the relevant fields to obtain the task tree.

[0034] Specifically, the pre-set knowledge base stores structured knowledge, rules, experience, code templates, model parameters, and raw data or metadata for data mining in the relevant fields, such as power grid topology data and data mining process paradigms.

[0035] For example, the received data mining task requirement is: predict the overall utilization rate of the power grid in region A next week.

[0036] A large-scale language model analyzes the data mining task requirements, retrieves domain knowledge related to power grid utilization from the database, decomposes the data mining task requirements, and generates a task tree. The root node of the task tree is to predict the overall utilization rate of the power grid in region A next week. The child nodes under the root node include: predicting total load and predicting total power generation. The grandchild nodes under the child node "predicting total power generation" include predicting photovoltaic power generation, predicting wind power generation, and predicting thermal power generation. The grandchild node "predicting photovoltaic power generation" further includes the great-grandchild node: predicting weather.

[0037] Further, in step S3, the large language module is controlled to select the current leaf node task from the bottom up of the task tree, define the input and output of the current leaf node task, and send the current leaf node task and its defined input and output to the reinforcement learning module.

[0038] Specifically, the large language module starts from the bottom of the task tree, selects a node as the current leaf node task, and defines its input and output. For example, for the current leaf node task of "predicting the weather", its input is defined as historical meteorological data and its output is "future weather sequence", which is then sent to the reinforcement learning module.

[0039] Further, refer to Figure 4 In step S4, controlling the reinforcement learning module to execute the current leaf node task includes: S41. The control reinforcement learning module selects the corresponding model from the preset model tool library according to the current leaf node task. S42. Obtain the corresponding model parameters from the preset knowledge base and load them into the model selected by the reinforcement learning module; S43. The selected model and its corresponding model parameters are taken as actions, and data related to the current leaf node task are obtained from the knowledge base. Based on the description of the current leaf node task and the data related to the current leaf node task, the reinforcement learning module generates the state by observing the environment, executes the current leaf node task, and calculates the reward value. S44. Perform model search and model parameter optimization based on the reward value until the stopping condition is met.

[0040] Specifically, the preset model tool library contains various pre-trained models, data processing tools, and algorithm libraries, such as ARIMA and LSTM. The preset knowledge base also stores the model parameters of the corresponding models.

[0041] The reinforcement learning module selects a corresponding model from a preset model tool library, obtains the corresponding model parameters from a preset knowledge base, uses the selected model and its corresponding model parameters as actions, and obtains data related to the current leaf node task from the knowledge base. Based on the description of the current leaf node task and the data generation environment related to the current leaf node task, the reinforcement learning module executes the current leaf node task by observing the generated state of the environment. Based on the model's output results, it calculates accuracy, execution time, etc., and obtains a reward value. Based on the reward value, it performs model search and model parameter optimization until the stopping condition is met.

[0042] In step S44, model search and model parameter optimization are performed based on the reward value, including: The current state, the action performed, the reward obtained, and the next state observed after performing the current action are combined into an experience tuple and stored in the experience replay buffer. The loss value is calculated by randomly sampling a preset number of empirical tuples from the empirical replay buffer. Backpropagation and gradient descent algorithms are used to update the reinforcement learning policy based on the loss value; The reinforcement learning module selects the next action based on the current state and the updated reinforcement learning strategy.

[0043] Specifically, in some embodiments, the current state can be the current neural network architecture, the action to be performed can be adding or modifying a layer in the current neural network architecture, the reward obtained can be the accuracy of the validation set, and the next state observed after performing the current action can be a new neural network architecture.

[0044] Updating the reinforcement learning policy can involve updating the parameters of the policy network and / or value network of the reinforcement learning module to optimize its decision-making strategy. The current state can also be the loss value / accuracy of the child node's data mining task.

[0045] Furthermore, executing the task of the current leaf node also includes: calculating the corresponding model accuracy based on the output of the currently selected model; The stopping condition is that the accuracy of the current model is higher than a preset threshold, or the action is executed a preset number of times. The task is determined to be successful when the model accuracy of the current model is higher than a preset threshold; the task is determined to be unsuccessful when the number of times the action is executed reaches a preset number and the model accuracy of the selected models is lower than the preset threshold. The success or failure of the task, and the corresponding model output data, are fed back to the large language model as the execution result.

[0046] Specifically, when a model's accuracy is found to be higher than a preset threshold, or when it still fails after a preset number of attempts, the final result (including whether the task is successful or failed, and the corresponding model output data) is fed back to the large language model.

[0047] The large language model is invoked to determine whether the task succeeded or failed based on the received execution result. If the task succeeds, the output of the current leaf node task is used as the input of the next leaf node, and the next leaf node (i.e., the parent node of the previous leaf node) is selected as the current leaf node task to continue execution until the root node task is completed. After the root node task is completed, the outputs of each model are integrated, and the final result is output.

[0048] For example, if the task of predicting the weather is successful, its output "future weather sequence" is used as the input of its parent node "predict photovoltaic power generation" to continue execution.

[0049] If the task fails, a large language model is invoked for backtracking analysis. The reason for the failure may be unreasonable task definition, missing key input data, etc.

[0050] Strategy adjustments when a task fails include: The large language model generates adjustment instructions, which are sent to the reinforcement learning module. The reinforcement learning module obtains a new data generation environment from a preset knowledge base according to the adjustment instructions, and then re-executes the current leaf node task, calculates the reward value, performs model search, and optimizes model parameters; or, the large language model is used to replan the task tree.

[0051] Specifically, the reinforcement learning module obtains a new data generation environment from a preset knowledge base according to the adjustment instruction, i.e., changes the environment, takes the selected model and its corresponding model parameters as actions, and obtains data related to the current leaf node task from the knowledge base. By observing the environment generation state, it executes the current leaf node task again and calculates the reward value. Based on the reward value, it performs model search and model parameter optimization until the stopping condition is met. That is, the current state, the executed action, the obtained reward, and the next state observed after executing the current action are combined into an experience tuple and stored in the experience replay buffer. A preset number of experience tuples are randomly sampled from the experience replay buffer to calculate the loss value. Backpropagation and gradient descent algorithms are used to update the reinforcement learning policy based on the loss value. The reinforcement learning module selects the next action based on the current state and the updated reinforcement learning policy.

[0052] Furthermore, large language models can be used to re-plan the task tree based on the tuning logs in the reinforcement learning module.

[0053] For example, for the leaf node task of predicting the weather, if the task fails, the reinforcement learning module is adjusted to retrieve new data from the preset knowledge base and execute the leaf node task again. If this task consistently fails, the large language model will restructure the task tree, adjust the leaf node tasks under "predicting total power generation," and increase the weight of thermal power prediction.

[0054] The method provided in this embodiment will be further explained through specific application scenarios below.

[0055] Application Scenario 1: Power Grid Load Forecasting.

[0056] The operator of a large photovoltaic power plant needs to accurately predict the power generation of its plant for the next 24 hours in order to optimize its bidding strategy in the electricity market and arrange its operation and maintenance plans. This is a typical complex forecasting task because photovoltaic output is not only a time series issue, but also highly dependent on a variety of dynamically changing meteorological factors.

[0057] The user entered: "Predict the power generation curve of the 'Sunshine One' photovoltaic power station every 15 minutes tomorrow." Task decomposition: After receiving the task, the large language model parses the requirements and, based on energy domain knowledge in a pre-set knowledge base, decomposes this single instruction into a task tree with inherent logical dependencies. Root task: Predict the power generation of photovoltaic power plants; Subtask 1 (Prerequisite Dependencies): Predict key weather factors. This task is further decomposed into parallel atomic tasks: Atomic Mission 1.1: Predict Global Horizontal Irradiance (GHI) over the site; Atomic Mission 1.2: Predicting Ambient Temperature; Atomic Task 1.3: Predicting Cloud Cover Changes.

[0058] Subtask 2: Construct a photovoltaic power output prediction model based on predicted meteorological data and historical power plant data.

[0059] The large language model starts from the bottom of the task tree, selects atomic task 1.1: predicting the global horizontal irradiance (GHI) over the field station, defines its input and output, and sends it to the reinforcement learning module.

[0060] The reinforcement learning module first tries a baseline approach: directly calling the publicly available Numerical Weather Prediction (NWP) API to obtain GHI forecast values, comparing them with historical data, and finding that this approach has a large error (and yields a low reward).

[0061] Subsequently, the reinforcement learning module performs another action: selecting the GradientBoosting model from the model toolkit and using historical measured GHI data from the power plant to correct the bias in the NWP prediction results. By continuously adjusting hyperparameters such as the model's learning rate and tree depth, the reinforcement learning module ultimately finds a model configuration that minimizes the corrected GHI prediction error (e.g., MAE), thus obtaining a high reward value.

[0062] During execution, if the high-precision local weather station data interface used to correct the model (the input to atomic task 1.1) is abnormally interrupted, the large language model will receive this "failure" feedback and backtrack. It will call the large language model to analyze the current situation and make a decision: to replan the task tree.

[0063] Large language models generate new instructions for reinforcement learning modules, such as: "Local data source failed, switch to backup: call satellite remote sensing cloud image data of the nearby area and build a GHI prediction model based on cloud image features."

[0064] Alternatively, if all high-precision data sources become unavailable, the large language model can initiate an interaction with the user: "The critical meteorological data source is interrupted, and the prediction accuracy will decrease. Should we continue to use the low-precision physical model for estimation and prediction?" This fully demonstrates the system's robustness and intelligent interactive capabilities.

[0065] Once all meteorological factor prediction tasks are successfully completed, their output (GHI, temperature, and other sequence data for the next 24 hours) will be used as input to subtask 2. The reinforcement learning module will then search for the optimal solution from LSTM, physical models, hybrid models, etc., for the task of "building a photovoltaic power output prediction model," ultimately completing the root task and outputting the power prediction curve.

[0066] Application Scenario 2: A telecommunications company wants to predict customers that may be lost in the next three months.

[0067] User input: "Find the customers most likely to churn in the next 3 months and their characteristics."

[0068] The task is broken down using a large language model into: 1) Data integration (merging user basic information, consumption records, and complaint records); 2) Data preprocessing and feature engineering (calculating derived features such as call duration and data usage change rate); 3) Building a binary classification model (predicting churn); 4) Model interpretation and outputting a list of high-risk customers.

[0069] When performing task 3), the reinforcement learning module may test various classifiers such as logistic regression, XGBoost, and neural networks, and automatically handle class imbalance problems (such as through upsampling or downsampling), and finally select the model with the highest F1-Score.

[0070] In different application scenarios, simply updating the knowledge data in the relevant domain in the knowledge base is sufficient to apply it to data mining tasks in different domains without changing the overall framework.

[0071] In some embodiments, reference Figure 5 It also provides a data mining apparatus based on a large language model and reinforcement learning, including: The receiving module 201 is used to receive data mining task requirements input by the user; The scheduling module 202 is used to decompose the data mining task requirements using a large language model, generate a task tree, and control the large language model to select the current leaf node task from the task tree and send the current leaf node task to the reinforcement learning module. Execution module 203 is used to control the reinforcement learning module to execute the task of the current leaf node and obtain the execution result; The adjustment module 204 is used to call the large language model to evaluate the execution result and determine whether the task is successful. When the task is successful, the output of the current leaf node task is used as the input of the next selected leaf node task, and the next leaf node task to be executed is selected. When the task fails, the strategy is adjusted.

[0072] Furthermore, the data mining task requirements are described using natural language.

[0073] Furthermore, the scheduling module 202 utilizes a large-scale language model to decompose the data mining task requirements and generate a task tree, including: The large language model is used to analyze the data mining task requirements, and knowledge related to the data mining task requirements is obtained from the preset knowledge base 204 based on the analysis results. A root node is generated based on the data mining task requirements, and leaf node tasks are generated according to the logical relationships of the obtained knowledge in related fields to obtain the task tree.

[0074] Specifically, the scheduling module 202 controls the interaction between the large language model and the knowledge base, which stores knowledge data in related fields, such as power grid topology data and data mining process paradigms.

[0075] Furthermore, the scheduling module 202 controls the large language model to select the current leaf node task from the bottom up of the task tree, defines the input and output of the current leaf node task, and sends the current leaf node task and its defined input and output to the reinforcement learning module.

[0076] Furthermore, the execution module 203 controls the reinforcement learning module to execute the current leaf node task, including: The control reinforcement learning module selects the appropriate model from the preset model tool library based on the current leaf node task; Obtain the corresponding model parameters from the preset knowledge base and load them into the model selected by the reinforcement learning module; The selected model and its corresponding model parameters are used as actions, and data related to the current leaf node task is obtained from the knowledge base. Based on the description of the current leaf node task and the data related to the current leaf node task, the reinforcement learning module generates the environment by observing the environment, executes the current leaf node task, and calculates the reward value. The model search and model parameter optimization are performed based on the reward value until the stopping condition is met.

[0077] The model tool library contains various pre-trained models, data processing tools, and algorithm libraries, such as ARIMA and LSTM.

[0078] Furthermore, the execution module 293 controls the reinforcement learning module to perform model search and model parameter optimization based on the reward value, including: The current state, the action performed, the reward obtained, and the next state observed after performing the current action are combined into an experience tuple and stored in the experience replay buffer. The loss value is calculated by randomly sampling a preset number of empirical tuples from the empirical replay buffer. Backpropagation and gradient descent algorithms are used to update the reinforcement learning policy based on the loss value; The reinforcement learning module selects the next action based on the current state and the updated reinforcement learning strategy.

[0079] Furthermore, the execution module controls the reinforcement learning module to execute the task of the current leaf node, and also includes: calculating the corresponding model accuracy based on the output of the currently selected model; The stopping condition is that the accuracy of the current model is higher than a preset threshold, or the action is executed a preset number of times. The task is determined to be successful when the model accuracy of the current model is higher than a preset threshold; the task is determined to be unsuccessful when the number of times the action is executed reaches a preset number and the model accuracy of the selected models is lower than the preset threshold. The success or failure of the task, and the corresponding model output data, are fed back to the large language model as the execution result.

[0080] Furthermore, the adjustment module 204 adjusts the strategy when the task fails, including: The adjustment module 204 uses a large language model to generate adjustment instructions to the reinforcement learning module. The reinforcement learning module obtains a new data generation environment from a preset knowledge base according to the adjustment instructions, and then executes the current leaf node task, calculates the reward value, performs model search, and optimizes model parameters again.

[0081] Furthermore, the adjustment module 204 adjusts the strategy when the task fails, including: The task tree is restructured using a large language model.

[0082] The data mining method and apparatus based on large-scale language models and reinforcement learning provided in the above embodiments have at least the following beneficial effects: (1) It is proposed to use the semantic understanding and logical reasoning capabilities of large language models to automatically decompose fuzzy, high-level business requirements into structured, executable multi-level task trees, and to dynamically adjust the plan based on execution feedback. A two-layer intelligent structure of large language model and reinforcement learning is constructed. The upper layer integrates the scheduling module of large language model to be responsible for macro strategic planning (what to do and in what order), while the lower layer reinforcement learning module is responsible for micro tactical execution and optimization (how to do it specifically). The two are closely combined through a closed-loop feedback mechanism to realize complex data mining tasks. (2) The reinforcement learning module models the execution process of the leaf nodes in the task tree as a reinforcement learning problem. Through exploration and learning, it adaptively searches for the optimal solution in the huge model library and hyperparameter space. Compared with traditional AutoML, it has stronger generalization and learning ability. (3) When any leaf node task fails or performs poorly, control is returned to the large language model for high-level decision-making, enabling workflow backtracking and reconstruction, which greatly enhances the robustness and problem-solving capabilities of the system. (4) Users do not need to be data mining experts. They only need to describe their final goal in natural language, and the system can automatically complete the entire process from data processing to model building, thus making data analysis capabilities more accessible to everyone. (5) Dynamic adaptation and backtracking capabilities enable the system to proactively seek alternative paths and avoid process interruptions when faced with complex realities such as poor data quality or unsuitable models. The combination of large-scale language models and reinforcement learning is more likely to discover innovative and efficient data processing links and model combinations than simply relying on expert experience or fixed processes; (6) The core framework of the method (large language model + reinforcement learning) is highly versatile and can be applied to any field that requires complex data mining, such as finance (e.g., credit risk control), healthcare (e.g., disease prediction), and manufacturing (e.g., quality control).

[0083] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A data mining method based on a large language model and reinforcement learning, characterized by, The method comprises the following steps: receiving a user input data mining task requirement; task decomposition based on a large language model to generate a task tree; controlling the large language model to select a current leaf node task from the task tree and sending the current leaf node task to a reinforcement learning module; controlling the reinforcement learning module to execute the current leaf node task and obtaining an execution result; calling the large language model to evaluate the execution result, determining whether the task is successful, and when the task is successful, taking the output of the current leaf node task as the input of the next selected leaf node task and selecting the next executed leaf node task; when the task fails, adjusting the strategy.

2. The method of claim 1, wherein, Task decomposition based on a large language model to generate a task tree, comprising: parsing the data mining task requirement based on the large language model, and obtaining knowledge related to the data mining task requirement from a pre-set knowledge base according to the parsing result; generating a root node based on the data mining task requirement, and generating leaf node tasks according to the logical relationship of the obtained related knowledge to obtain the task tree.

3. The method of claim 1, wherein, Controlling the large language model to select a current leaf node task from the bottom up in the task tree, and defining the input and output of the current leaf node task, and sending the current leaf node task and its defined input and output to the reinforcement learning module.

4. The method of claim 1, wherein, Controlling the reinforcement learning module to execute the current leaf node task, comprising: controlling the reinforcement learning module to select a corresponding model from a pre-set model tool library according to the current leaf node task; obtaining corresponding model parameters from a pre-set knowledge base and loading them into the model selected by the reinforcement learning module; selecting the model and corresponding model parameters as actions, and obtaining data related to the current leaf node task from the knowledge base, generating an environment according to the description of the current leaf node task and the data related to the current leaf node task, and the reinforcement learning module generates a state by observing the environment, executes the current leaf node task, and calculates a reward value; model search and model parameter optimization according to the reward value until a stop condition is met.

5. The method of claim 4, wherein, Model search and model parameter optimization according to the reward value, comprising: forming an experience tuple consisting of the current state, the executed action, the obtained reward, and the next state observed after executing the current action, and storing it in an experience replay buffer; randomly sampling a pre-set number of experience tuples from the experience replay buffer to calculate a loss value; updating the reinforcement learning strategy according to the loss value using backpropagation and gradient descent algorithm; the reinforcement learning module selects the next action based on the current state and the updated reinforcement learning strategy.

6. The method of claim 4, wherein, Executing the current leaf node task also includes calculating the corresponding model accuracy according to the output result of the currently selected model; the stop condition is that the model accuracy of the current model is higher than a pre-set threshold, or the execution of the action reaches a pre-set number of times; The task is determined to be successful when the model accuracy of the current model is higher than the preset threshold, and the task is determined to be failed when the number of times of performing the action reaches the preset number of times and the model accuracy of the selected model is less than the preset threshold; The task success or task failure and the corresponding model output data are fed back to the large language model as the execution result.

7. The method of claim 1, wherein, When the task fails, the strategy is adjusted, including: The large language model generates an adjustment instruction, which is sent to the reinforcement learning module. The reinforcement learning module obtains new data from a preset knowledge base to generate an environment, and executes the current leaf node task again, calculates the reward value, searches for a model, and optimizes the model parameters.

8. The method of claim 1, wherein, When the task fails, the strategy is adjusted, including: The large language model replans the task tree.

9. A data mining apparatus based on a large language model and reinforcement learning, characterized by, The method comprises the steps of: The receiving module receives the data mining task requirement input by the user; The scheduling module uses the large language model to perform task decomposition on the data mining task requirement, generates a task tree, and controls the large language model to select a current leaf node task from the task tree and send the current leaf node task to the reinforcement learning module; The execution module controls the reinforcement learning module to execute the current leaf node task and obtain an execution result; The adjustment module calls the large language model to evaluate the execution result, determines whether the task is successful, and, when the task is successful, takes the output of the current leaf node task as the input of the next selected leaf node task and selects the next executed leaf node task; when the task fails, the strategy is adjusted.

10. An electronic device, comprising: The processor and the memory are included, the memory stores a plurality of instructions, and the processor is used to read the instructions and execute the method in any one of claims 1-8.

Images