Charging station management system intelligent question and answer method, electronic equipment and storage medium
Through the Agent-RAG fusion architecture and the charging station management system with reinforcement learning fine-tuning inference model, the shortcomings of the traditional manual customer service model are solved, and automated Q&A with fast response and stable service are achieved, operating costs are reduced, and the economic benefits of the charging station are improved.
Patent Information
- Application Number
- CN202510526814.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-25
- Publication Date
- 2025-08-08
AI Technical Summary
The traditional manual customer service model of charging stations has slow response speed, unstable service quality, and is difficult to achieve all-weather support. It also increases operating costs and reduces the economic benefits of charging stations.
The charging station management system adopts the Agent-RAG fusion architecture and the reinforcement learning fine-tuning inference model to achieve high-quality automated question-and-answer services by obtaining user input questions, determining task attributes, and generating answers based on RAG and reinforcement learning.
It realizes all-weather support with fast response and stable service quality, reduces operating costs, improves user satisfaction and economic benefits of charging stations.
Smart Images
Figure CN120448603A_ABST
Abstract
Description
Technical field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intelligent question-answering method, electronic equipment, and storage medium for a charging station management system. [Background Technology]
[0002] With the explosive growth of the electric vehicle industry, electric vehicle charging services have become a core link in the new energy transportation ecosystem.
[0003] Users will encounter a variety of problems during the charging process, such as charging equipment failure, billing questions, charging site inquiries, charging policy consultations, etc. The diversity and professionalism of these needs pose extremely high challenges to the customer service system.
[0004] At present, the customer service system of charging stations still uses the traditional manual customer service model. This traditional manual customer service model has revealed many pain points in the face of an increasingly large user base, such as slow response speed, unstable service quality, difficulty in achieving all-weather support, and increased operating costs of charging stations, reducing the overall economic benefits of charging stations. [Summary of the invention]
[0005] The embodiments of the present invention aim to provide an intelligent question-and-answer method, electronic device, and storage medium for a charging station management system, aiming to solve the problems of slow response, unstable service quality, difficulty in achieving all-weather support, and increased operating costs of charging stations caused by the traditional manual customer service model currently used in charging stations.
[0006] To solve the above technical problems, the first embodiment of the present invention provides an intelligent question-answering method for a charging station management system, comprising:
[0007] Get the charging problem input by the user;
[0008] The agent based on the Agent-RAG fusion architecture determines the task attributes of the charging problem input by the user;
[0009] According to the task attributes of the charging problem, the RAG and reinforcement learning fine-tuning reasoning large model based on the Agent-RAG fusion architecture generates a charging problem answer that matches the task attributes of the charging problem;
[0010] The generated answer to the charging question is delivered to the user.
[0011] Optionally, the task attributes of the charging problem are divided by the Agent into basic information query tasks, complex decision support tasks and analysis and calculation tasks according to the complexity of the charging problem.
[0012] Optionally, in the Agent-RAG fusion architecture, RAG provides real-time data to support Agent's decision-making by retrieving the RAG knowledge base, wherein the data in the RAG knowledge base is divided into two categories: structured data and unstructured data. The structured data includes charging pile equipment specification data and pricing rule data, and the unstructured data includes user feedback, technical documents, regulations and policies, and market trend analysis.
[0013] Optionally, the unstructured data is managed using a hierarchical storage structure combined with a tree index method, including: the unstructured data adopts a three-level classification system, and a data hierarchy table is designed in the RAG database to store the hierarchical relationship of the data, and a kd tree is used to establish a tree index mechanism.
[0014] Optionally, generating a charging problem answer that matches the task attributes of the charging problem by fine-tuning the RAG and reinforcement learning reasoning large model based on the Agent-RAG fusion architecture includes:
[0015] If the task attribute of the charging problem is determined to be a basic information query task, the agent directly calls the database retrieval tool to obtain the answer from the RAG knowledge base.
[0016] Optionally, generating a charging problem answer that matches the task attributes of the charging problem by fine-tuning the RAG and reinforcement learning reasoning large model based on the Agent-RAG fusion architecture includes:
[0017] If the task attribute of the charging problem is determined to be a complex decision support task, the complex decision support task is broken down into a set of subtasks. Each subtask agent calls a database retrieval tool to complete information retrieval in its respective field, integrates the acquired knowledge resources, and then uses a trained reinforcement learning fine-tuned reasoning model to perform reasoning and analysis to generate a professional answer to the charging problem.
[0018] Optionally, generating a charging problem answer that matches the task attributes of the charging problem by fine-tuning the RAG and reinforcement learning reasoning large model based on the Agent-RAG fusion architecture includes:
[0019] If the task attribute of the charging problem is determined to be an analytical computing task, the agent first retrieves relevant information as key context, and then uses the trained reinforcement learning fine-tuned reasoning model to perform reasoning and analysis to generate a professional answer.
[0020] Optionally, the reinforcement learning fine-tuning reasoning large model is obtained by introducing a group relative strategy to optimize the reinforcement learning strategy, by constructing high-quality question-answer pairs, selecting DeepSeek-R1 as the basic model, designing a reward function, and adopting a training method based on group relative strategy optimization through multiple rounds of iterative training to continuously optimize the reasoning ability.
[0021] Accordingly, an embodiment of the second aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and running on the processor. When the computer program is executed by the processor, the intelligent question and answer method for the charging station management system described in the embodiment of the first aspect of the present invention is implemented.
[0022] Accordingly, an embodiment of the third aspect of the present invention provides a storage medium, on which is stored a program of an intelligent question-and-answer method for a charging station management system. When the program of the intelligent question-and-answer method for a charging station management system is executed by a processor, the intelligent question-and-answer method for a charging station management system described in the embodiment of the first aspect of the present invention is implemented.
[0023] Compared to the prior art, embodiments of the present invention provide an intelligent question-and-answer method, electronic device, and storage medium for a charging station management system. A method for intelligent pricing of charging station operations is applied to an operations server. The intelligent question-and-answer method for a charging station management system includes: obtaining a charging question input by a user; determining the task attributes of the charging question input by an agent based on an agent-RAG fusion architecture; generating, based on the task attributes of the charging question, an answer matching the task attributes of the charging question using the agent-RAG fusion architecture's RAG and a reinforcement learning fine-tuned reasoning macro model; and delivering the generated charging question answer to the user. By employing the agent-RAG fusion architecture to process charging questions of varying complexity input by users, and combining it with a reinforcement learning fine-tuned reasoning macro model to generate the answer, and then delivering the generated charging question answer to the user, high-quality automated question-and-answer services are implemented with fast response times, stable service quality, and 24 / 7 support, thereby improving user satisfaction. Furthermore, the method eliminates the need for manual service, significantly reducing charging station operating costs and improving the economic benefits of charging stations. This can solve the problems of slow response, unstable service quality, difficulty in achieving all-weather support, and increased operating costs of charging stations caused by the traditional manual customer service model currently used by charging stations.
Brief Description of the Drawings
[0024] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0025] Figure 1 This is a flow chart of an intelligent question-answering method for a charging station management system provided by the present invention;
[0026] Figure 2 This is a detailed flowchart of an intelligent question-answering method for a charging station management system provided by the present invention;
[0027] Figure 3 This is a schematic diagram of the specific training process of the reinforcement learning fine-tuning reasoning large model in the intelligent question-answering method for the charging station management system provided by the present invention;
[0028] Figure 4 It is a structural schematic diagram of an electronic device provided by the present invention. [Specific implementation method]
[0029] In order to facilitate the understanding of the present invention, the present invention will be described in more detail below with reference to the accompanying drawings and specific embodiments. It should be noted that when an element is described as being "fixed to" another element, it can be directly on the other element, or there can be one or more centered elements therebetween. When an element is described as being "electrically connected" to another element, it can be directly connected to the other element, or there can be one or more centered elements therebetween. The orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "bottom" etc. used in this specification is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0030] Unless otherwise defined, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which this invention pertains. The terms used in this specification are intended only to describe specific embodiments and are not intended to limit the invention. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.
[0031] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0032] With the explosive growth of the electric vehicle industry, electric vehicle charging services have become a core link in the new energy transportation ecosystem.
[0033] Users will encounter a variety of problems during the charging process, such as charging equipment failure, billing questions, charging site inquiries, charging policy consultations, etc. The diversity and professionalism of these needs pose extremely high challenges to the customer service system.
[0034] At present, the customer service system of charging stations still uses the traditional manual customer service model. This traditional manual customer service model has revealed many pain points in the face of an increasingly large user base, such as slow response speed, unstable service quality, difficulty in achieving all-weather support, and increased operating costs of charging stations, reducing the overall economic benefits of charging stations.
[0035] To address this industry challenge, the present invention provides an intelligent question-answering method for a Charging Station Management System (CSMS) based on an Agent-RAG fusion architecture and a large-scale reinforcement learning fine-tuned inference model. This method employs the Agent-RAG fusion architecture to address charging industry issues of varying complexity. It also incorporates hierarchical storage and tree indexing technologies to efficiently manage unstructured data, and fine-tunes the large-scale model using a Group Relative Policy Optimization (GRPO) reinforcement learning strategy. This method provides high-quality automated question-answering services while significantly reducing operating costs.
[0036] In one embodiment, Figure 1 As shown, the present invention provides an intelligent question-answering method for a charging station management system, based on an Agent-RAG fusion architecture and a reinforcement learning fine-tuning reasoning model, including:
[0037] S1. Obtain charging problem input by the user;
[0038] S2, the agent based on the Agent-RAG fusion architecture determines the task attributes of the charging problem input by the user;
[0039] S3. Based on the task attributes of the charging problem, the RAG and reinforcement learning fine-tuned reasoning model based on the Agent-RAG fusion architecture generates a charging problem answer that matches the task attributes of the charging problem;
[0040] S4. Deliver the generated answer to the charging question to the user.
[0041] In this embodiment, a smart question-answering method for a charging station management system based on an agent-RAG fusion architecture and a reinforcement learning fine-tuned reasoning model is provided. The method includes: obtaining a charging question input by a user; determining the task attributes of the charging question input by an agent based on the agent-RAG fusion architecture; generating a charging question answer that matches the task attributes of the charging question based on the agent-RAG fusion architecture's RAG and the reinforcement learning fine-tuned reasoning model; and delivering the generated charging question answer to the user. By utilizing the agent-RAG fusion architecture to process charging questions of varying complexity input by users, and combining it with the reinforcement learning fine-tuned reasoning model to generate the charging question answer, and then delivering the generated charging question answer to the user, a high-quality automated question-answering service is achieved with fast response times, stable service quality, 24 / 7 support, and improved user satisfaction. Furthermore, by eliminating the need for manual service, the operating costs of charging stations can be significantly reduced, and the economic benefits of charging stations can be improved. This addresses the current issues of charging stations using traditional manual customer service models, which result in slow response times, unstable service quality, difficulty in achieving 24 / 7 support, and increased operating costs.
[0042] In one embodiment, in step S2, the agent based on the Agent-RAG fusion architecture determines the task attributes of the charging problem input by the user, wherein the task attributes of the charging problem are divided by the agent into basic information query tasks, complex decision support tasks, and analysis and calculation tasks according to the complexity of the charging problem.
[0043] In the present invention, the Agent-RAG (retrieval-augmented generation) fusion architecture is a RAG (Retrieval-Augmented Generation) architecture with Agent (agent, or intelligent agent) as the entrance. It combines the decision-making ability of Agent with the dynamic knowledge retrieval ability of RAG to build a more flexible, autonomous and reliable AI (Artificial Intelligence) system.
[0044] In the Agent-RAG fusion architecture, agents possess modules capable of autonomous decision-making, including task planning, tool invocation (such as APIs and databases), and iterative optimization. RAG combines information retrieval (IR) and natural language generation (NLG) technologies to improve the accuracy and reliability of content generated by large language models (LLMs). RAG enhances the accuracy of generated content by searching the RAG knowledge base. The goal of the Agent-RAG fusion architecture is to enable agents to proactively manage RAG processes (such as deciding when to search and how to filter information), using RAG to provide real-time data to support agent decision-making.
[0045] In the Agent-RAG fusion architecture, the Agent has the ability to divide charging problems into three types of task attributes according to their complexity: basic information query tasks, complex decision support tasks, and analysis and calculation tasks.
[0046] Therefore, after obtaining the charging problem input by the user, the agent determines the task attributes of the charging problem input by the user based on the complexity of the charging problem to determine whether the charging problem input by the user belongs to the basic information query task, the complex decision support task, or the analysis and calculation task, so as to achieve efficient processing of charging problems of different complexities.
[0047] As mentioned earlier, RAG combines information retrieval and text generation techniques to improve the accuracy and reliability of content generated by large language models. RAG enhances the accuracy of generated content by searching the RAG knowledge base.
[0048] The data in the RAG knowledge base is divided into two categories: structured data and unstructured data. Structured data includes charging pile equipment specifications and pricing rules, covering detailed specifications of different charging equipment, electricity pricing rules, and billing models. This data is stored in a clear, fixed format, facilitating rapid retrieval and analysis by the system, providing users with accurate charging-related information. Unstructured data encompasses user feedback, technical documentation, regulations, policies, and market trend analysis. Specifically, it includes historical records of user charging behavior and feedback, comprehensive charging technical documentation and professional maintenance manuals, charging industry regulations and policies, and analysis of new energy market trends and trends.
[0049] To effectively manage unstructured data, we use a hierarchical storage structure combined with a tree index. This hierarchical storage and tree indexing architecture not only systematically categorizes and stores various types of information, but also significantly improves data retrieval efficiency through optimized indexing mechanisms, thereby enhancing overall system performance and user experience.
[0050] Before using the database for hierarchical storage, unstructured data must be classified. Unstructured data uses a three-level classification system:
[0051] First-level classification: Unstructured data is divided into several major categories based on its nature and purpose. For example, in the charging industry, unstructured data is divided into the following four major categories: user feedback, technical documentation, regulations and policies, and market dynamics.
[0052] Secondary classification: Within each primary classification, the broad categories are further broken down into more specific subcategories. For example, in the charging industry, the broad category of technical documentation can be broken down into the following subcategories: charging equipment technical documentation, battery technical documentation, and charging station construction documentation; the broad category of user feedback can be broken down into the following subcategories: charging speed feedback, charging station failure feedback, and cost feedback.
[0053] Tertiary classification: Based on the secondary classification, subcategories are further subdivided into more specific subcategories. For example, in the charging industry scenario, the subcategory of charging equipment technical documentation can be further subdivided into the following subcategories: DC charging pile documentation, AC charging pile documentation.
[0054] After classifying unstructured data, use the database to store it hierarchically. To store the classified unstructured data in the database, design a data hierarchy (data_hierarchy) table to represent the hierarchical relationships of the data. The database stores these hierarchical relationships in the data_hierarchy table, which contains the id, data name, data storage path, parent node id, and classification information at each level. Specifically, the data_hierarchy table contains the following fields:
[0055] id: unique identifier;
[0056] data_name: data name;
[0057] data_path: data storage path;
[0058] parent_id: parent node id, used to indicate hierarchical relationship;
[0059] Category: the first-level category to which the data belongs;
[0060] sub_category: the secondary category to which the data belongs;
[0061] sub_sub_category: The third-level classification to which the data belongs.
[0062] In order to effectively manage unstructured data, a hierarchical storage structure is used in conjunction with a tree index. In terms of tree index, the present invention uses a kd-tree with a multi-level classification structure to process complex unstructured data.
[0063] A KD tree (K-Dimensional Tree) is a spatial partitioning data structure used to organize point data in k-dimensional space. It is an extension of a binary search tree in multidimensional space. K represents the dimension of the data. For example, a 2-D tree is used in two-dimensional space, and a 3-D tree is used in three-dimensional space.
[0064] Using a KD tree to process unstructured data begins with data preprocessing: extracting key features (categorical information, keywords, and timestamps) from the unstructured data and converting them into numerical labels. The KD tree is then recursively constructed, selecting dimensions (such as primary categories), determining the split point (typically the median), and constructing the left and right subtrees. This structure supports range queries, nearest neighbor queries, and dynamic updates, improving data retrieval efficiency and overall system performance.
[0065] The kd tree with a multi-level classification structure is used to process complex unstructured data. The specific process is as follows:
[0066] 1. First, perform data preprocessing: extract key features (classification information, keywords, timestamps) from unstructured data and convert them into numerical labels.
[0067] The data preprocessing process requires extracting key features from unstructured data, including classification information such as primary classification (such as technical documentation, user feedback, etc.), secondary classification (such as charging equipment technical documentation, charging speed problem feedback, etc.), and tertiary classification (such as DC charging pile documents, etc.). It is also necessary to extract keywords (keywords extracted from the document content) and timestamps (the time when the document was created or updated). The extracted key features (classification information, keywords, timestamps) are converted into numerical labels. For example, the classification information at all levels is encoded as numbers, the keywords are retained as the original keyword information, and the timestamps are normalized to a specific range.
[0068] For example, when constructing a kd-tree, unstructured data is converted into feature vectors, as shown in Table 1:
[0069] file name First-level classification Secondary classification Three-level classification Keywords Timestamp Document A 0 0 0 DC charging pile 0.1 Document B 1 1 2 Low temperature charging 0.2 Document C 2 2 3 New energy subsidies 0.3
[0070] 2. Recursively construct a kd-tree based on the extracted key features and numerical labels.
[0071] The construction of a KD tree involves three key steps: selecting a dimension (which can be primary, secondary, or tertiary categories, keywords, or timestamps), choosing a split point (the median of the selected dimension), and recursively constructing the left and right subtrees. This hierarchical construction ensures tree balance and query efficiency.
[0072] Taking Table 1 above as an example, the details are as follows:
[0073] A. Select the current dimension, for example: first-level category → second-level category → third-level category → keyword → timestamp.
[0074] C. Select the split point and recursively construct the kd tree.
[0075] Select the split point and find the median of the current dimension as the root node.
[0076] First layer: Select the first-level classification as the partition axis, the median is 1, and document B is the root node.
[0077] Second layer: recursively construct left and right subtrees.
[0078] Left subtree: Select the secondary classification as the split axis, the median is 0, and document A is the left child node.
[0079] Right subtree: Select the secondary classification as the split axis, the median is 2, and document C is the right child node.
[0080] The third layer: recursively construct the subtrees of the left and right subtrees.
[0081] Subtree of document A: Select the three-level classification as the partition axis, the median is 0, and document A is the leaf node.
[0082] Subtree of document C: Select the three-level classification as the partition axis, the median is 3, and document C is the leaf node.
[0083] By building a KD tree-driven index for unstructured data, two main indexing applications can be supported: range queries (such as locating documents within a specific category) and nearest neighbor queries (such as finding the document most relevant to a user's question). KD trees also support dynamic updates, allowing for the flexible insertion of new data and deletion of old data, ensuring query efficiency by maintaining the tree's balance.
[0084] In this embodiment, by establishing a hierarchical storage and tree indexing mechanism, that is, adopting a three-level classification system to manage unstructured data, and combining it with the kd-tree index structure, through key feature extraction, classification information conversion and recursive construction of the tree structure, it supports range queries, nearest neighbor queries and dynamic updates, thereby improving data retrieval efficiency and overall system performance.
[0085] In one embodiment, in step S3, according to the task attributes of the charging problem, the RAG based on the Agent-RAG fusion architecture and the reinforcement learning fine-tuning reasoning large model generates a charging problem answer that matches the task attributes of the charging problem.
[0086] Specifically, after obtaining the charging problem input by the user, the agent determines the task attributes of the charging problem input by the user based on the complexity of the charging problem to determine whether the charging problem input by the user belongs to a basic information query task, a complex decision support task, or an analytical calculation task.
[0087] So, if Figure 2As shown, in step S3, based on the task attributes of the charging problem, the RAG and reinforcement learning fine-tuning reasoning large model based on the Agent-RAG fusion architecture generates a charging problem answer that matches the task attributes of the charging problem, specifically including:
[0088] S31. If the task attribute of the charging problem is determined to be a basic information query task, the Agent directly calls the database retrieval tool to obtain the answer from the RAG knowledge base.
[0089] In basic information query tasks, the agent directly calls the database retrieval tool to accurately locate the required answers from the RAG knowledge base and delivers the results to the user, ensuring that the user can seamlessly obtain key information and improving the overall query experience.
[0090] S32. If the charging problem is identified as a complex decision support task, the complex decision support task is broken down into a set of subtasks. Each subtask agent then uses a database search tool to retrieve information in its own field and integrate the acquired knowledge resources. The trained reinforcement learning fine-tuned reasoning model then performs reasoning and analysis to generate a professional answer to the charging problem. This includes:
[0091] S321. Decompose complex decision support tasks into a set of subtasks with clear boundaries. Each subtask is responsible for acquiring and processing information in a specific dimension.
[0092] S322. Each subtask agent calls a search tool to perform targeted domain information retrieval, obtains knowledge in its respective domain, and integrates the acquired knowledge resources to provide structured support for decision-making.
[0093] S323. After the subtask agent completes information retrieval and preliminary integration, the trained reinforcement learning fine-tuning reasoning model performs multi-level logical analysis and associative reasoning on the integrated knowledge resources to generate answers to charging questions that meet professional standards and have practical value, thereby meeting users' needs for high-value information in complex decision support scenarios.
[0094] In this embodiment, complex decision support tasks employ a systematic process. First, the task is broken down into subtasks. Subtask agents invoke search tools to obtain information, and then a large model performs reasoning and analysis to generate professional answers. This integration of key steps—task decomposition, professional information retrieval, knowledge integration, and the deployment of a trained reinforcement learning-based fine-tuned reasoning large model—achieves a systematic approach to complex decision-making problems, meeting users' needs for high-value information in complex decision support scenarios.
[0095] S33. If the task attribute of the charging problem is determined to be an analytical computing task, the agent first retrieves relevant information as key context, and then uses the trained reinforcement learning fine-tuned reasoning model to perform reasoning and analysis to generate a professional answer.
[0096] When processing analytical computing tasks, the agent first retrieves relevant information and then passes it as key context to the trained reinforcement learning fine-tuned reasoning model, which then conducts comprehensive analysis and generates the final answer.
[0097] Reinforcement learning fine-tuning of large reasoning models enhances their capabilities in understanding professional knowledge, multi-step reasoning, and logical analysis through deep fine-tuning and parameter optimization based on domain-specific corpora. By introducing the Group Relative Policy Optimization (GRPO) reinforcement learning strategy, this model provides clear learning guidance for large-scale model training, significantly improving its training efficiency in reasoning tasks, particularly in complex tasks requiring step-by-step reasoning.
[0098] Through GRPO, reinforcement learning fine-tuning of large reasoning models can achieve the following three key capabilities: breaking down complex problems into coherent reasoning steps, using chain thinking to provide evidence-based answers, and following business guidelines to ensure the professionalism and accuracy of answers. Specifically:
[0099] Break down complex problems into coherent reasoning steps: Reinforcement learning fine-tuning the reasoning model can gradually break down complex problems, forming a clear and coherent reasoning path, ensuring that each step of reasoning has a clear goal and logical basis.
[0100] Use chain thinking to provide well-founded answers: The reinforcement learning fine-tuning reasoning model can use chain thinking throughout the entire reasoning process, deriving the logic of the next step based on the conclusion of the previous step, thereby providing logically rigorous and well-founded answers.
[0101] Follow business guidelines to ensure the professionalism and correctness of answers: The reinforcement learning fine-tuning reasoning model strictly follows specific business guidelines during the reasoning process, ensuring that the generated answers are not only professional but also highly accurate and reliable in business scenarios.
[0102] The training process for the large reinforcement learning fine-tuned reasoning model includes constructing 200,000 high-quality question-answer pairs, selecting Deepseek-R1 as the base model, designing a reward function (to evaluate question relevance and structured output), and implementing it using a GRPO-based training method. For each question, eight candidate outputs are generated, a reward evaluation mechanism is applied, rewards are normalized, and an advantage function is calculated to update model parameters. The model's reasoning capabilities are continuously optimized through multiple rounds of iterative training.
[0103] The specific training process of the reinforcement learning fine-tuning inference large model is as follows Figure 3 As shown:
[0104] X1. Data loading: Using expert annotation and large-scale model generation, we construct 200,000 high-quality question-and-answer pairs for complex scenarios, covering tasks such as charging equipment selection, peak power consumption prediction, charging efficiency optimization, and price strategy impact analysis.
[0105] The fine-tuning data format is: [{"role":"system","content":system prompt words},{"role":"user","content":user questions}].
[0106] X2. Load the training model: Select Deepseek-R1 as the basic model for training.
[0107] X3. Load the reward function: Ensure that the model does not receive rewards for wrong or meaningless answers through carefully designed answer checking and evaluation criteria.
[0108] Question Relevance: We use OpenAI's text-embedding-ada-002 model as a text embedding model, which accurately measures the semantic relevance between generated answers and reference standards. We calculate the cosine similarity of the vectors for each pair (answer, reference answer) to comprehensively assess the degree to which the answer matches the expected answer. Points are deducted for blank responses or answers that clearly deviate from the topic. The goal is to reward responses that not only include relevant content but also closely match the expected answer.
[0109] Structured output: To ensure that the output content is clearly structured and logically coherent, the generated answers must strictly follow the " <think> Reasoning process< / think> Each pair of labels that meets the requirements is identified and awarded with additional points. This approach optimizes the presentation of answers, making them easier to understand and evaluate, thereby improving the overall quality of answers.
[0110] X4. Using the framework provided by OpenR1 as the basis, implement the model training method based on group relative policy optimization (GRPO), as follows:
[0111] Diverse output sampling: For each question query, the model generates 8 candidate outputs based on the current policy model (Deepseek-R1) to ensure the diversity of the sample space.
[0112] Reward evaluation mechanism: Apply a reward function to each generated output to score it and obtain the corresponding reward score.
[0113] Normalized rewards: 1) Calculate the average reward within the group; 2) Calculate the standard deviation of the reward within the group; 3) Normalize the reward score of each output.
[0114] Compute the advantage function: For each output, its advantage function at each time step is set to the normalized reward.
[0115] Policy optimization: The model uses the calculated advantages to update the parameters of the policy model to maximize the objective function.
[0116] Iterative training: During the training process, the system answers questions, obtains feedback, and optimizes strategies through multiple rounds of iteration, thereby continuously improving its reasoning and analytical capabilities.
[0117] In this embodiment, the reinforcement learning fine-tuning reasoning large model optimizes the reinforcement learning strategy by introducing the group relative strategy. By constructing high-quality question-answer pairs, selecting DeepSeek-R1 as the basic model, designing a special reward function, and adopting a training method based on group relative strategy optimization and multiple rounds of iterative training, the specific field corpus is deeply fine-tuned and parameter optimized, so that the model has the ability to decompose problems, chain thinking and professional and accurate output, and enhances its ability in understanding professional knowledge, multi-step reasoning and logical analysis. It can significantly improve the training efficiency of the large model in reasoning tasks, especially in complex tasks that require step-by-step reasoning.
[0118] Based on the same concept, the present invention also provides an electronic device, such as Figure 4 As shown, the electronic device 900 includes: a memory 902, a processor 901, and one or more computer programs stored in the memory 902 and executable on the processor 901. The memory 902 and the processor 901 are coupled together via a bus system 903. When the one or more computer programs are executed by the processor 901, the following steps of an intelligent question-answering method for a charging station management system provided in an embodiment of the present invention are implemented:
[0119] S1. Obtain charging problem input by the user;
[0120] S2, the agent based on the Agent-RAG fusion architecture determines the task attributes of the charging problem input by the user;
[0121] S3. Based on the task attributes of the charging problem, the RAG and reinforcement learning fine-tuned reasoning model based on the Agent-RAG fusion architecture generates a charging problem answer that matches the task attributes of the charging problem;
[0122] S4. Deliver the generated answer to the charging question to the user.
[0123] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 901. Processor 901 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be performed by hardware integrated logic circuits or software instructions within processor 901. Processor 901 can be a general-purpose processor, a DSP, or other programmable logic device, a discrete gate or transistor logic device, or discrete hardware components. Processor 901 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules within the decoding processor. The software modules can be located in a storage medium located in memory 902. Processor 901 reads information from memory 902 and, in conjunction with its hardware, completes the steps of the above method.
[0124] It can be understood that the memory 902 in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device; the volatile memory can be random access memory (RAM), by way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM) Memory), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.
[0125] It should be noted that the above-mentioned electronic device embodiment and method embodiment belong to the same concept, and their specific implementation process is detailed in the method embodiment, and the technical features in the method embodiment are applicable to the operation server embodiment, which will not be repeated here.
[0126] In addition, in an exemplary embodiment, an embodiment of the present invention further provides a computer storage medium, specifically a computer-readable storage medium, for example, including a memory 902 storing a computer program. The computer storage medium stores one or more programs of a charging station management system intelligent question-answering method. When the one or more programs of the charging station management system intelligent question-answering method are executed by the processor 901, the following steps of the charging station management system intelligent question-answering method provided in an embodiment of the present invention are implemented:
[0127] S1. Obtain charging problem input by the user;
[0128] S2, the agent based on the Agent-RAG fusion architecture determines the task attributes of the charging problem input by the user;
[0129] S3. Based on the task attributes of the charging problem, the RAG and reinforcement learning fine-tuned reasoning model based on the Agent-RAG fusion architecture generates a charging problem answer that matches the task attributes of the charging problem;
[0130] S4. Deliver the generated answer to the charging question to the user.
[0131] It should be noted that the program embodiment of the intelligent question-and-answer method for the charging station management system on the above-mentioned computer-readable storage medium and the method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment, and the technical features in the method embodiment are correspondingly applicable in the embodiment of the above-mentioned computer-readable storage medium, which will not be repeated here.
[0132] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.
[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Under the concept of the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the present invention as described above. For the sake of simplicity, they are not provided in detail. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. An intelligent question-answering method for a charging station management system, characterized in that: include: Get the charging problem input by the user; The agent based on the Agent-RAG fusion architecture determines the task attributes of the charging problem input by the user; According to the task attributes of the charging problem, the RAG and reinforcement learning fine-tuning reasoning large model based on the Agent-RAG fusion architecture generates a charging problem answer that matches the task attributes of the charging problem; The generated answer to the charging question is delivered to the user.
2. The intelligent question-answering method for a charging station management system according to claim 1, characterized in that: The task attributes of the charging problem are divided by the Agent into basic information query tasks, complex decision support tasks and analysis and calculation tasks according to the complexity of the charging problem.
3. The intelligent question-answering method for a charging station management system according to claim 2, characterized in that: In the Agent-RAG fusion architecture, RAG provides real-time data to support Agent decision-making by searching the RAG knowledge base. The data in the RAG knowledge base is divided into two categories: structured data and unstructured data. The structured data includes charging pile equipment specification data and pricing rule data, and the unstructured data includes user feedback, technical documents, regulations and policies, and market trend analysis.
4. The intelligent question-answering method for a charging station management system according to claim 3, characterized in that: The unstructured data is managed using a hierarchical storage structure combined with a tree index method, including: the unstructured data adopts a three-level classification system, and a data hierarchy table is designed in the RAG database to store the hierarchical relationship of the data, and a kd tree is used to establish a tree index mechanism.
5. The intelligent question-answering method for a charging station management system according to claim 3, characterized in that: According to the task attributes of the charging problem, the RAG based on the Agent-RAG fusion architecture and the reinforcement learning fine-tuning reasoning large model generate a charging problem answer that matches the task attributes of the charging problem, including: If the task attribute of the charging problem is determined to be a basic information query task, the agent directly calls the database retrieval tool to obtain the answer from the RAG knowledge base.
6. The intelligent question-answering method for a charging station management system according to claim 3, characterized in that: According to the task attributes of the charging problem, the RAG based on the Agent-RAG fusion architecture and the reinforcement learning fine-tuning reasoning large model generate a charging problem answer that matches the task attributes of the charging problem, including: If the task attribute of the charging problem is determined to be a complex decision support task, the complex decision support task is broken down into a set of subtasks. Each subtask agent calls a database retrieval tool to complete information retrieval in its respective field, integrates the acquired knowledge resources, and then uses a trained reinforcement learning fine-tuned reasoning model to perform reasoning and analysis to generate a professional answer to the charging problem.
7. The intelligent question-answering method for a charging station management system according to claim 3, characterized in that: According to the task attributes of the charging problem, the RAG based on the Agent-RAG fusion architecture and the reinforcement learning fine-tuning reasoning large model generate a charging problem answer that matches the task attributes of the charging problem, including: If the task attribute of the charging problem is determined to be an analytical computing task, the agent first retrieves relevant information as key context, and then uses the trained reinforcement learning fine-tuned reasoning model to perform reasoning and analysis to generate a professional answer.
8. The intelligent question-answering method for a charging station management system according to claim 6 or 7, characterized in that: The reinforcement learning fine-tuning reasoning large model is obtained by introducing a group relative strategy to optimize the reinforcement learning strategy, constructing high-quality question-answer pairs, selecting DeepSeek-R1 as the basic model, designing a reward function, and adopting a training method based on group relative strategy optimization through multiple rounds of iterative training to continuously optimize the reasoning ability.
9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the computer program is executed by the processor, the intelligent question-answering method for the charging station management system according to any one of claims 1 to 8 is implemented.
10. A storage medium, characterized in that: The storage medium stores a program of the intelligent question-answering method for the charging station management system. When the program of the intelligent question-answering method for the charging station management system is executed by the processor, the intelligent question-answering method for the charging station management system according to any one of claims 1 to 8 is implemented.