Agent-VLM-RAG-based electric customer service response method and related device

Abstract

The invention discloses an Agent-VLM-RAG-based power customer service response method and a related device, and belongs to the technical field of power industry service systems. The method comprises the following steps: receiving query information input by a user through an agent, analyzing the query information, and identifying the intention of the user; if the query information contains non-text information, calling a visual language model (VLM) to recognize the non-text information, and generating structured text data; and generating an RAG by adopting retrieval enhancement, retrieving related information contents from the power knowledge base in combination with the user intention and the structured text data, and generating reply contents for feedback. According to the method, an Agent agent lifting model, a VLM (Visual Language Model) and a retrieval enhanced generation (RAG) technology are fused, and more intelligent, efficient and accurate customer service experience is provided. According to the invention, the response speed and the processing accuracy of the electric customer service can be obviously improved, the user demand can be better met, and the user satisfaction can be improved.

Description

Technical Field

[0001] This invention belongs to the technical field of power industry service systems, and relates to a power customer service response method and related devices based on Agent-VLM-RAG. Background Technology

[0002] Currently, customer service systems in the power industry primarily rely on two traditional models: automated response systems based on Interactive Voice Response (IVR) and human customer service agents. IVR systems typically use pre-set, highly structured scripts and limited response templates to handle customer inquiries. This model demonstrates some efficiency when dealing with standardized, high-frequency issues, such as electricity bill inquiries and branch office information. However, its core flaw lies in a severe lack of flexibility and intelligence. When user questions exceed the scope of pre-set templates, especially those involving complex, non-standard descriptions of power faults, billing disputes, or cross-business inquiries, IVR systems often fail to accurately understand the user's intent, providing only general answers or mechanically directing the user to a human agent, resulting in numerous gaps in the customer experience and lengthy problem-solving cycles. While human customer service agents possess a degree of flexibility and judgment, their operating costs are high, and service quality is constrained by the professional level, work status, and training depth of the agents. Faced with a massive volume of diverse user inquiries, human systems struggle to guarantee consistent response speed and service standards. Especially during peak electricity consumption periods or when regional faults occur, a surge in inquiries can easily lead to line congestion and significantly extend user waiting times. This not only reduces customer satisfaction but also puts enormous pressure on the service capabilities of power companies.

[0003] With the widespread adoption of mobile internet and smart devices, users tend to use more convenient methods (such as taking photos or uploading documents) to help describe problems. For example, users might directly upload meter readings, screenshots of electricity bills, photos of malfunctioning equipment, or images of damaged power facilities. However, existing systems generally lack effective computer vision technology support. When users upload images, the system usually cannot directly recognize and understand the image content, either ignoring this crucial information or storing it as an attachment, relying entirely on human customer service for review and judgment. This approach has significant drawbacks: first, it disrupts the information processing flow, creating an information barrier between automated customer service and human processing, hindering immediate feedback; second, it greatly reduces processing efficiency, requiring a lengthy manual process for a problem that could be quickly located through automatic image recognition; and finally, it increases the risk of human error and operational costs. On the other hand, existing customer service systems lack sufficient intelligence in knowledge base utilization and answer generation. Traditional keyword-based retrieval methods struggle to understand the deep semantics and contextual relationships of user queries, often retrieving irrelevant or outdated information, resulting in low accuracy of responses. At the same time, most systems lack the ability to dynamically generate answers, and their responses rely on pre-entered fixed question-and-answer pairs (FAQs), making it impossible to reason and generate comprehensive answers for complex, combined questions.

[0004] In summary, traditional power customer service systems often suffer from long response times and high error rates when handling complex power-related issues (such as billing problems and fault reporting). This is because they are unable to effectively understand and process customer inquiries containing images and complex documents. Furthermore, existing systems lack sufficient intelligent support in information retrieval and response generation, often failing to provide fast and accurate customer service. Summary of the Invention

[0005] The purpose of this invention is to provide an Agent-VLM-RAG-based electricity customer service response method and related apparatus to solve the technical problems of low efficiency and inaccurate information processing in the existing technology when dealing with a large number of electricity service-related issues.

[0006] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, the present invention provides a power customer service response method based on Agent-VLM-RAG, comprising the following steps: The agent receives query information input by the user, parses the query information, and identifies the user's intent. If the query information contains non-text information, the Visual Language Model (VLM) is invoked to identify the non-text information and generate structured text data. The system employs retrieval enhancement to generate a Relationship Aggregator (RAG). By combining user intent with structured text data, it retrieves relevant information from the power knowledge base and generates responses to provide feedback.

[0007] Furthermore, the step of receiving user-input query information through the intelligent agent, parsing the query information, and identifying the user's intent specifically includes: A model based on the Transformer architecture is constructed and trained using dialogue data from the power industry to identify the intent categories of user queries; the intent categories include at least one of billing inquiries, fault reporting, service appointments, and business consultations. Based on the intent category identification results, a text query task is generated.

[0008] Furthermore, the step of calling the Visual Language Model (VLM) to identify the non-textual information and generate structured text data if the query information contains non-textual information specifically includes: If the query information contains non-textual information, the Visual Language Model (VLM) is invoked. By combining Convolutional Neural Network (CNN) with Natural Language Processing (NLP) techniques, the text and objects in the non-textual information are identified and converted into structured text data for output.

[0009] Furthermore, the step of using retrieval enhancement to generate a RAG, combining user intent with structured text data, retrieving relevant information from the power knowledge base, and generating a response for feedback specifically includes: The user intent parsed by the intelligent agent and the structured text data generated by the visual language model (VLM) are matched for similarity in a vectorized power knowledge base to retrieve relevant information. Based on relevant information and the current dialogue context, a natural language response is generated and provided.

[0010] Furthermore, the content stored in the power knowledge base includes at least power policies and regulations, equipment manuals, frequently asked questions (FAQs), and historical work order data.

[0011] Furthermore, the step of generating a natural language response and providing feedback based on relevant information and the current dialogue context specifically includes: Determine the complexity of the problem based on the retrieved relevant information; If the question is determined to be simple, the retrieval enhancement generates a natural language response containing the solution using RAG synthesis. If the issue is determined to be complex, a work order containing a problem summary and user contact information will be automatically generated and transferred to the human customer service terminal.

[0012] Secondly, the present invention provides an Agent-VLM-RAG-based power customer service response system, comprising: The query information recognition module is used to receive query information input by the user through the intelligent agent, parse the query information, and identify the user's intent; The non-text information recognition module is used to call the Visual Language Model (VLM) to recognize the non-text information and generate structured text data if the query information contains non-text information. The retrieval and response module is used to generate a Retrieval Augmentation (RAG) by employing retrieval enhancement, combining user intent with structured text data, retrieving relevant information from the power knowledge base, and generating response content for feedback.

[0013] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the Agent-VLM-RAG-based power customer service response method described above.

[0014] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the Agent-VLM-RAG-based power customer service response method described above.

[0015] Fifthly, the present invention provides a computer program product comprising at least one instruction or at least one program segment, wherein the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the steps of the Agent-VLM-RAG-based power customer service response method as described above.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention discloses an Agent-VLM-RAG-based power customer service response method and related apparatus. By receiving and parsing user query information through an intelligent agent to identify intent, it can quickly and accurately grasp user needs, laying the foundation for subsequent responses and greatly improving the relevance of the response, avoiding irrelevant answers. Secondly, when the query information contains non-textual information, the Visual Language Model (VLM) is invoked for recognition and the generation of structured text data, effectively expanding the customer service system's ability to process information types. Non-textual content such as images and charts can be accurately interpreted, enriching information acquisition channels and making responses more comprehensive and accurate. Finally, a search-enhanced RAG is generated, combining user intent with structured text data to retrieve information from the power knowledge base and generate responses, fully utilizing the vast resources of the power knowledge base and ensuring the professionalism and authority of the responses. Compared to traditional customer service systems, this invention reduces response time by an average of 40% and improves query accuracy by 50% when processing power-related queries. When handling complex queries containing non-textual content, the system achieves a success rate of 85%, significantly higher than the processing effect of ordinary OCR technology. Furthermore, customer satisfaction surveys show that after using the system of this invention, customer satisfaction increased by 60%, effectively reducing customer waiting time and the difficulty of resolving problems. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the system of the present invention; Figure 3 This is a schematic diagram of the mini-program homepage according to an embodiment of the present invention; Figure 4 This is a diagram of the customer service interface of the system in an embodiment of the present invention; Figure 5 This is the power outage reporting page of the system in this embodiment of the invention; Figure 6 This is the feedback and complaint page for the system in this embodiment of the invention. Detailed Implementation

[0019] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0020] The following detailed description is exemplary and intended to provide further detailed explanation of the invention. Unless otherwise specified, all technical terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention.

[0021] See Figure 1 This invention discloses a power customer service response method based on Agent-VLM-RAG, comprising the following steps: S1, receives query information input by the user through the intelligent agent, parses the query information, and identifies the user's intent; A model based on the Transformer architecture was constructed and trained using dialogue data from the power industry to identify the intent categories of user queries. These intent categories include at least one of the following: bill inquiry, fault reporting, service appointment, and business consultation. Then, based on the intent category identification results, a text query task was generated. This provides clear "action instructions" for the subsequent collaborative work of the VLM and RAG modules.

[0022] S2, If the query information contains non-text information, then the Visual Language Model (VLM) is invoked to identify the non-text information and generate structured text data; If the query information contains non-textual information, the Visual Language Model (VLM) is invoked. By combining a Convolutional Neural Network (CNN) with Natural Language Processing (NLP) techniques, the VLM identifies the text and objects within the non-textual information and transforms them into structured text data for output. VLM possesses a deep understanding of the visual semantics of non-textual information such as images. For example, it can accurately identify meter readings, extract key data from complex bill layouts, or determine fault characteristics from equipment images. It transforms this unstructured information into structured text data that the system can understand, achieving a "holographic" perception of multimodal user queries. S3 employs retrieval enhancement to generate a RAG, combining user intent with structured text data to retrieve relevant information from the power knowledge base and generate a response for feedback.

[0023] S301, the user intent parsed by the intelligent agent and the structured text data generated by the visual language model (VLM) are matched for similarity in a vectorized power knowledge base to retrieve relevant information; the power knowledge base stores at least power policies and regulations, equipment manuals, frequently asked questions (FAQs), and historical work order data.

[0024] S302, Based on relevant information and the current dialogue context, generate a natural language response and provide feedback.

[0025] 1) Determine the complexity of the problem based on the retrieved relevant information; 2) If the question is determined to be simple, the search enhancement will generate a natural language response containing the solution using RAG synthesis. 3) If the problem is determined to be complex, a work order containing a problem summary and user contact information will be automatically generated and transferred to the human customer service terminal.

[0026] The retrieval mechanism used in this step does not rely on simple keyword matching, but rather on semantic vectorized matching. This means that the system can understand the deeper meaning of the user's query, and even if the question is posed in an obscure or colloquial way, it can retrieve the most semantically relevant information fragments from the knowledge base.

[0027] Traditional power customer service systems often suffer from long response times and high error rates when handling complex power-related issues (such as billing problems and fault reporting). This is because they struggle to effectively understand and process customer inquiries containing images and complex documents. Furthermore, existing systems lack sufficient intelligent support in information retrieval and response generation, often failing to provide fast and accurate customer service. To overcome these shortcomings, this invention integrates an agent-enhanced model, a visual language model (VLM), and retrieval-enhanced generation (RAG) technologies, aiming to provide a more intelligent, efficient, and accurate customer service experience. This invention can significantly improve the response speed and processing accuracy of power customer service, better meeting user needs and increasing user satisfaction.

[0028] See Figure 2 This invention discloses an Agent-VLM-RAG-based power customer service response system, including a query information recognition module, a non-text information recognition module, and a retrieval and response module. Details are as follows: The query information recognition module receives user-input query information through an intelligent agent, parses the query information, and identifies the user's intent. Specifically, the agent serves as the system's front end, directly interacting with the user. This agent possesses an advanced dialogue management system, capable of parsing user language input, understanding intent, and generating corresponding responses. The core of the agent is a Transformer-based deep learning model, trained on extensive power industry dialogue data, enabling it to understand power-related terminology and its context.

[0029] The non-text information recognition module is used to identify non-text information if the query information contains such information, by calling the Visual Language Model (VLM) to generate structured text data. Specifically, this module processes non-text information uploaded by users, such as images or PDF files. By combining convolutional neural networks (CNNs) with natural language processing techniques, VLM can recognize text and objects in images and convert them into structured data that the system can understand. For example, it can extract consumption data from electricity bill images or identify fault types from equipment images.

[0030] The retrieval-augmented generation (RAG) module uses user intent and structured text data to retrieve relevant information from the power knowledge base and generate responses. Specifically, this module enhances the accuracy and relevance of the generated answers. The RAG system utilizes both knowledge-based retrieval and pattern-based generation mechanisms to retrieve the most relevant information from the system's knowledge base and then combines this with the context of the current dialogue to generate accurate and reasonable answers.

[0031] The working process of this invention is as follows: Users input questions through the interface, which can be text or images. The agent first analyzes the text input to determine the user's intent. If images or complex documents are involved, the Virtual Library (VLM) intervenes, automatically processing the data or tagging information in the image and converting it into text. Based on the extracted information and the user's query, the agent uses RAG technology to retrieve relevant data from the backend knowledge base and generates an answer. Finally, the generated answer is returned to the user through the interface. If the user has further queries or feedback, the system can iteratively process them.

[0032] The electricity customer service system of this invention, based on an Agent-VLM-RAG structure, significantly improves the efficiency and accuracy of handling electricity-related issues. By using an Agent intelligent agent, the system can automatically identify and parse complex electricity-related questions raised by users, and process queries including textual and non-textual content (such as tables and complex images) through the VLM visual language model, achieving comprehensive information retrieval and understanding. Furthermore, the RAG retrieval enhancement generation mechanism further enhances the quality of responses and the generation of relevant information.

[0033] Economically, the implementation of this system significantly reduces human customer service costs. Preliminary estimates suggest that using this invention's electricity customer service system can save approximately 30% in human resource expenditures. Simultaneously, the system's implementation also improves customer satisfaction and reduces churn, leading to better revenue performance and lower customer retention costs. It can automatically handle 80% of common electricity-related queries, reducing the average response time from minutes to seconds. By integrating VLM and RAG technologies, the system can provide an accuracy rate of up to 95%, significantly outperforming traditional customer service systems.

[0034] In terms of social benefits, this invention improves the response speed and service quality of public power services through an efficient intelligent customer service system, promoting innovation and development in the power industry's service model. It makes power services faster and more accurate, greatly enhancing the public's life experience and satisfaction, while also contributing to the application and popularization of intelligent technology in more fields. In actual deployment and testing, compared to traditional customer service systems, this system reduces the average response time for power-related queries by 40% and improves query accuracy by 50%. When handling complex queries containing non-textual content, the system achieves a success rate of 85%, significantly higher than the processing effect of ordinary OCR technology. Furthermore, customer satisfaction surveys show that after using this invention's system, customer satisfaction increased by 60%, effectively reducing customer waiting time and the difficulty of resolving problems.

[0035] Example: This embodiment discloses an electricity customer service response system based on Agent-VLM-RAG, whose specific architecture mainly consists of an intelligent customer service Agent module, a visual language model (VLM) module, and a retrieval enhancement generation (RAG) module.

[0036] Intelligent Customer Service Agent Module: This module is responsible for interacting with users, receiving their inquiries, and analyzing the type of question. The intelligent agent can handle various electricity-related issues, including bill inquiries, fault reports, and service appointments. It uses natural language processing technology to parse the user's input text and categorizes the questions before sending them to the appropriate processing modules.

[0037] Visual Language Model (VLM) module: The key technology involved in the VLM module is the Visual Language Model, which can process information from images and text. In this system, the VLM module is activated when a user's question involves a bill image or a complex picture of electrical facilities. It first attempts to convert the text in the image into text using OCR technology. For content that is difficult to convert directly, such as tables or specific icons, the module uses its visual capabilities to directly mark and identify these specific areas.

[0038] Search Enhancement Generation (RAG) module: The RAG module, as a supplement to the intelligent agent, enhances the accuracy and relevance of the system's responses. This module retrieves information from the parsed user question against an existing database, identifies the most relevant information fragments, and then generates an answer based on these fragments, thus providing the user with a more accurate and comprehensive response.

[0039] Workflow: First, the system receives input from the user, and the intelligent agent module categorizes the questions. Questions are then assigned to the VLM module for image-related queries, or directly to the RAG module to retrieve existing data and generate answers. All processing results are ultimately integrated to form the final answer, which is then delivered to the user through the intelligent agent module.

[0040] A WeChat mini-program has been developed based on the above method. Users can open WeChat and scan the mini-program's QR code to start using it. See the mini-program homepage for details. Figure 3 There are three options: customer service, power outage reporting, and feedback / complaints. Detailed usage instructions are as follows: 1. Customer service page See Figure 4 Users can enter their question in the input box at the bottom and then click the send button to chat with the AI ​​customer service.

[0041] 2. Power Outage Reporting Page See Figure 5 When users enter the page, they will be prompted to authorize the mini-program to obtain location information to help them quickly fill in the accurate location of the power outage. Users can also manually modify the automatically obtained location, and then enter their name and phone number to submit to the power department.

[0042] 3. Feedback and Complaints Page See Figure 6 Users can fill in their complaint on the page and then click "save" to submit it.

[0043] Power industry workers can join WeChat Work. When a user submits a power outage report, the power outage reporting program in WeChat Work will automatically send the power outage information to members of the company.

[0044] In one embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of an Agent-VLM-RAG-based power customer service response method.

[0045] This invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the Agent-VLM-RAG-based power customer service response method in the above embodiments.

[0046] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0047] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0048] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0049] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A power customer service response method based on Agent-VLM-RAG, characterized in that, Includes the following steps: The agent receives query information input by the user, parses the query information, and identifies the user's intent. If the query information contains non-text information, the Visual Language Model (VLM) is invoked to identify the non-text information and generate structured text data. The system employs retrieval enhancement to generate a Relationship Aggregator (RAG). By combining user intent with structured text data, it retrieves relevant information from the power knowledge base and generates responses to provide feedback.

2. The power customer service response method based on Agent-VLM-RAG according to claim 1, characterized in that, The steps of receiving user-input query information through the intelligent agent, parsing the query information, and identifying user intent specifically include: A model based on the Transformer architecture is constructed and trained using dialogue data from the power industry to identify the intent categories of user queries; the intent categories include at least one of billing inquiries, fault reporting, service appointments, and business consultations. Based on the intent category identification results, a text query task is generated.

3. The power customer service response method based on Agent-VLM-RAG according to claim 1, characterized in that, The step of calling the Visual Language Model (VLM) to identify the non-text information and generate structured text data if the query information contains non-text information specifically includes: If the query information contains non-textual information, the Visual Language Model (VLM) is invoked. By combining Convolutional Neural Network (CNN) with Natural Language Processing (NLP) techniques, the text and objects in the non-textual information are identified and converted into structured text data for output.

4. The power customer service response method based on Agent-VLM-RAG according to claim 1, characterized in that, The steps of generating a Retrieval Augmentation (RAG) using retrieval enhancement, combining user intent with structured text data, retrieving relevant information from the power knowledge base, and generating a response to provide feedback specifically include: The user intent parsed by the intelligent agent and the structured text data generated by the visual language model (VLM) are matched for similarity in a vectorized power knowledge base to retrieve relevant information. Based on relevant information and the current dialogue context, a natural language response is generated and provided.

5. A power customer service response method based on Agent-VLM-RAG according to claim 4, characterized in that, The power knowledge base stores at least the following information: power policies and regulations, equipment manuals, frequently asked questions (FAQs), and historical work order data.

6. The power customer service response method based on Agent-VLM-RAG according to claim 4, characterized in that, The step of generating a natural language response and providing feedback based on relevant information and the current dialogue context specifically includes: Determine the complexity of the problem based on the retrieved relevant information; If the question is determined to be simple, the retrieval enhancement generates a natural language response containing the solution using RAG synthesis. If the issue is determined to be complex, a work order containing a problem summary and user contact information will be automatically generated and transferred to the human customer service terminal.

7. A power customer service response system based on Agent-VLM-RAG, characterized in that, include: The query information recognition module is used to receive query information input by the user through the intelligent agent, parse the query information, and identify the user's intent; The non-text information recognition module is used to call the Visual Language Model (VLM) to recognize the non-text information and generate structured text data if the query information contains non-text information. The retrieval and response module is used to generate a Retrieval Augmentation (RAG) by employing retrieval enhancement, combining user intent with structured text data, retrieving relevant information from the power knowledge base, and generating response content for feedback.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the Agent-VLM-RAG-based power customer service response method as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the Agent-VLM-RAG-based power customer service response method as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the steps of the Agent-VLM-RAG-based power customer service response method as described in any one of claims 1-6.

Images