Automatic customer service reply method and system in expressway scene based on large language model

Abstract

The invention discloses an automatic customer service reply method and system in an expressway scene based on a large language model, and relates to the technical field of expressway customer service, and the method comprises the steps: obtaining the audio information and vehicle data information of a user; converting the audio information into a text; performing semantic analysis on the text by using the natural language processing capability of a large language model, and generating a response text and a special situation processing flow in combination with the vehicle data information; and converting the response text and the special condition processing flow into audio information, and broadcasting and displaying the audio information. According to the method, the voice is converted into the text by locally deploying the voice recognition engine and configuring the hot words, the text is synthesized into the voice through the voice synthesis engine of the terminal, semantic analysis and problem analysis are performed on the problem based on a large language model, and judgment on a service processing scene is realized through information retrieval; and turning to a corresponding business processing flow according to the customer service processing capability, thereby realizing identification of user appeals and self-service processing of common problems.

Description

Technical Field

[0001] The present invention relates to the technical field of highway customer service, and more specifically to a customer service automatic reply method and system in a highway scenario based on a large language model. Background Art

[0002] In order to meet the development goals of "improving toll collection and traffic efficiency, enhancing the level of intelligent application, optimizing the staffing structure, and implementing quality improvement, cost reduction and efficiency increase", more and more highway toll stations are turning to unmanned card issuance and self-service payment. In the context of staff reduction, how to quickly and effectively handle special situations in the lane has become increasingly critical.

[0003] At present, toll stations and toll centers have customer service teams specifically responsible for handling various special situations. When a special situation occurs, the car owner can quickly contact the customer service staff through the voice intercom system, and the customer service staff will provide remote guidance to solve the problem based on the car owner's description. However, this method not only requires customer service staff to be on call at all times, which significantly increases labor costs, but also still faces the problem of "many special situations and few customer service" and untimely handling during peak passenger flow periods.

[0004] Therefore, how to provide a new customer service automatic reply method to improve processing efficiency and reduce costs is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a customer service automatic reply method and system in a highway scenario based on a large language model to solve the problems existing in the background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A customer service automatic reply method in a highway scenario based on a large language model, comprising:

[0008] Obtain the user's audio information and vehicle data information;

[0009] Converting the audio information into text;

[0010] Use the natural language processing capabilities of the large language model to perform semantic analysis on the text, and generate a response text and special situation processing flow in combination with vehicle data information;

[0011] Convert the response text and special situation handling process into audio information for broadcast display.

[0012] Optionally, the specific process of obtaining the user's audio information is:

[0013] When a special situation occurs in the lane, the user turns on the intercom function of the toll collection robot and describes the current situation to obtain the user's audio information.

[0014] Optionally, the large language model specifically includes:

[0015] Collect historical lane special situation handling data, clean and pre-process the data, and remove invalid data and noise;

[0016] Based on the pre-processed historical vehicle special situation handling records, a general vector data set for highway customer service is constructed using a graph database or a vector database to generate handling plans for various special situations.

[0017] Use the natural language processing capabilities of the large language model to perform semantic analysis on the text, and combine it with the data information collected by the lane to obtain information fragments related to the question;

[0018] Combine information fragments with disposal plans and automatically generate response text and special situation handling procedures.

[0019] Optionally, the generation of the response text and the special situation handling process specifically include:

[0020] Preliminary establishment of special situation sample sets based on business scenarios in the transportation field, and use tools to index text data;

[0021] Classify the special situation problems mentioned in the text, retrieve the general vector data set of highway customer service, and obtain the special situation handling plan;

[0022] The natural language processing model is used to generate response text and special situation processing flow according to the special situation processing plan and context information.

[0023] Optionally, it also includes an evaluation based on the special situation handling situation. Problems that are not automatically identified will be automatically generated as special situation samples based on the final processing method of the business system. After meeting the evaluation weights, they will be automatically included in the general vector data set of highway customer service, realizing the optimization strategy of self-updating data sets.

[0024] Optionally, the special situations are divided into operation procedures, billing disputes, and special vehicle handling;

[0025] In special cases related to operation procedures, the corresponding operation procedures and response texts are converted into voice prompts or interface displays to guide the car owner to complete the charging process;

[0026] In case of billing dispute, the external system is linked to obtain the routing information and billing information, and then the information is displayed. If the abnormality still exists, manual intervention is performed.

[0027] In special vehicle handling situations, the vehicle information will be verified. Once confirmed to be correct, the vehicle owner will be guided to complete the charging process based on the special vehicle handling process.

[0028] A customer service automatic reply system in highway scenarios based on a large language model, including: a toll collection robot, audio and video services, and a cloud customer service system;

[0029] The toll collection robot is equipped with an intercom device, which is connected to the audio and video service through the network. When a special situation occurs in the lane, the user turns on the intercom function of the toll collection robot and describes the current situation. After the toll collection robot transmits the acquired voice data to the audio and video service, it completes the audio analysis and converts the voice file into formatted text data, and saves the original audio and video files locally for later data retrieval.

[0030] The audio and video service forwards the converted formatted text data to the cloud customer service system, which performs semantic analysis on the text based on the large language model;

[0031] The cloud customer service system combines the semantic analysis results and the vehicle data information collected by the toll collection robot to perform information retrieval, classify the special situations and obtain the corresponding process information, and generate the response text and special situation processing process;

[0032] The cloud customer service system pushes the response text and special situation handling process to the toll collection robot, which guides the car owner to complete the toll collection process by converting the response text into voice prompts or interface displays.

[0033] It can be known from the above technical solutions that compared with the prior art, the present invention discloses a method and system for automatic customer service reply in a highway scenario based on a large language model, including: obtaining the user's audio information and vehicle data information; converting the audio information into text; using the natural language processing capability of the large language model to perform semantic analysis on the text, and generating a response text and a special situation processing flow in combination with the vehicle data information; converting the response text and the special situation processing flow into audio information for broadcast display. The present invention realizes voice-to-text conversion by locally deploying a speech recognition engine and hot word configuration, realizes text-to-speech synthesis through the terminal's speech synthesis engine, realizes semantic analysis and problem resolution of problems based on a large language model, realizes discrimination of business processing scenarios through information retrieval, and turns to the corresponding business processing flow according to the customer service processing capability, thereby realizing identification of user demands and self-service processing of common problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0035] Figure 1 A schematic diagram of the method provided by the present invention;

[0036] Figure 2 A schematic diagram of the large language model training process provided by the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] The embodiment of the present invention discloses a method for automatically replying customer service in a highway scenario based on a large language model, such as Figure 1 As shown, including:

[0039] Obtain the user's audio information and vehicle data information;

[0040] Convert audio information into text;

[0041] Use the natural language processing capabilities of the large language model to perform semantic analysis on the text, and combine vehicle data information to generate response text and special situation processing procedures;

[0042] Convert the response text and special situation handling process into audio information for broadcast display.

[0043] In a specific embodiment, the specific process of obtaining the user's audio information is:

[0044] When a special situation occurs in the lane, the user turns on the intercom function of the toll collection robot and describes the current situation to obtain the user's audio information.

[0045] In a specific embodiment, the large language model specifically includes:

[0046] Collect historical lane special situation handling data, clean and pre-process the data, and remove invalid data and noise;

[0047] Based on the pre-processed historical vehicle special situation handling records, a general vector data set for highway customer service is constructed using a graph database or a vector database to generate handling plans for various special situations.

[0048] Use the natural language processing capabilities of the large language model to perform semantic analysis on the text, and combine it with the data information collected by the lane to obtain information fragments related to the question;

[0049] Combine information fragments with disposal plans and automatically generate response text and special situation handling procedures.

[0050] In a specific embodiment, generating a response text and a special situation processing flow specifically include:

[0051] First, we preliminarily established a special situation sample set based on business scenarios in the transportation field, and used tools to index text data;

[0052] Classify the special situation problems mentioned in the text, retrieve the general vector data set of highway customer service, and obtain the special situation handling plan;

[0053] Finally, the natural language processing model is used to generate the response text and special situation processing flow according to the special situation processing plan and context information.

[0054] In a specific embodiment, it also includes an evaluation based on the special situation handling situation. Problems that are not automatically identified will be automatically generated into special situation samples based on the final processing method of the business system. After meeting the evaluation weight, they will be automatically included in the general vector data set of highway customer service, realizing the optimization strategy of self-updating data sets.

[0055] In a specific embodiment, Figure 2 As shown, the large language model training phase:

[0056] Collect historical lane special situation handling data, clean and preprocess the data to remove invalid data and noise. Label data for specific tasks (such as sentiment classification, intent recognition) to improve the performance of the trained model.

[0057] Based on the collected data, the pre-trained model is tuned to enable it to better understand the content of conversations in the transportation field, and to generate answers that meet customer needs through a large language model. Use the retrieval-enhanced generation engine to achieve effective classification and understanding of user questions. First, a preliminary question sample set is established based on business scenarios in the transportation field, and data is indexed using tools such as Elasticsearch to quickly retrieve relevant information. Local private large models such as Llama3 are used to classify questions, retrieve relevant question sample sets, and turn to the business subsystem to handle the questions. Finally, natural language processing models such as GPT are used to generate initial answers based on the processing results of the question and contextual information, and then further post-processing is performed, such as removing redundant information and formatting text. After completing the overall steps, an evaluation is conducted based on the problem handling situation. Questions that are not automatically identified will automatically generate question samples based on the final processing method of the business system. After meeting the evaluation weight, they will be automatically included in the question sample set to achieve an optimization strategy for self-updating data sets.

[0058] Integrate the trained model into the existing customer service platform and provide external question-and-answer services in the form of API.

[0059] Special situations can be roughly divided into operation procedures, billing disputes, and special vehicle handling. For special situations of operation procedures, the cloud customer service system will push the corresponding operation procedures and response texts to the toll robot. The toll robot will guide the car owner to complete the toll collection process by converting the response text into voice prompts, interface displays, etc.; for special situations of billing disputes, the cloud customer service system needs to link external systems such as the path fitting system and the online billing system. The path fitting system can retrieve the vehicle's license plate capture record and gantry travel record by inputting vehicle information (license plate information, time of entering the highway), sorting them by time, and forming a complete vehicle passage path. After receiving the vehicle passage path, the online billing system excludes abnormal paths and completes the vehicle toll calculation based on the basic rate information. After obtaining the path information and billing information, push them to the toll robot for display. If there are still abnormalities, manual intervention can be used for processing; for special vehicle handling special situations, the cloud customer service system needs to link the preferential exemption system to verify the vehicle information. After confirming that it is correct, the special vehicle handling process will be pushed to the toll robot to guide the car owner to complete the toll collection process.

[0060] For special situations that cannot be effectively identified on site, record the entire processing process, including: manual processing flow, processing mode, and response content, continuously optimize the generated model, improve the post-processing steps, and improve the generalization ability of the model to improve the relevance and accuracy of the answers.

[0061] The present invention is based on the historical vehicle special situation processing records, and uses a graph database or a vector database to construct a general vector data set suitable for the field of highway customer service to generate disposal plans for various special situations. The system will use the natural language processing capabilities of the large language model to fully analyze the user's demands, and combined with the data information collected by the lane, retrieve information fragments related to the problem from a large number of data sources. The system combines the information fragments with the disposal plan and automatically generates a reply text. After system optimization and ensuring the accuracy and logical coherence of the content, the text is converted into speech and sent to the front end together with the disposal plan, realizing voice prompts and executing the corresponding plan at the same time.

[0062] When the user connects to the intercom system, the system will collect the user's voice and various types of information recognized by the toll lane, conduct on-site special situation analysis based on the large language model, and provide corresponding voice prompts according to the analysis results to guide the completion of subsequent processing. Due to the "learning" characteristics of the large language model, for situations that cannot be clearly classified at present, manual intervention will be carried out first. The subsequent system will automatically use user feedback or external evaluation to continuously improve the quality of the data set, adjust the retrieval strategy, optimize the prompt word project, optimize the generation model, or improve the post-processing steps. Gradually realize the "system instead of manual" to complete the lane special situation handling work, and achieve the purpose of reducing costs, increasing efficiency, and rapid handling.

[0063] Speech-to-text conversion is achieved through locally deployed speech recognition engines and hot word configuration, text-to-speech synthesis is achieved through the terminal's speech synthesis engine, semantic analysis and problem resolution are achieved based on a large language model, business processing scenarios are identified through information retrieval, and the corresponding business processing flow is switched based on customer service processing capabilities, thereby identifying user demands and self-service processing of common problems.

[0064] By combining user demands, system knowledge base content and other data information available in the highway system, automatic analysis of special situations can be achieved and corresponding solutions can be proposed.

[0065] Through voice and system process control, users are guided to complete special situation processing on their own, thereby achieving rapid handling of lane special situations.

[0066] If the system still cannot find a matching special situation after multiple conversations with the user and knowledge base searches, it will determine that the special situation cannot be processed automatically and allow manual intervention. After the records are sorted, the system will automatically adjust the search strategy, optimize the prompt word engineering, optimize the generation model, improve the post-processing steps, and further improve the knowledge base.

[0067] A customer service automatic reply system in highway scenarios based on a large language model, including: a toll collection robot, audio and video services, and a cloud customer service system;

[0068] The toll collection robot is equipped with an intercom device, which is connected to the audio and video service through the network. When a special situation occurs in the lane, the user turns on the intercom function of the toll collection robot and describes the current situation. After the toll collection robot transmits the acquired voice data to the audio and video service, it completes the audio analysis and converts the voice file into formatted text data, and saves the original audio and video files locally for later data retrieval.

[0069] The audio and video service forwards the converted formatted text data to the cloud customer service system, which performs semantic analysis on the text based on the large language model;

[0070] The cloud customer service system combines the semantic analysis results and the vehicle data information collected by the toll collection robot to perform information retrieval, classify the special situations and obtain the corresponding process information, and generate the response text and special situation processing process;

[0071] The cloud customer service system pushes the response text and special situation handling process to the toll collection robot, which guides the car owner to complete the toll collection process by converting the response text into voice prompts or interface displays. The system automatically checks and corrects the response text to ensure semantic accuracy and coherence.

[0072] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0073] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A customer service automatic reply method in a highway scenario based on a large language model, characterized in that: include: Obtain the user's audio information and vehicle data information; Converting the audio information into text; Use the natural language processing capabilities of the large language model to perform semantic analysis on the text, and generate a response text and special situation processing flow in combination with vehicle data information; Convert the response text and special situation handling process into audio information for broadcast display.

2. According to claim 1, a method for automatic customer service reply in a highway scenario based on a large language model is characterized in that: The specific process of obtaining the user's audio information is as follows: When a special situation occurs in the lane, the user turns on the intercom function of the toll collection robot and describes the current situation to obtain the user's audio information.

3. According to claim 1, a method for automatic customer service reply in a highway scenario based on a large language model is characterized in that: The large language model specifically includes: Collect historical lane special situation handling data, clean and pre-process the data, and remove invalid data and noise; Based on the pre-processed historical vehicle special situation handling records, a general vector data set for highway customer service is constructed using a graph database or a vector database to generate handling plans for various special situations. Use the natural language processing capabilities of the large language model to perform semantic analysis on the text, and combine it with the data information collected by the lane to obtain information fragments related to the question; Combine information fragments with disposal plans and automatically generate response text and special situation handling procedures.

4. According to claim 3, a method for automatic customer service reply in a highway scenario based on a large language model is characterized in that: The generation of the response text and the special situation processing flow specifically include: Preliminary establishment of special situation sample sets based on business scenarios in the transportation field, and use tools to index text data; Classify the special situation problems mentioned in the text, retrieve the general vector data set of highway customer service, and obtain the special situation handling plan; The natural language processing model is used to generate response text and special situation handling process according to the special situation handling plan and context information.

5. According to the method of automatic customer service reply in a highway scenario based on a large language model described in claim 4, it also includes evaluating the special situation handling situation. Problems that are not automatically identified will automatically generate special situation samples based on the final processing method of the business system. After meeting the evaluation weight, they will be automatically included in the general vector data set of highway customer service, realizing the optimization strategy of self-updating data sets.

6. According to the method of automatic customer service reply in a highway scenario based on a large language model in claim 4, the special situation is divided into operation process, billing dispute, and special vehicle processing; In special cases related to operation procedures, the corresponding operation procedures and response texts are converted into voice prompts or interface displays to guide the car owner to complete the charging process; In case of billing dispute, the external system is linked to obtain the routing information and billing information, and then the information is displayed. If the abnormality still exists, manual intervention is performed. In special vehicle handling situations, the vehicle information will be verified. Once confirmed to be correct, the vehicle owner will be guided to complete the charging process based on the special vehicle handling process.

7. A customer service automatic reply system in a highway scenario based on a large language model, characterized in that: A method for automatically replying customer service in a highway scenario based on a large language model as described in any one of claims 1 to 6 is applied, comprising: a toll collection robot, an audio and video service, and a cloud customer service system; The toll collection robot is equipped with an intercom device, which is connected to the audio and video service through the network. When a special situation occurs in the lane, the user turns on the intercom function of the toll collection robot and describes the current situation. After the toll collection robot transmits the acquired voice data to the audio and video service, it completes the audio analysis and converts the voice file into formatted text data, and saves the original audio and video files locally for later data retrieval. The audio and video service forwards the converted formatted text data to the cloud customer service system, which performs semantic analysis on the text based on the large language model; The cloud customer service system combines the semantic analysis results and the vehicle data information collected by the toll collection robot to perform information retrieval, classify the special situations and obtain the corresponding process information, and generate the response text and special situation processing process; The cloud customer service system pushes the response text and special situation handling process to the toll collection robot, which guides the car owner to complete the toll collection process by converting the response text into voice prompts or interface displays.

Images