Customer service information processing method, customer service system, electronic equipment and storage medium

Abstract

The invention provides a customer service information processing method, a customer service system, electronic equipment and a storage medium. The method comprises the following steps: in a session process with a session object, obtaining a question text of the session object; determining a plurality of sub-question texts corresponding to the question text; retrieving a vector database based on the question text and the plurality of sub-question texts to obtain a retrieval result; the vector database at least comprises text vectors corresponding to multi-modal data; and inputting the retrieval result, the question text, the plurality of sub-question texts and a preset prompt text into a large language model to obtain a target reply text corresponding to the question text, so that the question reply efficiency and accuracy of a customer service system can be improved, and meanwhile, the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a customer service information processing method, a customer service system, an electronic device and a storage medium. Background Art

[0002] Traditional customer service needs to rely on manual responses, and customer service personnel also need to receive targeted training for different fields. With the rapid development of machine learning technology in recent years, using related technologies to build customer service systems has become the choice of more and more companies. The customer service system currently built can only generate corresponding reply texts based on the input question text. The generated reply texts often do not correspond to the question texts, and there is a problem of low accuracy in the customer service system's question responses. In addition, because the built customer service system cannot correctly understand the question text in some cases, it cannot generate the corresponding reply text, and manual processing is required, resulting in low efficiency in the customer service system's question responses. Summary of the invention

[0003] Embodiments of the present application provide a customer service information processing method, a customer service system, an electronic device, and a storage medium to solve one or more problems existing in the related art.

[0004] According to a first aspect of the present application, a customer service information processing method is provided, comprising:

[0005] During the conversation with the conversation object, obtaining the question text of the conversation object;

[0006] Determine a plurality of sub-question texts corresponding to the question text;

[0007] Searching a vector database based on the question text and the multiple sub-question texts to obtain a search result; the vector database at least includes text vectors corresponding to the multimodal data;

[0008] The search result, the question text, the multiple sub-question texts and the preset prompt text are input into a large language model to obtain a target reply text corresponding to the question text.

[0009] According to an embodiment of the present application, before obtaining the question text of the conversation object, the method further includes:

[0010] Acquire multimodal data; the multimodal data includes at least text data and non-text data;

[0011] Performing text segmentation on the text data to obtain first text block data;

[0012] Inputting the non-text data and a preset first prompt text into the multimodal large model to obtain a description text corresponding to the non-text data; the first prompt text is used to prompt the multimodal large model to perform data processing on the non-text data;

[0013] Performing text segmentation on the description text to obtain second text block data;

[0014] Performing vectorization processing on the first text block data and the second text block data to obtain a text vector corresponding to the multimodal data;

[0015] The vector database is constructed based on the text vectors corresponding to the multimodal data.

[0016] According to an implementation manner of the present application, determining a plurality of sub-question texts corresponding to the question text includes:

[0017] The question text and a preset second prompt text are input into a large language model to obtain a plurality of sub-question texts corresponding to the question text; the second prompt text is used to prompt the large language model to decompose the question text according to multiple dimensions.

[0018] According to an implementation manner of the present application, searching a vector database based on the question text and the multiple sub-question texts to obtain a search result includes:

[0019] Performing vectorization processing on the question text and the multiple sub-question texts to obtain a question vector;

[0020] Searching the vector database based on the question vector to obtain a plurality of search text block data;

[0021] The plurality of search text block data are screened for relevance to obtain the search result.

[0022] According to an implementation manner of the present application, searching the vector database based on the question vector to obtain a plurality of search text block data includes:

[0023] Searching the vector database based on the question vector to obtain a plurality of search text block data;

[0024] Determining text similarity between the question vector and the search text block data;

[0025] The plurality of search text block data are screened based on the text similarity to obtain screened search text block data.

[0026] According to an implementation manner of the present application, the relevance screening of the multiple search text block data to obtain the search results includes:

[0027] Inputting a preset third prompt text, the plurality of search text block data, the question text and the plurality of sub-question texts into the large language model to obtain a first correlation between the search text block data and the question text and a second correlation between the search text block data and the sub-question text; the third prompt text is used to prompt the large language model to perform correlation screening on the plurality of search text block data;

[0028] Based on the first relevance and the second relevance, the multiple search text block data are screened for relevance to obtain the search result.

[0029] According to an embodiment of the present application, the method further includes:

[0030] Calling a network interface to perform information retrieval on the question text and the multiple sub-question texts to obtain information retrieval results;

[0031] The fourth prompt text, the information retrieval result, the question text and the multiple sub-question texts are input into the large language model to obtain a target reply text corresponding to the question text; the fourth prompt text is used to prompt the large language model to generate the target reply text.

[0032] According to a second aspect of the present application, a customer service system is provided, including:

[0033] An acquisition module, used for acquiring the question text of the conversation object during the conversation with the conversation object;

[0034] A determination module, used for determining a plurality of sub-question texts corresponding to the question text;

[0035] A retrieval module, configured to search a vector database based on the question text and the plurality of sub-question texts to obtain a retrieval result; the vector database at least includes text vectors corresponding to the multimodal data;

[0036] The reply module is used to input the search result, the question text, the multiple sub-question texts and the preset prompt text into the large language model to obtain the target reply text corresponding to the question text.

[0037] According to a third aspect of the present application, an electronic device is provided, including:

[0038] at least one processor; and

[0039] a memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present application.

[0041] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present application.

[0042] The method of the embodiment of the present application obtains the question text of the conversation object during the conversation with the conversation object; determines multiple sub-question texts corresponding to the question text; searches the vector database based on the question text and the multiple sub-question texts to obtain the search result; the vector database at least includes the text vector corresponding to the multimodal data; inputs the search result, the question text, the multiple sub-question texts and the preset prompt text into the large language model to obtain the target reply text corresponding to the question text. In this way, the efficiency and accuracy of the customer service system's question responses can be improved, while also improving the user experience.

[0043] It should be understood that the teachings of the present application are not required to achieve all of the beneficial effects described above, but specific technical solutions can achieve specific technical effects, and other embodiments of the present application can also achieve beneficial effects not mentioned above. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:

[0045] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0046] Figure 1 The following is a schematic diagram showing the processing flow of the customer service information processing method provided in the embodiment of the present application. Figure 1 ;

[0047] Figure 2 The following is a schematic diagram showing the processing flow of the customer service information processing method provided in the embodiment of the present application. Figure 2 ;

[0048] Figure 3 The following is a schematic diagram showing the processing flow of the customer service information processing method provided in the embodiment of the present application. Figure 3 ;

[0049] Figure 4 The following is a schematic diagram showing the processing flow of the customer service information processing method provided in the embodiment of the present application. Figure 4 ;

[0050] Figure 5 The following is a schematic diagram showing the processing flow of the customer service information processing method provided in the embodiment of the present application. Figure 5 ;

[0051] Figure 6 The following is a schematic diagram showing the processing flow of the customer service information processing method provided in the embodiment of the present application. Figure 6 ;

[0052] Figure 7 An application scenario diagram of the customer service information processing method provided in an embodiment of the present application is shown;

[0053] Figure 8 Another application scenario diagram of the customer service information processing method provided in an embodiment of the present application is shown;

[0054] Fig. 9 An optional schematic diagram of a customer service information processing device provided in an embodiment of the present application is shown;

[0055] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0056] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0057] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0058] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0060] The processing flow of the customer service information processing method provided in the embodiment of the present application is described. Figure 1 , Figure 1 This is a schematic diagram of the processing flow of the customer service information processing method provided in the embodiment of the present application. Figure 1 , will combine Figure 1 Steps S101-S104 are shown for explanation.

[0061] Step S101, during a conversation with a conversation object, obtaining a question text of the conversation object.

[0062] In some embodiments, the question text may include: the question text that the conversation object wants to query through the client provided by the customer service system. The client may include a web page or an application. The embodiment of the present application does not limit the specific client. The conversation process with the conversation object may be a dialogue process between the intelligent customer service and the conversation object. The conversation object may include: a user communicating with the customer service system.

[0063] Step S102, determining a plurality of sub-question texts corresponding to the question text.

[0064] In some embodiments, the sub-question text may include: a plurality of related questions extracted from the question text. The sub-question text may also include: a refined question of the question text. The sub-question text may also include: a question obtained by decomposing the question text from different dimensions.

[0065] As an example, determining multiple sub-question texts corresponding to the question text may include: inputting the question text and a preset second prompt text into a large language model to obtain multiple sub-question texts corresponding to the question text. The second prompt text may be used to prompt the large language model to decompose the question text according to multiple dimensions. The large language model may include: a natural language model. Specifically, the large language model may be a Qwen-max model.

[0066] In some embodiments, the question text may be: "I need to know how to maintain my laptop." The second prompt text may include: Please decompose the question text according to the following dimensions: hardware maintenance, software maintenance, daily use suggestions. The sub-question text generated by the large language model may include:

[0067] Hardware Maintenance: "How to clean the hardware parts of a laptop?"

[0068] Software Maintenance: "How do I update my laptop's operating system?"

[0069] Daily use advice: "How to properly shut down your laptop to extend its life?"

[0070] Step S103, searching the vector database based on the question text and the multiple sub-question texts to obtain search results; the vector database at least includes text vectors corresponding to the multimodal data.

[0071] In some embodiments, the vector database may include text vectors corresponding to multimodal data. The vector database can be used to store text vectors, and the vector database can be pre-constructed using multimodal data. Multimodal data may include: data in multiple forms. Specifically, multimodal data may include: text, images, audio, tables and other data. The embodiments of the present application do not limit specific multimodal data. The text vector may include: a representation in vector form converted from the text. The vector database is searched based on the text vectors corresponding to the question text and multiple sub-question texts to obtain a search result.

[0072] Step S104, input the search results, question text, multiple sub-question texts and preset prompt text into the large language model to obtain the target response text corresponding to the question text.

[0073] In some embodiments, the preset prompt text may include: a text that is preset to be input into the large language model. The prompt text may be used to guide the model to generate a specific type of output. Different prompt texts are used for different tasks. The target reply text may include: reply information generated for the question text.

[0074] For example, the question text may be: "I need to know how to maintain my laptop." The corresponding prompt text may include: Please provide a step-by-step guide to maintaining the laptop based on data A, data B, and data C, and organize the answers in the order of importance of the question. Data A is the search result, data B is the question text, and data C is multiple sub-question texts.

[0075] The target response information generated may include: Maintaining your laptop is a comprehensive process, including hardware maintenance, software maintenance, and daily use recommendations. The following are specific recommendations for each aspect: Hardware maintenance: To clean the hardware components of the laptop, first turn off the power and unplug all external devices. Use a soft dry cloth to gently wipe the surface, please make sure to take anti-static measures and do it under the guidance of a professional. Software maintenance: Updating the operating system of the laptop is key to maintaining its performance and security. Usually, you can find the update option through the system's settings menu. Daily use recommendations: In order to extend the life of the laptop, it is recommended that you shut down the device properly when not in use. Click the "Start" button, select the "Power" icon, and then select "Shut down". Avoid using the computer when the battery is low, try to charge when the battery is less than 20%, and unplug the charger when it is full.

[0076] The method of the embodiment of the present application improves the efficiency and accuracy of the customer service system in generating reply texts by integrating multimodal data processing, a large language model, and vector database retrieval technology. The sub-question text corresponding to the question text is generated by a large language model, and then the vector database is accurately searched based on the question text and the sub-question text, thereby improving the efficiency and accuracy of the customer service system in answering questions. Intelligent screening is performed based on the similarity between the question vector and the retrieved text block data to ensure the relevance of the reply text. An accurate target reply text is generated by combining a large language model with a preset prompt text, effectively improving the user experience. In addition, the network interface is actively called for information retrieval to further enrich the content of the reply text, making the customer service information processing process more efficient, improving the efficiency and accuracy of the customer service system in answering questions, and at the same time improving the user experience.

[0077] In some embodiments, the processing flow of the customer service information processing method is as follows: Figure 2 ,like Figure 2 As shown, including:

[0078] Step S201, acquiring multimodal data.

[0079] Step S202, segmenting the text data to obtain first text block data.

[0080] In this embodiment, the multimodal data includes at least text data and non-text data. The text data may include: product information, manual reply information and other text data, and the format of the text data may include: text data in doc format, docx format and pdf format. The format of the text data may also include other text formats, and the embodiment of the present application does not limit the specific format of the text data. Text segmentation may include: the process of dividing the original text data into text block data according to the set rules.

[0081] Step S203: input the non-text data and the preset first prompt text into the multimodal large model to obtain a description text corresponding to the non-text data.

[0082] In this embodiment, the first prompt text is used to prompt the multimodal large model to process the non-text data. The non-text data may include: data in the format of images, sounds, videos, tables, etc. The embodiment of the present application does not limit the specific non-text data. The multimodal large model may include: a deep learning model capable of analyzing non-text data. The specific multimodal large model may be Qwen-VL. The descriptive text corresponding to the non-text data may include: a text description of the non-text data.

[0083] Step S204, segmenting the description text to obtain second text block data.

[0084] In this embodiment, text segmentation of the description text may include: a process of segmenting the text data of the description text into text block data according to a set rule.

[0085] Step S205, vectorizing the first text block data and the second text block data to obtain text vectors corresponding to the multimodal data.

[0086] Step S206, constructing a vector database based on the text vectors corresponding to the multimodal data.

[0087] In this embodiment, the text vector may include: a numerical vector obtained after the text data is vectorized. The vectorization process may include: a process of converting text data into a numerical vector through an Embedding (vectorization) model. The specific Embedding model may be a bge-large-zh-v1.5 model. The obtained text vector is stored in a vector database to construct a vector database. The vector database may include text vectors corresponding to multimodal data. The vector database may be used to store and retrieve text vectors.

[0088] The method of the embodiment of the present application improves the efficiency and accuracy of the customer service system in generating reply texts by integrating multimodal data processing, a large language model, and vector database retrieval technology. The sub-question text corresponding to the question text is generated by a large language model, and then the vector database is accurately searched based on the question text and the sub-question text, thereby improving the efficiency and accuracy of the customer service system in answering questions. Intelligent screening is performed based on the similarity between the question vector and the retrieved text block data to ensure the relevance of the reply text. An accurate target reply text is generated by combining a large language model with a preset prompt text, effectively improving the user experience. In addition, the network interface is actively called for information retrieval to further enrich the content of the reply text, making the customer service information processing process more efficient, improving the efficiency and accuracy of the customer service system in answering questions, and at the same time improving the user experience.

[0089] In some embodiments, the processing flow of the customer service information processing method is as follows: Figure 3 ,like Figure 3 As shown, in step S103, the vector database is searched based on the question text and the multiple sub-question texts to obtain the search results, which may specifically include:

[0090] Step S301, vectorize the question text and multiple sub-question texts to obtain a question vector.

[0091] Step S302: searching the vector database based on the question vector to obtain a plurality of search text block data.

[0092] Step S303, performing relevance screening on the multiple search text block data to obtain search results.

[0093] In this embodiment, the question vector may include: a numerical vector obtained after the question text and the sub-question text are vectorized. Retrieval methods may include: HNSW (Hierarchical Navigable Small World graphs) and cosine similarity retrieval. The embodiment of the present application does not limit the specific retrieval method. The vector database is searched based on the question vector to obtain the corresponding text vector, and the corresponding retrieval text block data is determined according to the text vector. The correlation between each retrieved retrieval text block data and the question vector is evaluated through a large language model, and multiple retrieval text block data are screened for correlation to obtain a retrieval result.

[0094] The method of the embodiment of the present application improves the efficiency and accuracy of the customer service system in generating reply texts by integrating multimodal data processing, a large language model, and vector database retrieval technology. The sub-question text corresponding to the question text is generated by a large language model, and then the vector database is accurately searched based on the question text and the sub-question text, thereby improving the efficiency and accuracy of the customer service system in answering questions. Intelligent screening is performed based on the similarity between the question vector and the retrieved text block data to ensure the relevance of the reply text. An accurate target reply text is generated by combining a large language model with a preset prompt text, effectively improving the user experience. In addition, the network interface is actively called for information retrieval to further enrich the content of the reply text, making the customer service information processing process more efficient, improving the efficiency and accuracy of the customer service system in answering questions, and at the same time improving the user experience.

[0095] In some embodiments, the processing flow of the customer service information processing method is as follows: Figure 4 ,like Figure 4 As shown, in step S302, the vector database is searched based on the question vector to obtain multiple search text block data, which may specifically include:

[0096] Step S401, searching a vector database based on a question vector to obtain a plurality of search text block data.

[0097] Step S402, determining the text similarity between the question vector and the search text block data.

[0098] Step S403, screening multiple search text block data based on text similarity to obtain screened search text block data.

[0099] As an example, in a vector database, retrieve text vectors similar to the question vector to obtain text block data corresponding to the text vector. Calculate the cosine similarity between the question vector and each retrieved retrieval text block data. Determine the TopK retrieval text block data with the highest similarity, where K can be any positive integer. Reorder the K retrieval text block data based on the reordering model to obtain filtered retrieval text block data. The specific reordering model may include a bge-reanker-large model. The embodiments of the present application do not limit the specific reordering model.

[0100] The method of the embodiment of the present application improves the efficiency and accuracy of the customer service system in generating reply texts by integrating multimodal data processing, a large language model, and vector database retrieval technology. The sub-question text corresponding to the question text is generated by a large language model, and then the vector database is accurately searched based on the question text and the sub-question text, thereby improving the efficiency and accuracy of the customer service system in answering questions. Intelligent screening is performed based on the similarity between the question vector and the retrieved text block data to ensure the relevance of the reply text. An accurate target reply text is generated by combining a large language model with a preset prompt text, effectively improving the user experience. In addition, the network interface is actively called for information retrieval to further enrich the content of the reply text, making the customer service information processing process more efficient, improving the efficiency and accuracy of the customer service system in answering questions, and at the same time improving the user experience.

[0101] In some embodiments, the processing flow of the customer service information processing method is as follows: Figure 5 ,like Figure 5 As shown, the relevance screening of the multiple search text block data in step S303 to obtain the search results may include:

[0102] Step S501, input a preset third prompt text, multiple search text block data, question text and multiple sub-question texts into a large language model to obtain a first correlation between the search text block data and the question text and a second correlation between the search text block data and the sub-question text.

[0103] Step S502: Based on the first relevance and the second relevance, multiple search text block data are screened for relevance to obtain search results.

[0104] In this embodiment, the third prompt text can be used to prompt the large language model to perform relevance screening on multiple search text block data. The first relevance can include a relevance score between the search text block data and the question text. The second relevance can include a relevance score between the search text block data and the sub-question text.

[0105] As an example, the preset third prompt text may include: integrating the first data, the second data and the third data, determining the correlation between the third data and the first data, and determining the correlation between the third data and the second data. The first data may be a question text, the second data may be a sub-question text, and the third data may be a retrieval text block data. After the large language model processes the input data according to the third prompt text, it will output the correlation score between each retrieval text block data and the question text, as well as the correlation score between the retrieval text block data and the sub-question text. A scoring threshold is set to filter out the text block data whose score corresponding to the first correlation or the score corresponding to the second correlation exceeds this threshold. The retrieval text block data whose first correlation or second correlation exceeds the scoring threshold is used as the retrieval result. If there is no retrieval text block data whose first correlation or second correlation exceeds the scoring threshold, the retrieval result is empty.

[0106] The method of the embodiment of the present application improves the efficiency and accuracy of the customer service system in generating reply texts by integrating multimodal data processing, a large language model, and vector database retrieval technology. The sub-question text corresponding to the question text is generated by a large language model, and then the vector database is accurately searched based on the question text and the sub-question text, thereby improving the efficiency and accuracy of the customer service system in answering questions. Intelligent screening is performed based on the similarity between the question vector and the retrieved text block data to ensure the relevance of the reply text. An accurate target reply text is generated by combining a large language model with a preset prompt text, effectively improving the user experience. In addition, the network interface is actively called for information retrieval to further enrich the content of the reply text, making the customer service information processing process more efficient, improving the efficiency and accuracy of the customer service system in answering questions, and at the same time improving the user experience.

[0107] In some embodiments, the processing flow of the customer service information processing method is as follows: Figure 6 ,like Figure 6 As shown, the customer service information processing method may also include:

[0108] Step S601, calling a network interface to perform information retrieval on a question text and a plurality of sub-question texts to obtain information retrieval results.

[0109] Step S602: input the fourth prompt text, the information retrieval result, the question text and the multiple sub-question texts into the large language model to obtain the target reply text corresponding to the question text.

[0110] In this embodiment, the fourth prompt text can be used to prompt the large language model to generate a target reply text. The information retrieval results may include: a collection of various information related to the question text and the sub-question text retrieved through the network interface. The network interface may include: an API (Application Programming Interface) interface. By calling the API interface, information retrieval can be performed on the question text and multiple sub-question texts to obtain information retrieval results. The information retrieval results may include: information such as web pages, texts, and pictures. The information retrieval results may also include other information, which is not limited in the embodiments of the present application.

[0111] The method of the embodiment of the present application improves the efficiency and accuracy of the customer service system in generating reply texts by integrating multimodal data processing, a large language model, and vector database retrieval technology. The sub-question text corresponding to the question text is generated by a large language model, and then the vector database is accurately searched based on the question text and the sub-question text, thereby improving the efficiency and accuracy of the customer service system in answering questions. Intelligent screening is performed based on the similarity between the question vector and the retrieved text block data to ensure the relevance of the reply text. An accurate target reply text is generated by combining a large language model with a preset prompt text, effectively improving the user experience. In addition, the network interface is actively called for information retrieval to further enrich the content of the reply text, making the customer service information processing process more efficient, improving the efficiency and accuracy of the customer service system in answering questions, and at the same time improving the user experience.

[0112] refer to Figure 7 , an application scenario diagram of the customer service information processing method provided in the embodiment of the present application, which is applied to a customer service system. Including: an intelligent customer service system. Among them, the intelligent customer service system may include: a client, a knowledge base, a question rewriting module, an information retrieval module and a reply generation module.

[0113] The client can be used to: receive the question text of the session object and display the generated target reply text.

[0114] The knowledge base can be pre-constructed, and the construction process may include: the data contained in the data set includes product information, manual response information, etc. The text data in the data set is segmented to obtain the first text block data. For the non-text data in the data set, the preset prompt (prompt text) is input into the multimodal large model together with the non-text data to generate a description text for the non-text data, and the description text is segmented into the second text block data. The prompt here is used to prompt the multimodal large model to process the non-text data to obtain the description text. Through the Embedding model, the first text block data and the second text block data are vectorized and converted into vector form using the Embedding model to obtain the corresponding text vector, and the knowledge base is constructed based on the text vector.

[0115] The question rewriting module can be used to: utilize a large language model to decompose the question text raised by the conversation partner into sub-question texts from different perspectives.

[0116] The information retrieval module can be used to search the knowledge base based on the question text and multiple sub-question texts to obtain retrieval results.

[0117] The reply generation module can be used to combine the question text, sub-question text and search results with the preset prompt, input them into the large language model, and obtain the final target reply text. The prompt here is used to prompt the large language model to generate the reply text.

[0118] refer to Figure 8 , another application scenario diagram of the customer service information processing method provided in the embodiment of the present application, which is applied to the customer service information processing of the intelligent customer service system. It includes:

[0119] Pre-construct a vector database: obtain an original data set, which includes multimodal data. Multimodal data includes at least text data and other modal data. Perform text segmentation on the text data to obtain a first text block data. Input other modal data and a preset prompt into the multimodal large model to obtain a description text corresponding to the other modal data. Here, prompt is used to prompt the multimodal large model to perform data processing on non-text data. Perform text segmentation on the description text to obtain a second text block data. Perform vectorization processing on the first text block data and the second text block data to obtain a text vector corresponding to the multimodal data. Construct a vector database based on the text vector corresponding to the multimodal data.

[0120] Reply generation: First, obtain the user's original question. Input the original question and the preset prompt into the large language model to obtain multiple sub-questions corresponding to the original question. Here, prompt is used to prompt the large language model to decompose the original question according to multiple dimensions. The original question and multiple sub-questions are vectorized to obtain a question vector. The vector database is searched based on the question vector to obtain a search result, which includes multiple search text block data. The multiple search text block data are sorted using the reordering model, and a preset number of search text block data are retained.

[0121] Input the preset prompt, the original question and multiple sub-questions into the large language model to obtain the first correlation between the retrieval text block data and the original question and the second correlation between the retrieval text block data and the sub-questions; the prompt is used to prompt the large language model to perform correlation screening on multiple retrieval text block data. Based on the first correlation and the second correlation, multiple retrieval text block data are relevance screened to obtain screened retrieval results. The retrieval text block data whose first correlation or second correlation exceeds the preset scoring threshold is used as the screened retrieval result. If there is no retrieval text block data whose first correlation or second correlation exceeds the scoring threshold, the screened retrieval result is empty.

[0122] If the filtered search results are not empty, the filtered search results, the original question, multiple sub-questions and the preset prompt are input into the large language model to obtain the target reply text corresponding to the original question. The prompt here is used to generate the target reply text corresponding to the original question.

[0123] If the search results after screening are empty, the network interface is called to perform information retrieval on the original question and multiple sub-questions to obtain information retrieval results. The preset prompt, information retrieval results, original question and multiple sub-questions are input into the large language model to obtain the target response text corresponding to the original question. The prompt here is used to prompt the large language model to generate the target response text.

[0124] The following is a description of the exemplary structure of the software modules included in the customer service system 90 provided in the embodiment of the present application. In some embodiments, Fig. 9As shown, the customer service system 90 may include: an acquisition module 901, which is used to acquire the question text of the conversation object during the conversation with the conversation object; a determination module 902, which is used to determine multiple sub-question texts corresponding to the question text; a retrieval module 903, which is used to search the vector database based on the question text and the multiple sub-question texts to obtain the retrieval results; the vector database at least includes the text vector corresponding to the multimodal data; a reply module 904, which is used to input the retrieval results, the question text, the multiple sub-question texts and the preset prompt text into the large language model to obtain the target reply text corresponding to the question text.

[0125] In some embodiments, the customer service system 90 may further include a database construction module, which may be used to: obtain multimodal data; the multimodal data includes at least text data and non-text data; perform text segmentation on the text data to obtain first text block data; input the non-text data and a preset first prompt text into the multimodal large model to obtain a description text corresponding to the non-text data; the first prompt text is used to prompt the multimodal large model to perform data processing on the non-text data; perform text segmentation on the description text to obtain second text block data; perform vectorization processing on the first text block data and the second text block data to obtain text vectors corresponding to the multimodal data; and construct a vector database based on the text vectors corresponding to the multimodal data.

[0126] In some embodiments, the determination module 902 can be used to: input the question text and the preset second prompt text into the large language model to obtain multiple sub-question texts corresponding to the question text; the second prompt text is used to prompt the large language model to decompose the question text according to multiple dimensions.

[0127] In some embodiments, the retrieval module 903 can be used to: vectorize the question text and multiple sub-question texts to obtain a question vector; search the vector database based on the question vector to obtain multiple retrieval text block data; and perform relevance screening on the multiple retrieval text block data to obtain retrieval results.

[0128] In some embodiments, the retrieval module 903 can be used to: search the vector database based on the question vector to obtain multiple retrieval text block data; determine the text similarity between the question vector and the retrieval text block data; filter the multiple retrieval text block data based on the text similarity to obtain filtered retrieval text block data.

[0129] In some embodiments, the retrieval module 903 can be used to: input a preset third prompt text, multiple retrieval text block data, a question text, and multiple sub-question texts into a large language model to obtain a first correlation between the retrieval text block data and the question text and a second correlation between the retrieval text block data and the sub-question text; the third prompt text is used to prompt the large language model to perform relevance screening on multiple retrieval text block data; based on the first correlation and the second correlation, multiple retrieval text block data are screened for relevance to obtain retrieval results.

[0130] In some embodiments, the customer service system 90 may further include a search module, which may be used to: call a network interface to perform information retrieval on a question text and multiple sub-question texts to obtain information retrieval results; input a fourth prompt text, information retrieval results, question text, and multiple sub-question texts into a large language model to obtain a target reply text corresponding to the question text; and the fourth prompt text is used to prompt the large language model to generate a target reply text.

[0131] It should be noted that the description of the system in the embodiment of the present application is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment, so it will not be repeated. Figures 1 to 8 The present invention can be understood by referring to the description of any one of the accompanying drawings.

[0132] According to an embodiment of the present application, the present application also provides an electronic device and a non-transitory computer-readable storage medium.

[0133] Fig.10 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0134] like Fig.10As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0135] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0136] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the customer service information processing method. For example, in some embodiments, the customer service information processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the customer service information processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the customer service information processing method in any other appropriate manner (e.g., by means of firmware).

[0137] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0139] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0141] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0142] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0143] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0144] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A customer service information processing method, characterized in that: include: During the conversation with the conversation object, obtaining the question text of the conversation object; Determine a plurality of sub-question texts corresponding to the question text; Searching a vector database based on the question text and the multiple sub-question texts to obtain a search result; the vector database at least includes text vectors corresponding to the multimodal data; The search result, the question text, the multiple sub-question texts and the preset prompt text are input into a large language model to obtain a target reply text corresponding to the question text.

2. The method according to claim 1, characterized in that Before obtaining the question text of the conversation object, the method further includes: Acquire multimodal data; the multimodal data includes at least text data and non-text data; Performing text segmentation on the text data to obtain first text block data; Inputting the non-text data and a preset first prompt text into the multimodal large model to obtain a description text corresponding to the non-text data; the first prompt text is used to prompt the multimodal large model to perform data processing on the non-text data; Performing text segmentation on the description text to obtain second text block data; Performing vectorization processing on the first text block data and the second text block data to obtain a text vector corresponding to the multimodal data; The vector database is constructed based on the text vectors corresponding to the multimodal data.

3. The method according to claim 1, characterized in that The step of determining a plurality of sub-question texts corresponding to the question text comprises: The question text and a preset second prompt text are input into a large language model to obtain a plurality of sub-question texts corresponding to the question text; the second prompt text is used to prompt the large language model to decompose the question text according to multiple dimensions.

4. The method according to claim 1, characterized in that The step of searching a vector database based on the question text and the plurality of sub-question texts to obtain a search result includes: Performing vectorization processing on the question text and the multiple sub-question texts to obtain a question vector; Searching the vector database based on the question vector to obtain a plurality of search text block data; The plurality of search text block data are screened for relevance to obtain the search result.

5. The method according to claim 4, characterized in that The searching of the vector database based on the question vector to obtain a plurality of search text block data includes: Searching the vector database based on the question vector to obtain a plurality of search text block data; Determining text similarity between the question vector and the search text block data; The plurality of search text block data are screened based on the text similarity to obtain screened search text block data.

6. The method according to claim 4, characterized in that The relevance screening of the plurality of search text block data to obtain the search result includes: Inputting a preset third prompt text, the plurality of search text block data, the question text and the plurality of sub-question texts into the large language model to obtain a first correlation between the search text block data and the question text and a second correlation between the search text block data and the sub-question text; the third prompt text is used to prompt the large language model to perform correlation screening on the plurality of search text block data; Based on the first relevance and the second relevance, the multiple search text block data are screened for relevance to obtain the search result.

7. The method according to claim 1, characterized in that The method further comprises: Calling a network interface to perform information retrieval on the question text and the multiple sub-question texts to obtain information retrieval results; The fourth prompt text, the information retrieval result, the question text and the multiple sub-question texts are input into the large language model to obtain a target reply text corresponding to the question text; the fourth prompt text is used to prompt the large language model to generate the target reply text.

8. A customer service system, characterized in that: include: An acquisition module, used for acquiring the question text of the conversation object during the conversation with the conversation object; A determination module, used for determining a plurality of sub-question texts corresponding to the question text; A retrieval module, configured to search a vector database based on the question text and the plurality of sub-question texts to obtain a retrieval result; the vector database at least includes text vectors corresponding to the multimodal data; The reply module is used to input the search result, the question text, the multiple sub-question texts and the preset prompt text into the large language model to obtain the target reply text corresponding to the question text.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.

Images