Information generation method, information generation device and electronic equipment

Abstract

The embodiment of the invention discloses an information generation method, an information generation device and electronic equipment. A specific embodiment of the method comprises the following steps: in response to a received information consultation request sent by a user, vectorizing the information consultation request to obtain a request vector; according to the request vector, a target knowledge fragment matched with the information consultation request is extracted from a preset knowledge base, and knowledge fragment vectors of related business knowledge are stored in the knowledge base; and according to the information consultation request and the target knowledge fragment, performing data splicing by adopting a method based on language prompt, and generating reply information of the information consultation request. The implementation mode is related to an automatic reply technology of a marketing scene, knowledge fragments can be extracted and rewritten under the condition of externally connecting a knowledge base on the basis of retrieval enhancement and generation capability of a large model, so that more accurate and controllable reply is generated, and the method is particularly suitable for automatic reply in a private domain marketing scene.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the technical field of automatic reply in marketing scenarios, and in particular to an information generation method, an information generation device, and an electronic device. Background Art

[0002] In recent years, as online traffic has gradually dried up and the cost of incremental traffic has risen, major companies and brands have begun to accelerate the construction and operation of their own private domains. The core purpose is to deposit public domain users into specific chat groups or programs that companies / brands can reach at any time and at low cost, so that companies / brands can conduct efficient marketing reach and improve conversion and repeat purchases. Use technical means to build a private domain traffic field and automatically push marketing content to reach fans. However, as the number of fans in the private domain increases, automatic replies have become an issue that cannot be ignored while automatically sending.

[0003] However, the inventors found that some of the related technologies basically rely on manpower to complete. That is, by monitoring the comments and content of fans in the private domain community, and then processing them according to pre-set strategies. As the size of the community grows, manual response will not only increase costs, but also affect the timeliness of the response.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose an information generation method, an information generation device, an electronic device, a computer-readable medium, and a computer program product to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide an information generation method, including: in response to receiving an information consultation request sent by a user, vectorizing the information consultation request to obtain a request vector; based on the request vector, extracting a target knowledge fragment matching the information consultation request from a preset knowledge base, wherein the knowledge base stores knowledge fragment vectors of relevant business knowledge; based on the information consultation request and the target knowledge fragment, using a language prompt-based method to perform data splicing to generate reply information for the information consultation request.

[0008] In some embodiments, based on the request vector, target knowledge fragments matching the information consultation request are extracted from a preset knowledge base, including: determining the similarity value between each knowledge fragment vector and the request vector in the preset knowledge base; extracting a first number of knowledge fragments from the knowledge base in descending order of similarity value as recalled knowledge fragments; adjusting the sorting of recalled knowledge fragments according to preset rules, and extracting a second number of recalled knowledge fragments therefrom according to the adjusted sorting as target knowledge fragments, wherein the second number is less than or equal to the first number.

[0009] In some embodiments, the order of recalled knowledge fragments is adjusted according to preset rules, including: sorting according to the similarity values ​​of each recalled knowledge fragment, and re-adjusting the order of each recalled knowledge fragment based on the time difference between the time of writing into the knowledge base and the current time, wherein the smaller the time difference between the recalled knowledge fragment and the current time, the greater the adjusted weight.

[0010] In some embodiments, adjusting the order of recalled knowledge fragments according to preset rules also includes: traversing each recalled knowledge fragment in the recalled knowledge fragments, and adjusting the order of each recalled knowledge fragment using a maximum boundary correlation algorithm based on the similarity value between the recalled knowledge fragment and the request vector, and the similarity value between the recalled knowledge fragment and the traversed recalled knowledge fragments, wherein the recalled knowledge fragment that has a high similarity with the request vector and is different from the traversed knowledge fragments has a larger adjusted weight.

[0011] In some embodiments, the knowledge base is obtained by the following method: extracting text elements from original data describing relevant business knowledge, wherein the original data includes at least one of a web page, a document, and a page; splitting the extracted text elements into at least one knowledge fragment according to the layout structure of the elements in the original data; vectorizing at least one knowledge fragment to obtain at least one knowledge fragment vector, and storing at least one knowledge fragment vector to obtain a knowledge base.

[0012] In some embodiments, a language prompt-based method is used to perform data splicing according to an information consultation request and a target knowledge fragment to generate reply information for the information consultation request, including: verifying the target knowledge fragment according to preset conditions; in response to determining that the target knowledge fragment has passed the verification, using a language prompt-based method to splice the information consultation request and the target knowledge fragment according to a preset format; inputting the spliced ​​data into a preset model to generate reply information for the information consultation request; in response to determining that the target knowledge fragment has failed the verification, sending the information consultation request to the target terminal for manual reply.

[0013] In some embodiments, the method further includes: using a pre-trained verification model to perform factual verification on the generated reply information, wherein the verification model is trained through training data, the training data uses sample information consultation requests and corresponding sample reply information as sample data, and uses user feedback on the sample reply information as sample labels; in response to determining that the factual verification has passed, sending the reply information to the user.

[0014] In some embodiments, the method further includes: vectorizing the reply information of the information consultation request to obtain a new knowledge fragment vector; and storing the new knowledge fragment vector in the knowledge base to update the knowledge base.

[0015] In a second aspect, some embodiments of the present disclosure provide an information generating device, comprising: a request processing unit, configured to, in response to receiving an information consultation request sent by a user, perform vectorized processing on the information consultation request to obtain a request vector; a knowledge extraction unit, configured to extract a target knowledge fragment matching the information consultation request from a preset knowledge base according to the request vector, wherein the knowledge base stores knowledge fragment vectors of relevant business knowledge; a reply information generating unit, configured to perform data splicing based on a language prompt method according to the information consultation request and the target knowledge fragment, to generate reply information for the information consultation request.

[0016] In some embodiments, the knowledge extraction unit is further configured to determine the similarity value between each knowledge fragment vector and the request vector in a preset knowledge base; extract a first number of knowledge fragments from the knowledge base in order of similarity value from high to low as recalled knowledge fragments; adjust the sorting of the recalled knowledge fragments according to preset rules, and extract a second number of recalled knowledge fragments from them according to the adjusted sorting as target knowledge fragments, wherein the second number is less than or equal to the first number.

[0017] In some embodiments, the knowledge extraction unit is further configured to readjust the order of each recalled knowledge fragment based on the similarity value of each recalled knowledge fragment and the time difference between the time of writing into the knowledge base and the current time, wherein the smaller the time difference between the recalled knowledge fragment and the current time, the greater the adjusted weight.

[0018] In some embodiments, the knowledge extraction unit is further configured to traverse each recalled knowledge fragment in the recalled knowledge fragments, and adjust the sorting of each recalled knowledge fragment using the maximum boundary correlation algorithm according to the similarity value between the recalled knowledge fragment and the request vector, as well as the similarity value between the recalled knowledge fragment and the recalled knowledge fragment that has been traversed, wherein the recalled knowledge fragment that has a high similarity with the request vector and is different from the recalled knowledge fragment that has been traversed has a larger adjusted weight.

[0019] In some embodiments, the information generating device also includes a knowledge base generating unit, which is configured to extract text elements from original data describing relevant business knowledge, wherein the original data includes at least one of a web page, a document, and a page; according to the layout structure of the elements in the original data, the extracted text elements are split into at least one knowledge fragment; at least one knowledge fragment is vectorized to obtain at least one knowledge fragment vector, and at least one knowledge fragment vector is stored to obtain a knowledge base.

[0020] In some embodiments, the reply information generation unit is further configured to verify the target knowledge fragment according to preset conditions; in response to determining that the target knowledge fragment has passed the verification, using a language prompt-based method to splice the information consultation request and the target knowledge fragment in accordance with a preset format; inputting the spliced ​​data into a preset model to generate reply information for the information consultation request; in response to determining that the target knowledge fragment has not passed the verification, sending the information consultation request to the target terminal for manual reply.

[0021] In some embodiments, the information generating device also includes a reply information verification unit, which is configured to use a pre-trained verification model to perform factual verification on the generated reply information, wherein the verification model is trained through training data, the training data uses sample information consultation requests and corresponding sample reply information as sample data, and uses user feedback on the sample reply information as sample labels; in response to determining that the factual verification has passed, the reply information is sent to the user.

[0022] In some embodiments, the information generating device further includes a knowledge base updating unit configured to vectorize the reply information of the information consultation request to obtain a new knowledge fragment vector; and store the new knowledge fragment vector in the knowledge base to update the knowledge base.

[0023] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the information generation method described in any implementation method in the above-mentioned first aspect.

[0024] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the information generating method described in any implementation manner in the above-mentioned first aspect is implemented.

[0025] In a fifth aspect, some embodiments of the present disclosure provide a computer program product, including a computer program, which, when executed by a processor, implements the information generation method described in any implementation manner in the above-mentioned first aspect.

[0026] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the information generation method of some embodiments of the present disclosure can extract knowledge fragments from a pre-built knowledge base based on the retrieval enhancement and generation capabilities of the large model. Thus, by splicing through the prompt method, it can be rewritten into a more accurate, controllable, and anthropomorphic reply. Since the knowledge fragment vectors of relevant business knowledge are pre-stored in the knowledge base, when extracting knowledge fragments, only similarity calculation is required to complete the recall of knowledge fragments. No complex data recognition analysis is required. In this way, while ensuring the accuracy of data extraction, it can not only simplify the processing process, reduce complexity, and improve data processing efficiency, but also reduce resource requirements. In addition, by using the prompt method for data splicing, reply information of the required style or content can be generated. Thereby achieving the controllability of reply information and improving the accuracy of reply information generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0028] Figure 1 This is a schematic diagram of the architecture of a traditional question-answering system;

[0029] Figure 2 are flow charts of some embodiments of the information generation method of the present disclosure;

[0030] Figure 3 are flow charts of other embodiments of the information generating method disclosed herein;

[0031] Figure 4A It is a schematic diagram of some application scenarios of the information generation method disclosed in the present invention;

[0032] Figure 4B It is a schematic diagram of some embodiments of unstructured knowledge of web pages;

[0033] Figure 4C This is a schematic diagram of the information categories in the express delivery private domain group;

[0034] Figure 4D This is a comparison chart of the test results;

[0035] Figure 5 is a schematic diagram of the structure of some embodiments of the information generating device disclosed in the present invention;

[0036] Figure 6 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0037] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0038] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0039] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0040] In addition, the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0041] Figure 2 A process 200 of some embodiments of the information generation method according to the present disclosure is shown. The method may include the following steps:

[0042] Step 201, in response to receiving an information consultation request sent by a user, vectorize the information consultation request to obtain a request vector.

[0043] In some embodiments, the execution subject (question-and-answer server) of the information generation method of the present disclosure can communicate with other electronic devices (such as user terminals or knowledge databases, etc.) through wired or wireless connections. Here, various applications can be installed on the user terminal, such as shopping applications, chat applications, etc. Users can consult information through corresponding functional modules (such as contacting merchants) in shopping applications. Alternatively, users can join a group (such as a courier service group, etc.) in the chat application, and then consult information in the group. Figure 4A As shown, users can send a query: "Can we send large items?"

[0044] When the execution subject receives an information consultation request sent by a user, it can perform vectorization (embedding) processing on the information consultation request to obtain a request vector for the information consultation request. As an example, the execution subject can input the information consultation request into a large model, thereby outputting a vector representation of the input data. The large model here can adopt various language model structures, such as GPT-3 (artificial intelligence language model), LLM (Large Language Model), etc. Optionally, GPT-3 can be used as the basic model, and vectorization processing can be performed through the lang Chain tool to obtain a vector with a word embedding dimension of 1536. Among them, lang Chain is usually a programming framework that helps use large language models in applications. It can be understood that with the help of the deep semantic understanding ability of the large model, the request vector represents the deep semantic information of the user's request and can be calculated numerically.

[0045] Step 202: extracting target knowledge fragments matching the information consultation request from a preset knowledge base according to the request vector.

[0046] In some embodiments, according to the request vector obtained in step 201, the execution subject can extract the target knowledge fragment that matches the information consultation request from the preset knowledge base. Among them, the knowledge base stores the knowledge fragment vectors of relevant business knowledge. As an example, the execution subject can perform similarity calculations on each knowledge fragment vector in the knowledge base and the request vector respectively. Thus, several knowledge fragments with higher similarity values ​​can be used as target knowledge fragments. The similarity calculation method here can be set according to actual needs, such as using dot product or cosine distance.

[0047] In some optional implementations, first, the execution subject may determine the similarity value between each knowledge segment vector and the request vector in the preset knowledge base. For example, the similarity calculation may be in the form of a dot product to calculate the similarity between the user's request vector and the knowledge segment vector in the knowledge base:

[0048] sim ij =q i ·a j ;

[0049] Where q represents the user's information consultation request; q i represents the i-th request vector; a represents the knowledge fragment; a j Represents the j-th knowledge fragment vector.

[0050] Next, a first number of knowledge fragments may be extracted from the knowledge base in descending order of similarity values ​​as recalled knowledge fragments. Figure 4AThe recall layer of the knowledge extraction stage in the process. That is, the knowledge fragments that are most relevant to the request and helpful in answering the question are recalled. Afterwards, the order of the recalled knowledge fragments can be adjusted according to the preset rules, and the second number of recalled knowledge fragments can be extracted from them according to the adjusted order as the target knowledge fragments. Figure 4A The sorting layer of the knowledge extraction stage in the middle. Among them, the second number can usually be less than or equal to the first number. The preset rules here can also be set according to actual needs. In this way, the knowledge fragments that meet the requirements can be further screened out, the accuracy of the extraction results can be improved, and then it helps to improve.

[0051] It should be noted that, since we only consider the degree of relevance of the deep semantic information of the question during recall, we do not perform additional screening on the recalled knowledge fragments. This causes the knowledge fragments we recall to contain a large amount of repeated information, or records of a certain information at different time points. Therefore, it is necessary to further process the recalled knowledge fragments.

[0052] In some embodiments, the richness of the knowledge fragments can be improved so that the obtained target knowledge fragments contain information of different dimensions as much as possible, that is, information diversity, such as Figure 4A Specifically, the execution subject can traverse each recalled knowledge fragment in the recalled knowledge fragments, and according to the similarity value between the recalled knowledge fragment and the request vector, and the similarity value between the recalled knowledge fragment and the traversed recalled knowledge fragments, the maximum boundary correlation algorithm is used to adjust the order of each recalled knowledge fragment. Among them, the recalled knowledge fragments that have high similarity with the request vector and are different from the traversed knowledge fragments have a larger weight after adjustment.

[0053] As an example, the diversity of knowledge fragments can be improved through the MMR (Marginal Maximal Revelance) algorithm. Among them, MMR is usually an algorithm used in information retrieval and recommendation systems, which aims to generate summaries or recommendation lists with diversity and relevance. Here, the core idea of ​​the MMR algorithm is: by traversing the knowledge fragment elements in the knowledge base in full, those knowledge fragments that have a high correlation with the user's original request and are different from the traversed elements will be given a higher weight. On the contrary, those knowledge fragments that are not similar to the original request or are repeated with the traversed elements will be given a lower weight. The algorithm can be expressed as:

[0054]

[0055] Among them, λ represents the adjustable weight coefficient; the user request q is calculated i and recall knowledge fragment a j The similarity of the traversed recalled knowledge fragment a is calculated j′With the currently selected recalled knowledge fragment a j similarity; R represents the set of recalled knowledge fragments; S represents the set of recalled knowledge fragments that have been traversed; R\S represents the set of recalled knowledge fragments remaining except S.

[0056] From the above expression, we can see that the two similarities are in a subtractive relationship. Here, the MMR algorithm will eventually select the recalled knowledge fragment that can maximize the difference as the target knowledge fragment until all recalled knowledge fragments are traversed.

[0057] Optionally, in order to ensure that the knowledge fragments are relatively new, the ranking importance of some old knowledge fragments can be reduced based on exponential time decay, that is, the longer the time interval, the lower the adjusted weight, such as Figure 4A Specifically, the execution subject can readjust the order of each recalled knowledge fragment according to the similarity value of each recalled knowledge fragment and the time difference between the time of writing into the knowledge base and the current time. Among them, the smaller the time difference between the recalled knowledge fragment and the current time, the greater the adjusted weight. The algorithm expression can be:

[0058]

[0059] Among them, r ij Indicates user request q i The recalled knowledge fragment a j The sorting after recall, i.e. similarity sorting; a and b represent two adjustable parameters; t0 represents the current time point; t j Indicates the time point when the knowledge fragment is stored.

[0060] It is understandable that the above two sorting algorithms can be used simultaneously through a certain form of weighting, or can be connected in series through sequential calls. For example, the newer recalled knowledge fragments can be selected by calculating the time exponential decay, and then the multi-dimensional target knowledge fragments can be obtained through the MMR algorithm.

[0061] Step 203, according to the information consultation request and the target knowledge fragment, a method based on language prompts is used to perform data splicing to generate reply information for the information consultation request.

[0062] In some embodiments, according to the information consultation request and the target knowledge fragment, the execution subject can use a language prompt-based method (prompt) to splice data to generate a reply message for the information consultation request. Among them, the prompt method is generally a technology that provides a text prompt (prompt) to the model to guide it to generate a generated text of a specific style or content. Specifically, a language prompt-based method can be used to splice the information consultation request and the target knowledge fragment according to a preset format. Afterwards, the spliced ​​data can be input into the preset model to generate a reply message for the information consultation request. The preset format here is also not limited. For example Figure 4A As shown, you can use "Please rewrite a reply based on the question [query] raised by the customer service and use the following facts as supplementary materials."

[0063] It should be noted that there are some differences between automatic replies in private marketing and traditional one-to-one replies in terms of scenarios:

[0064] 1) The recipients of private reply messages are everyone in the group rather than individual customers.

[0065] 2) Private domain marketing reply information needs to guide customers to make purchases.

[0066] 3) Automatic replies in private domain marketing are highly timely. Compared with one-on-one manual replies, when the number of messages is large, it takes more time for humans to identify and judge, and ultimately the timeliness of manual replies is lower.

[0067] 4) Standardize automatic responses for private domain marketing. The same standard responses are used for the same type of questions, and there are no different responses for the same question.

[0068] 5) The automatic responses for private domain marketing are of high quality, accurately identifying user intent and providing problem solving or guidance.

[0069] 6) Automatic replies for private domain marketing can be used to build personalities and provide targeted services to people with different profiles.

[0070] Traditional technologies mostly develop automatic question-answering functions based on knowledge graphs, which mainly include the following three modes:

[0071] PairQA: Search for historical questions that are closest to the user’s current question from existing question-and-answer data and reply with corresponding answers.

[0072] DocQA: Extracting snippets from unstructured text as response information.

[0073] KBQA: Relies on the use of knowledge graph data to reason about response information from past historical structured data.

[0074] Among the three common modes mentioned above, the technical basis of PairQA and DocQA is information retrieval technology. That is, for the question data raised by the user, the one with the highest similarity in the historical answers is selected as the reply. The main problems of such solutions are:

[0075] The reply content is lagging: The reply text is generated based on historical QA (question and answer) data, rather than customized based on the current user's question. It may not answer the question correctly, or fail to use the correct reply tone.

[0076] It cannot handle multi-hop problems and problems with constraints: For example, "Which courier is the cheapest for about 4 kilograms?" It is necessary to first identify that 4 kilograms of goods is a constraint that must be met. The user is asking which courier product is the cheapest for 4 kilograms of goods, not which product is the cheapest under other weight conditions. For example, if the user's question is "Which courier do I need to find to send a courier?", it is usually necessary to identify the user's group and then find the courier associated with the road area corresponding to the group.

[0077] Although KBQA is better than traditional PairQA and DocQA in solving multi-hop and constrained problems, its structure is relatively complex and the construction cost is high. It is not feasible to quickly build an automatic reply system from scratch. At the same time, traditional reply texts are mostly pre-matched answer texts, which cannot achieve natural language-like question and answer. Figure 1 Taking the KBQA solution that is actually implemented as an example, the complexity of the solution without the help of large model capabilities is briefly explained. Because the early automatic QA solution lacks the deep semantic understanding ability of large language models, it is necessary to build multiple modules to realize the reply function, mainly including: query understanding module, relationship recognition module, subgraph recall module and answer sorting module.

[0078] Through the above introduction to the traditional automatic question-answering system, it is not difficult to find that in the absence of a large model, the construction of an automated question-answering system is a time-consuming and labor-intensive process, and it requires a large amount of open source / proprietary data to fine-tune the relevant pre-trained models. Benefiting from its deep semantic understanding ability, the large language model can solve the demand for automatic replies end-to-end, but due to the characteristics of generative AI, it cannot distinguish what kind of knowledge is real and what kind of knowledge is fictitious, which easily causes the large language model to produce "hallucinations" when replying, that is, generating inaccurate, fictitious or inconsistent answers with reality. In the Civic marketing scenario, it is usually difficult to tolerate the system passing on wrong knowledge to customers, which requires the system to be able to accurately reply to information.

[0079] Based on this, the information generation method disclosed in the present invention can extract knowledge fragments from a pre-built knowledge base based on the retrieval enhancement and generation capabilities of the large model. Thus, by splicing through the prompt method, it can be rewritten into a more accurate, controllable, and anthropomorphic reply. Since the knowledge fragment vectors of relevant business knowledge are pre-stored in the knowledge base, when extracting knowledge fragments, only similarity calculation is required to complete the recall of knowledge fragments. There is no need to perform complex data recognition and analysis. In this way, while ensuring the accuracy of data extraction, it can not only simplify the processing process, reduce complexity, and improve data processing efficiency, but also reduce resource requirements. In addition, by using the prompt method for data splicing, reply information of the required style or content can be generated. Thereby achieving the controllability of reply information and improving the accuracy of reply information generation.

[0080] In some embodiments, the execution entity may send the generated reply information directly to the user to request a reply.

[0081] It should be noted that before generating the reply information, it is usually necessary to do some necessary checks on the knowledge fragments (i.e., the target knowledge fragments) recalled in step 202. Since the previous process only sorted the knowledge fragments by their relative freshness and relative relevance, it is possible that the sorted knowledge fragments are still outdated or have low relevance. Therefore, these knowledge fragments should be filtered out before data splicing.

[0082] Specifically, Figure 3 As shown, first, the target knowledge fragment can be verified according to the preset conditions. The preset conditions here can also be set by yourself. For example, the time can be set to be less than half a year, and the correlation can be greater than 0.8. Then, when it is determined that the target knowledge fragment has passed the verification, the method based on language prompts can be used to splice the information consultation request and the target knowledge fragment according to the preset format. After that, the spliced ​​data can be input into the preset model to generate the reply information of the information consultation request. Alternatively, when it is determined that the target knowledge fragment has not passed the verification, the execution entity can send the information consultation request to the target terminal for manual reply.

[0083] It is understandable that the method disclosed in the present invention, through the verification of the target knowledge fragment, usually generates two main branches to achieve the generation of controllable reply information. When the knowledge fragment fails to pass the preset conditions (such as freshness and relevance), the fallback logic is generally triggered. That is, the customer can be informed that this request (question) cannot be answered temporarily according to a unified reply template, and the request can be submitted to the manual customer service asynchronously to generate a reply text for the request. When the knowledge fragment can pass the preset condition verification, the sorted target knowledge fragment and the user's original request will be spliced ​​together to form a complete prompt, which will be rewritten by the large model. The rewriting can make the text generated by the large model as close to the meaning of the original text as possible, which can make the request reply controllable.

[0084] Furthermore, in order to improve the accuracy of the information reply, after the reply information is generated, the reply information can be further verified. Figure 4A As shown, a pre-trained verification model can be used to perform factual verification on the generated reply information. The verification model here can also adopt various machine learning structures. The verification model is generally obtained by training with training data. Among them, the training data can be sample information consultation requests and corresponding sample reply information as sample data, and user feedback on sample reply information as sample labels. For example, the degree to which the reply answers the user's question can be used as a label. Here, if it is determined that the factual verification has passed, the reply information can be sent to the user. Information with factual errors may not be returned.

[0085] It should be noted that with the continuous response processing of information consultation requests and real-time feedback from users, the data is constantly increasing and updating. Therefore, after replying to the information consultation request, the execution subject can also vectorize the reply information of the information consultation request to obtain a new knowledge fragment vector. And the new knowledge fragment vector can be stored in the knowledge base to update the knowledge base. And according to the real-time feedback from users, the above verification model can be continuously trained and updated.

[0086] In some application scenarios, the above-mentioned knowledge base can be obtained by the following method. First, text elements can be extracted from the original data describing the relevant business knowledge. The original data may include at least one of a web page, a document, and a page. Then, the extracted text elements can be split into at least one knowledge fragment according to the layout structure of the elements in the original data. After that, at least one knowledge fragment can be vectorized to obtain at least one knowledge fragment vector. And at least one knowledge fragment vector is stored to obtain a knowledge base.

[0087] It should be noted that the main purpose of building a knowledge base is to convert unstructured knowledge into computable vectors through vectorization. Different from the vectorization processing of user requests, the processing of knowledge bases often requires more pre-operations. This is mainly because the knowledge in the business field is usually stored in an unstructured form, such as HTML (Hypertext Markup Language) web pages, online documents, confluence (enterprise knowledge management and collaboration software) pages, etc. Before vectorization, it is generally necessary to cut the text data as necessary.

[0088] by Figure 4B Taking the web page shown in the figure as an example, it is necessary to extract the elements under the specific sub-topic and split them into the following knowledge fragments:

[0089] Introduction and advantages of express heavy cargo service: Provide users with timely response, professional packaging, customized collection and delivery services for cargo weighing 20-500kg per ticket, with a loss rate better than the industry level and a fast-acting full-network land transportation product;

[0090] Applicable scenarios for express heavy cargo: B2C high-value e-commerce, B2B small and medium-sized manufacturers, wholesalers / distributors and individual users;

[0091] Express heavy cargo service scope: more than 300 cities across the country, serving you anytime, anywhere, 365 days a week;

[0092] Express heavy cargo service: simple billing, accurate timeliness, and worry-free.

[0093] In addition, because the amount of knowledge fragments is usually large, generally tens of thousands or even hundreds of thousands, you can choose some high-performance vector databases, such as Milvus, Faiss, or Elasticsearch (including Elasticsearch-Vector-Scoring capabilities) to store them, so as to facilitate the system to perform subsequent knowledge extraction.

[0094] It is understandable that the information generation method disclosed in this disclosure is mainly based on the retrieval enhancement and generation capabilities of the large model. In the case of an external knowledge base, the process of recalling, sorting, verifying, and rewriting knowledge fragments is used to generate accurate, controllable, and anthropomorphic responses to deal with QA (question and answer) problems faced in private domain group chats. This solves the following problems: the cumbersome construction process of the traditional automatic question and answer system makes it difficult to implement lightweight operations in the express private domain marketing scenario; the problem that it is difficult to achieve controllable responses using only a large language model; and the problem that it is difficult to achieve humanized responses to questions and answers based on knowledge graphs.

[0095] In addition, the method disclosed in this disclosure was applied to actual scenarios for testing. The service of this method was deployed in 50,000+ private domain express delivery groups, covering 1.8 million private domain customers. The customer's reply information in the group chat scenario is mainly concentrated on the following: Figure 4C After the function was launched, through AB test comparison, we found that under the guidance of the reply function of the disclosed method, the proportion of price discounts and shipping information in the experimental group increased significantly compared with the control group, such as Figure 4D shown.

[0096] Further references Figure 5 , as a response to the above Figures 2 to 3 In order to realize the method shown in the figure, the present disclosure provides some embodiments of an information generating device. These device embodiments are similar to Figures 2 to 3 The device can be specifically applied to various electronic devices.

[0097] like Figure 5 As shown, the information generating device 500 of some embodiments may include: a request processing unit 501, configured to, in response to receiving an information consultation request sent by a user, perform vectorized processing on the information consultation request to obtain a request vector; a knowledge extraction unit 502, configured to extract a target knowledge fragment matching the information consultation request from a preset knowledge base according to the request vector, wherein the knowledge base stores knowledge fragment vectors of relevant business knowledge; a reply information generating unit 503, configured to perform data splicing based on a language prompt method according to the information consultation request and the target knowledge fragment, to generate reply information for the information consultation request.

[0098] In some embodiments, the knowledge extraction unit 502 can be further configured to determine the similarity value between each knowledge fragment vector and the request vector in a preset knowledge base; extract a first number of knowledge fragments from the knowledge base in order of similarity value from high to low as recalled knowledge fragments; adjust the sorting of the recalled knowledge fragments according to preset rules, and extract a second number of recalled knowledge fragments from them according to the adjusted sorting as target knowledge fragments, wherein the second number is less than or equal to the first number.

[0099] In some embodiments, the knowledge extraction unit 502 can be further configured to readjust the order of each recalled knowledge fragment according to the similarity value of each recalled knowledge fragment and the time difference between the time of writing into the knowledge base and the current time, wherein the smaller the time difference between the recalled knowledge fragment and the current time, the greater the adjusted weight.

[0100] In some embodiments, the knowledge extraction unit 502 can be further configured to traverse each recalled knowledge fragment in the recalled knowledge fragments, and adjust the sorting of each recalled knowledge fragment using the maximum boundary correlation algorithm according to the similarity value between the recalled knowledge fragment and the request vector, and the similarity value between the recalled knowledge fragment and the recalled knowledge fragment that has been traversed, wherein the recalled knowledge fragment that has a high similarity with the request vector and is different from the recalled knowledge fragment that has been traversed has a larger adjusted weight.

[0101] In some embodiments, the information generating device 500 may also include a knowledge base generating unit (not shown in the figure), which is configured to extract text elements from original data describing relevant business knowledge, wherein the original data includes at least one of a web page, a document, and a page; according to the layout structure of the elements in the original data, the extracted text elements are split into at least one knowledge fragment; at least one knowledge fragment is vectorized to obtain at least one knowledge fragment vector, and at least one knowledge fragment vector is stored to obtain a knowledge base.

[0102] In some embodiments, the reply information generation unit 503 can be further configured to verify the target knowledge fragment according to preset conditions; in response to determining that the target knowledge fragment has passed the verification, using a language prompt-based method to splice the information consultation request and the target knowledge fragment in a preset format; input the spliced ​​data into a preset model to generate reply information for the information consultation request; in response to determining that the target knowledge fragment has not passed the verification, sending the information consultation request to the target terminal for manual reply.

[0103] In some embodiments, the information generating device 500 may further include a reply information verification unit (not shown in the figure), which is configured to use a pre-trained verification model to perform factual verification on the generated reply information, wherein the verification model is trained through training data, the training data uses sample information consultation requests and corresponding sample reply information as sample data, and uses user feedback on the sample reply information as sample labels; in response to determining that the factual verification has passed, the reply information is sent to the user.

[0104] In some embodiments, the information generating device 500 may also include a knowledge base updating unit (not shown in the figure), which is configured to vectorize the reply information of the information consultation request to obtain a new knowledge fragment vector; store the new knowledge fragment vector in the knowledge base to update the knowledge base.

[0105] It is understandable that the units recorded in the information generating device 500 are similar to those in the reference Figures 2 to 3Therefore, the operations, features and beneficial effects described above for the method are also applicable to the information generating device 500 and the units included therein, and will not be described in detail here.

[0106] Reference below Figure 6 , which shows a structural schematic diagram of an electronic device 600 suitable for implementing some embodiments of the present disclosure. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0107] like Figure 6 As shown, the electronic device 600 may include a processing device 601 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0108] Typically, the following devices may be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a speaker, a vibrator, etc.; storage devices 608 including, for example, a memory card, a disk, a hard disk, etc.; and communication devices 609. The communication devices 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0109] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0110] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0111] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0112] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to receiving an information consultation request sent by a user, vectorizes the information consultation request to obtain a request vector; based on the request vector, extracts a target knowledge fragment that matches the information consultation request from a preset knowledge base, wherein the knowledge base stores knowledge fragment vectors of relevant business knowledge; based on the information consultation request and the target knowledge fragment, uses a language prompt-based method to perform data splicing to generate reply information for the information consultation request.

[0113] In addition, computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The units described may also be arranged in a processor, for example, they may be described as: a processor includes a request processing unit, a knowledge extraction unit, and a reply information generation unit. The names of these units do not constitute limitations on the units themselves in certain cases, for example, the request processing unit may also be described as a "unit for vectorizing information consultation requests".

[0116] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0117] Some embodiments of the present disclosure further provide a computer program product, including a computer program, which implements any of the above-mentioned information generation methods when executed by a processor.

[0118] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for generating information, comprising: In response to receiving an information consultation request sent by a user, vectorizing the information consultation request to obtain a request vector; According to the request vector, a target knowledge fragment matching the information consultation request is extracted from a preset knowledge base, wherein the knowledge base stores knowledge fragment vectors of relevant business knowledge; According to the information consultation request and the target knowledge fragment, a language prompt-based method is used to perform data splicing to generate reply information for the information consultation request.

2. The information generation method according to claim 1, wherein: The step of extracting a target knowledge segment matching the information consultation request from a preset knowledge base according to the request vector includes: Determine the similarity value between each knowledge fragment vector and the request vector in a preset knowledge base; Extracting a first number of knowledge fragments from the knowledge base in descending order of similarity values ​​as recalled knowledge fragments; The order of the recalled knowledge fragments is adjusted according to a preset rule, and a second number of recalled knowledge fragments are extracted from the recalled knowledge fragments according to the adjusted order as target knowledge fragments, wherein the second number is less than or equal to the first number.

3. The information generation method according to claim 2, wherein: The step of adjusting the order of the recalled knowledge fragments according to a preset rule includes: The order of each recalled knowledge fragment is readjusted according to the similarity value of each recalled knowledge fragment and the time difference between the time of writing into the knowledge base and the current time, wherein the recalled knowledge fragment with a smaller time difference from the current time has a larger adjusted weight.

4. The information generation method according to claim 2, wherein: The step of adjusting the order of the recalled knowledge fragments according to a preset rule further includes: Traverse each of the recalled knowledge fragments, and adjust the order of each recalled knowledge fragment according to the similarity value between the recalled knowledge fragment and the request vector, as well as the similarity value between the recalled knowledge fragment and the traversed recalled knowledge fragments, using the maximum boundary correlation algorithm, wherein the recalled knowledge fragment that has a high similarity with the request vector and is different from the traversed knowledge fragments has a larger adjusted weight.

5. The information generation method according to claim 1, wherein: The knowledge base is obtained by the following method: Extracting text elements from raw data describing relevant business knowledge, wherein the raw data includes at least one of a web page, a document, and a page; Splitting the extracted text elements into at least one knowledge fragment according to the layout structure of the elements in the original data; The at least one knowledge fragment is vectorized to obtain at least one knowledge fragment vector, and the at least one knowledge fragment vector is stored to obtain a knowledge base.

6. The information generation method according to claim 1, wherein: The step of performing data splicing based on the information consultation request and the target knowledge fragment using a language prompt-based method to generate reply information to the information consultation request includes: Verifying the target knowledge fragment according to preset conditions; In response to determining that the target knowledge fragment has passed the verification, the information consultation request and the target knowledge fragment are spliced ​​according to a preset format using a method based on language prompts; Input the spliced ​​data into a preset model to generate reply information for the information consultation request; In response to determining that the target knowledge segment fails verification, the information consultation request is sent to a target terminal for manual reply.

7. The information generation method according to claim 6, wherein: The method further comprises: Using a pre-trained verification model to perform factual verification on the generated reply information, wherein the verification model is trained by training data, the training data uses sample information consultation requests and corresponding sample reply information as sample data, and uses user feedback on the sample reply information as sample labels; In response to determining that the factual verification passes, the reply information is sent to the user.

8. The information generation method according to any one of claims 1 to 7, wherein: The method further comprises: Vectorizing the reply information of the information consultation request to obtain a new knowledge fragment vector; The new knowledge fragment vector is stored in the knowledge base to update the knowledge base.

9. An information generating device, comprising: a request processing unit configured to, in response to receiving an information consultation request sent by a user, perform vectorization processing on the information consultation request to obtain a request vector; a knowledge extraction unit configured to extract, according to the request vector, a target knowledge fragment matching the information consultation request from a preset knowledge base, wherein the knowledge base stores knowledge fragment vectors of relevant business knowledge; The reply information generating unit is configured to perform data splicing based on the information consultation request and the target knowledge fragment by adopting a language prompt-based method to generate reply information for the information consultation request.

10. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the information generating method according to any one of claims 1 to 8.

11. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the information generating method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the information generation method according to any one of claims 1 to 8.

Images