Information processing method and related equipment

Abstract

The invention provides an information processing method. The information processing method comprises the following steps: receiving first interaction information from a user; calling a trained target generation model based on the first interaction information to obtain second interaction information corresponding to the first interaction information; obtaining knowledge content related to the second interaction information from a knowledge base; and verifying the second interaction information based on the knowledge content. The invention further provides an information processing system, electronic equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to an information processing method, system, electronic device, storage medium and program product. Background Art

[0002] A large language model (LLM) refers to a deep learning model trained with a large amount of text data that can generate natural language text or understand the meaning of language text. LLM can handle a variety of natural language tasks, such as text classification, question answering, and conversation, and is an important path to artificial intelligence. However, in actual use, the content generated by LLM may be inconsistent with the given input or inconsistent with the facts. Usually, the above situation is called "hallucination" of LLM. Therefore, how to solve the problem of "hallucination" of LLM is also one of the important topics of LLM research. Summary of the invention

[0003] In view of this, an embodiment of the present disclosure provides an information processing method, which can effectively solve the problem that the content generated by the target generation model is inconsistent with the given input or inconsistent with the facts.

[0004] The above-mentioned information processing method includes: receiving first interaction information from a user; calling a trained target generation model based on the first interaction information to obtain second interaction information corresponding to the first interaction information; acquiring knowledge content related to the second interaction information from a knowledge base; and verifying the second interaction information based on the knowledge content.

[0005] In an embodiment of the present disclosure, obtaining knowledge content related to the second interaction information from the knowledge base includes: extracting key information of the second interaction information from the second interaction information; calling the knowledge base to obtain relevant knowledge content based on the key information of the second interaction information; and using the obtained knowledge content as knowledge content related to the second interaction information.

[0006] In an embodiment of the present disclosure, obtaining knowledge content related to the second interaction information from the knowledge base includes: extracting key information of the first interaction information from the first interaction information; calling the knowledge base to obtain relevant knowledge content based on the key information of the first interaction information; and using the obtained knowledge content as knowledge content related to the second interaction information.

[0007] In an embodiment of the present disclosure, obtaining knowledge content related to the second interaction information from the knowledge base includes: extracting key information of the first interaction information from the first interaction information; extracting key information of the second interaction information from the second interaction information; calling the knowledge base to obtain relevant knowledge content based on the key information of the first interaction information and the key information of the second interaction information; and using the obtained knowledge content as knowledge content related to the second interaction information.

[0008] In an embodiment of the present disclosure, the extracting key information of the second interaction information from the second interaction information includes: obtaining the key information of the second interaction information from the target generation model; or, extracting the key information of the second interaction information from the second interaction information based on a trained key information extraction model; or, performing intent recognition and entity recognition on the second interaction information, and using the recognized intent information and entity information as the key information of the second interaction information.

[0009] In an embodiment of the present disclosure, the extracting key information of the first interaction information from the first interaction information includes: extracting the key information of the first interaction information from the first interaction information based on a trained key information extraction model; or, performing intent recognition and entity recognition on the first interaction information, and using the recognized intent information and entity information as the key information of the first interaction information.

[0010] In an embodiment of the present disclosure, the verification of the second interactive information based on the knowledge content includes: taking the knowledge content as a premise; taking the second interactive information as a hypothesis; calling a trained reasoning model to determine the logical relationship between the premise and the hypothesis; and determining whether the second interactive information passes the verification based on the logical relationship between the premise and the hypothesis.

[0011] In an embodiment of the present disclosure, the verifying the second interaction information based on the knowledge content includes: calling a trained accuracy prediction model based on the knowledge content and the second interaction information to determine whether the second interaction information passes the verification.

[0012] In an embodiment of the present disclosure, calling a trained target generation model based on the first interaction information to obtain second interaction information corresponding to the first interaction information includes: calling a knowledge base based on key information of the first interaction information to obtain knowledge content corresponding to the first interaction information; constructing prompt information corresponding to the first interaction information based on the key information and the knowledge content; and calling the target generation model based on the prompt information to obtain second interaction information corresponding to the prompt information.

[0013] Based on the above information processing method, some embodiments of the present disclosure further provide an information processing system, including:

[0014] A knowledge base, including one or more databases for storing knowledge content;

[0015] A user interface, configured to receive first interaction information from a user;

[0016] An interaction module, configured to call a trained target generation model based on the first interaction information to obtain the second interaction information;

[0017] A knowledge content extraction module, used for acquiring knowledge content related to the second interactive information from a knowledge base;

[0018] A risk control module is used to verify the second interactive information based on the knowledge content.

[0019] In addition, an embodiment of the present disclosure further provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned information processing method when executing the program.

[0020] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the above-mentioned information processing method.

[0021] An embodiment of the present disclosure further provides a computer program product, including computer program instructions, which, when executed on a computer, enable the computer to execute the above information processing method.

[0022] Through the above information processing method and related devices, after receiving the first interactive information input by the user, the second interactive information as a reply to the above first interactive information can be generated by calling the pre-trained target generation model. Next, the second interactive information generated by the above target generation model is verified based on the pre-established knowledge base. In this way, on the one hand, the consistency between the output second interactive information and the first interactive information input by the user can be guaranteed; on the other hand, the output second interactive information can also be guaranteed to be consistent with the facts, thereby effectively solving the problem of "hallucination" in the target generation model.

[0023] In addition, in the embodiments of the present disclosure, the above-mentioned target generation model can be obtained based on LLM training. Therefore, the above-mentioned information processing method can effectively support human-computer interaction applications based on LLM, make full use of the advantages of LLM in natural language processing, and realize efficient and accurate information processing.

[0024] Furthermore, the above process of verifying the second interactive information can also be implemented through a trained LLM, thereby further making full use of the performance advantages of LLM in natural language processing, improving the accuracy of user responses, and obtaining a better human-computer interaction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic diagram of an application scenario of an information processing system is shown;

[0027] Figure 2 The implementation process of the information processing method described in some embodiments of the present disclosure is shown;

[0028] Figure 3 The implementation process of the method of calling a trained target generation model based on first interaction information to obtain second interaction information corresponding to the first interaction information according to some embodiments of the present disclosure is shown;

[0029] Figure 4 The implementation process of the method for constructing prompt information corresponding to the first interaction information based on key information according to some embodiments of the present disclosure is shown;

[0030] Figure 5 The implementation process of the method for acquiring knowledge content related to the second interactive information from the knowledge base according to some embodiments of the present disclosure is shown;

[0031] Figure 6 The implementation process of the method for acquiring knowledge content related to the second interactive information from the knowledge base according to some other embodiments of the present disclosure is shown;

[0032] Figure 7 The implementation process of the method for acquiring knowledge content related to the second interactive information from the knowledge base according to some other embodiments of the present disclosure is shown;

[0033] Figure 8 The implementation process of the method for verifying the second interactive information based on the above-mentioned knowledge content described in some embodiments of the present disclosure is shown;

[0034] Fig. 9 The internal structure of the information processing system according to the embodiment of the present disclosure is shown;

[0035] Fig.10 A more specific schematic diagram of the hardware structure of an electronic device described in some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0037] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0038] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0039] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0040] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0041] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0042] Figure 1FIG. 1 shows a schematic diagram of an application scenario of the information processing system 100 described in an embodiment of the present disclosure when implementing a natural language interactive application. Figure 1 As shown, the user 110 can input interactive information in natural language form through the language user interface (LUI) 102 of the information processing system 100. Next, the interactive processing module 104 in the information processing system 100 analyzes and processes the interactive information received from the above-mentioned LUI to generate a corresponding reply, and feeds the reply back to the user 110 through the above-mentioned LUI. The above-mentioned information processing system can understand the meaning of natural language text and can generate natural language text, so that a variety of applications can be implemented to complete a variety of natural language tasks, such as text classification, question and answer, dialogue, etc. However, as mentioned above, in actual use, there may be a "hallucination" problem that the content generated by the LLM is inconsistent with the given input or does not conform to the facts.

[0043] In order to solve the above problems, the embodiments of the present disclosure provide an information processing method, which can effectively solve the "hallucination" problem that the content generated by the target generation model is inconsistent with the given input or inconsistent with the facts.

[0044] Figure 2 The implementation process of the information processing method described in the embodiment of the present disclosure is shown. It should be noted that the above information processing method can be executed by the above information processing system. Figure 2 As shown, the above information processing method may include the following steps:

[0045] In step 210, first interaction information is received from a user.

[0046] In the embodiments of the present disclosure, in the natural language interaction scenario, the first interaction information may generally be information in the form of natural language, for example, in the form of text, or in other forms such as voice. For the first interaction information in the form of voice, generally before performing subsequent operations, the information processing system may first perform voice recognition on the received first interaction information to obtain the corresponding text information; then, the subsequent processing may be continued based on the recognized text information.

[0047] It should be noted that the information processing method described in the embodiment of the present disclosure can also be applied to other interaction scenarios besides the natural language interaction scenario. For example, the information processing method described in the embodiment of the present disclosure can also be applied to computer vision scenarios. In this case, the first interaction information can be an image or other forms of information.

[0048] In step 220, a trained target generation model is called based on the first interaction information to obtain second interaction information corresponding to the first interaction information.

[0049] In an embodiment of the present disclosure, in a natural language interaction scenario, the target generation model may be a pre-trained LLM that can generate text in natural language form based on the input first interaction information. Thus, in the above step 220, the first interaction information may be input into the above target generation model, and the text in natural language form output by the above target generation model may be used as the above second interaction information. In other embodiments of the present disclosure, the above step 220 may also be completed by other methods, and the specific method will be described in detail later, which will be omitted here for the time being.

[0050] Similarly, in other interaction scenarios, the target generation model may be a pre-trained machine learning model that can generate second interaction information in the same form as the first interaction information based on the input first interaction information.

[0051] In step 230, knowledge content related to the second interactive information is acquired from a knowledge base.

[0052] In the embodiment of the present disclosure, the above-mentioned knowledge base may be one or more pre-established databases for storing knowledge content. How to obtain the knowledge content related to the above-mentioned second interaction information from the knowledge base will be described in detail later, which is omitted here for the time being.

[0053] In step 240, the second interaction information is verified based on the knowledge content.

[0054] In the embodiments of the present disclosure, the verification generally refers to the accuracy verification of the second interactive information. How to verify the second interactive information based on the knowledge content can also be implemented by a variety of methods. The specific implementation method will be described in detail later and will be omitted here for the time being.

[0055] It can be seen that in the above information processing method, after receiving the first interactive information input by the user, the above information processing system can generate the second interactive information as a reply to the above first interactive information by calling the pre-trained target generation model. Next, the accuracy of the second interactive information generated by the target generation model is verified based on the pre-established knowledge base. In this way, on the one hand, the consistency of the output second interactive information with the first interactive information input by the user can be guaranteed; on the other hand, the output second interactive information can also be guaranteed to be consistent with the facts, thereby effectively solving the problem of "hallucination" in the target generation model.

[0056] In addition, in the above-mentioned information processing method, the above-mentioned target generation model can be obtained based on LLM training. Therefore, the above-mentioned information processing method can effectively support human-computer interaction applications based on LLM, make full use of the advantages of LLM in natural language processing, and realize efficient and accurate information processing.

[0057] Furthermore, in the above-mentioned information processing method, the above-mentioned process of verifying the second interactive information can be implemented through a trained LLM, thereby making full use of the performance advantages of LLM in natural language processing, improving the accuracy of user responses, and obtaining a better human-computer interaction effect.

[0058] The specific implementation method of each step in the above information processing method will be further described in detail below with reference to specific examples and drawings.

[0059] With respect to the above step 220, the embodiments of the present disclosure provide a variety of methods for invoking a trained target generation model based on the above first interaction information to obtain second interaction information corresponding to the above first interaction information. Figure 3 The specific implementation process of the method described in some embodiments of the present disclosure of calling a trained target generation model based on the first interaction information to obtain the second interaction information corresponding to the first interaction information is shown.

[0060] In step 310, key information of the first interaction information is extracted from the first interaction information.

[0061] In an embodiment of the present disclosure, the key information may include intent information and entity information corresponding to the first interaction information. In some embodiments, in a natural language interaction scenario, the step 220 described in extracting the key information of the first interaction information from the first interaction information may be implemented by an existing natural language processing (NLP) method. For example, extracting the intent information corresponding to the first interaction information may be implemented by any intent recognition method in NLP; extracting the entity information corresponding to the first interaction information may be implemented by any entity recognition method in NLP. Alternatively, in other embodiments of the present disclosure, the operation of extracting the key information from the first interaction information (including intent recognition and entity recognition) may also be implemented using a trained LLM or other machine learning model, that is, using the capabilities of the LLM or other machine learning model to extract the key information of the first interaction information (including intent recognition and entity recognition). In order to distinguish from other models, the trained LLM or other machine learning model may also be referred to as a key information extraction model. In an application, the key information extraction model may be obtained by training in a supervised manner, and its training data set may include natural language text information and its corresponding key information. After the LLM or other machine learning model is trained using the training set and the preset conditions are met, the key information extraction model may be obtained. It should be noted that the embodiments of the present disclosure do not limit the specific implementation method of extracting key information in the above steps.

[0062] In addition to performing a key information extraction operation on the first interaction information, the received first interaction information may also be normalized, thereby performing error correction or rewriting on the received first interaction information, so as to further ensure the accuracy of the extracted key information.

[0063] In practical applications, the above-mentioned intent information may generally refer to the intent category to which the above-mentioned first interactive information belongs after the intent classification of the above-mentioned first interactive information, for example, the first interactive information belongs to the game category or the popular science category, etc. The above-mentioned entity information generally refers to the named entities contained in the above-mentioned first interactive information, such as names of people, names of organizations, names of places, and all other entities identified by names. More extensive named entities may also include numbers, dates, currencies, addresses, etc. It can be understood that the above-mentioned intent information and entity information will be used to construct the input of the target generation model - prompt information. Therefore, the accuracy of the above-mentioned intent information and entity information will affect the accuracy of the prompt information, and then affect whether the reply information output by the target generation model for the first interactive information - the second interactive information is accurate.

[0064] In step 320, the knowledge base is called based on the key information to obtain knowledge content corresponding to the first interaction information.

[0065] It should be noted that in the embodiments of the present disclosure, it may be determined first according to one or more pre-set rules whether it is necessary to further obtain the knowledge content corresponding to the first interactive information during the current information processing. If it is determined that it is necessary to further obtain the knowledge content corresponding to the first interactive information, the above step 320 may be further performed; otherwise, the above step 320 may not be performed.

[0066] As mentioned above, the above-mentioned knowledge base may be one or more pre-established databases for storing knowledge content. Normally, different knowledge bases may be set corresponding to different intent categories, so as to ensure the accuracy of the knowledge content obtained for the first interactive information. In this case, the above-mentioned calling of the knowledge base based on the above-mentioned key information to obtain the knowledge content corresponding to the first interactive information may include: selecting a target knowledge base from multiple knowledge bases based on the intent information in the above-mentioned key information; and obtaining the knowledge content corresponding to the above-mentioned first interactive information from the above-mentioned target knowledge base based on the entity information in the above-mentioned key information. It can be seen that the target knowledge base that is more relevant to the first interactive information input by the user can be determined first by matching the intent category, and then the relevant knowledge content is extracted from the target knowledge base based on the entity information. Therefore, the extracted knowledge content will be the knowledge content corresponding to the entity information in the first interactive information under the intent category to which the first interactive information belongs, so it is more accurate and more in line with the user's intention.

[0067] Step 330: construct prompt information (Prompt) corresponding to the first interactive information based on the key information and the knowledge content corresponding to the first interactive information.

[0068] In some embodiments of the present disclosure, the construction of the above-mentioned prompt information can also be implemented by using a trained LLM or other machine learning model, that is, the ability of the LLM or other machine learning model is used to implement the process of constructing the prompt information corresponding to the first interactive information based on the above-mentioned key information and knowledge content. In order to distinguish it from other models, the above-mentioned trained LLM can also be called a prompt information generation model. In application, the above-mentioned prompt information generation model can be obtained by training in a supervised manner, and its training data set may include key information, knowledge content and its corresponding prompt information. After the LLM or other machine learning model is trained using the above-mentioned training set and the preset conditions are met, the above-mentioned prompt information generation model can be obtained.

[0069] Figure 4 The specific implementation process of the method for constructing prompt information corresponding to the first interaction information based on the key information described in some embodiments of the present disclosure is shown. As mentioned above, the key information includes the intent information and entity information corresponding to the first interaction information. Figure 4 Said method comprises the following steps:

[0070] In step 410, a target prompt information template is extracted from a pre-established prompt information template library based on the above intention information.

[0071] In an embodiment of the present disclosure, different prompt information templates can be designed in advance for different intent categories, thereby establishing a prompt information template library. Among them, the prompt information template library contains prompt information templates corresponding to various intent categories. Moreover, the prompt information template is used to configure the specific format of the prompt information and the multiple information items it carries. It can be understood that the prompt information templates stored in the prompt information template library are usually blank templates that do not contain specific information content. However, when a certain prompt information template is determined and the specific information content is filled in the determined prompt information template, the corresponding prompt information can be obtained. Moreover, the prompt information determined by the prompt information template can usually meet the input requirements of the target generation model.

[0072] In step 420, the entity information and knowledge content are filled into corresponding positions in the target prompt information template to obtain prompt information corresponding to the first interaction information.

[0073] As mentioned above, the above prompt information template is configured with multiple information items carried thereon. Then in the above step 420, the above entity information and knowledge content can be added one by one to the corresponding positions of the above target prompt information template according to the configuration of the target prompt information template, so as to finally obtain the prompt information corresponding to the above first interactive information.

[0074] It can be seen that the prompt information corresponding to the above-mentioned first interaction information can be quickly obtained through the above-mentioned method. Moreover, since the above-mentioned prompt information is generated based on the prompt information template, the format and content are relatively standardized and effective, which can better meet the input requirements of the target generation model. It can be understood that since the selection of the above-mentioned target prompt information template is related to the intent information extracted from the first interaction information, the prompt information corresponding to the above-mentioned first interaction information obtained by the above-mentioned method may also further include information reflecting the intent information corresponding to the above-mentioned first interaction information, for example, it may include the intent category to which the first interaction information belongs, etc.

[0075] In step 340, a trained target generation model is called based on the prompt information to obtain second interaction information corresponding to the prompt information.

[0076] It can be understood that after the prompt information is constructed, the constructed prompt information can be input into the trained target generation model to obtain the second interactive information as a reply to the first interactive information. Moreover, the second interactive information corresponding to the prompt information is the second interactive information corresponding to the first interactive information.

[0077] It can be understood that the above prompt information is the input of the target generation model, and its accuracy will affect the accuracy of the target generation model output, that is, whether the second interactive information is accurate. Therefore, in the above method, by introducing the knowledge content corresponding to the above first interactive information in addition to the key information of the first interactive information input by the user in the process of constructing the prompt information, the content of the prompt information can be further enriched and the accuracy of the prompt information can be guaranteed. In this way, by providing the target generation model with richer and more accurate prompt information, the accuracy of the second interactive information can be effectively improved.

[0078] In order to further improve the accuracy of the constructed prompt information, in some other embodiments of the present disclosure, the above method may further include: obtaining information processing auxiliary information. In the embodiments of the present disclosure, the above information processing auxiliary information may include: relevant information of the user obtained with the authorization of the user or application data of each application on the terminal device used by the user obtained with the authorization of the user, etc. Specifically, in actual applications, the above application data can be obtained through an application programming interface (API). It can be seen that the above information processing auxiliary information can provide background information of the first interaction information, such as information related to the user or data obtained from other users, etc., so as to more effectively help the target generation model understand the intention of the first interaction information input by the user, and then make the second interaction information output by the target generation model more in line with the needs of the user.

[0079] It should be noted that in the embodiments of the present disclosure, it may be determined whether it is necessary to further obtain information processing auxiliary information during the current information processing process according to one or more pre-set rules. If it is determined that it is necessary to further obtain information processing auxiliary information, the above step of obtaining information processing auxiliary information is performed. If it is determined that it is not necessary to further obtain information processing auxiliary information, the above step of obtaining information processing auxiliary information may not be performed.

[0080] In practical applications, the same Figure 4 The prompt information corresponding to the above-mentioned first interactive information is constructed by a method similar to the process shown. Specifically, step 410 can be executed first to extract the target prompt information template from the pre-established prompt information template library based on the above-mentioned intention information. Then, in step 420, the above-mentioned entity information, knowledge content and the above-mentioned information processing auxiliary information are filled into the corresponding positions in the above-mentioned target prompt information template to obtain the prompt information corresponding to the above-mentioned first interactive information. In this way, the generated prompt information will not only include the intention information, entity information and knowledge content corresponding to the first interactive information, but also include information processing auxiliary information, so that the constructed prompt information can be made richer and more accurate.

[0081] As mentioned above, in addition to Figure 4In addition to the method shown, the construction of prompt information can also be realized by using a trained LLM or other machine learning model. In order to distinguish it from the aforementioned prompt information generation model, this model can also be referred to as the second prompt information generation model, and the aforementioned prompt information generation model is referred to as the first prompt information generation model. The difference between the second prompt information generation model and the first prompt information generation model described in this embodiment is only whether the model input contains information processing auxiliary information. It can be understood that due to the differences in the input of the model, the training data set and the test data set used when training different models will also be different. That is, the training data set and the test data set of the second prompt information generation model will also include information processing auxiliary information.

[0082] It can be seen that by introducing the knowledge content of the knowledge base and even information processing auxiliary information when constructing prompt information, the obtained prompt information can be made richer and more accurate, thereby improving the accuracy of the second interactive information, so that it can better meet the needs of users and improve the user experience.

[0083] With respect to the above step 230, in some embodiments of the present disclosure, the implementation process of the method for acquiring the knowledge content related to the second interactive information from the knowledge base may be as follows: Figure 5 As shown, it mainly includes:

[0084] In step 510, key information of the second interaction information is extracted from the second interaction information.

[0085] In step 520, the knowledge base is called based on the key information of the second interaction information to obtain relevant knowledge content.

[0086] In step 530, the acquired knowledge content is used as the knowledge content related to the second interactive information.

[0087] In the embodiments of the present disclosure, multiple methods may be used to extract key information from the second interaction information.

[0088] In some embodiments, the key information of the second interactive information can be directly obtained from the target generation model. For example, after obtaining the second interactive information, an instruction to obtain key information from the second interactive information can be further input into the target generation model, and the target generation model can automatically extract the key information therein. That is, the text processing and interaction capabilities of the target generation model are directly used to directly extract key information from the second interactive information.

[0089] In some other embodiments of the present disclosure, similar to the above-mentioned operation of extracting key information from the first interaction information, key information can be extracted from the above-mentioned second interaction information based on other trained key information extraction models, such as the aforementioned key information extraction model for extracting key information from the first interaction information.

[0090] In some other embodiments of the present disclosure, in a natural language interaction scenario, the intent recognition and entity recognition methods in natural language processing can be used to first perform intent recognition and entity recognition on the above-mentioned second interaction information, and then use the recognized intent information and entity information as key information of the above-mentioned second interaction information.

[0091] As for how to call the knowledge base after obtaining the key information, you can also refer to the relevant operations in the aforementioned embodiments. For example, when the key information includes intent information and entity information, you can first determine the target knowledge base from multiple databases based on the intent information, and then query the target knowledge base based on the entity information to obtain relevant knowledge content. Of course, in some examples, you can also directly query the knowledge base with the key information as an index to obtain relevant knowledge content.

[0092] Similarly, with respect to the above step 230, in some other embodiments of the present disclosure, the implementation process of the method for acquiring the knowledge content related to the second interactive information from the knowledge base can be as follows: Figure 6 As shown, it mainly includes:

[0093] In step 610, key information of the first interaction information is extracted from the first interaction information.

[0094] In step 620, the knowledge base is called based on the key information of the first interaction information to obtain relevant knowledge content.

[0095] In step 630, the acquired knowledge content is used as knowledge content related to the second interactive information.

[0096] In the embodiment of the present disclosure, the specific method of extracting the key information of the first interactive information from the first interactive information in the above step 610 can refer to the method in the above embodiment, and will not be repeated here. In addition, assuming that in its preceding step 220, the information processing system has extracted the key information of the first interactive information, the key information of the first exchange information can be further cached in the above step 220, and the key information of the cached first interactive information can be directly read in the above step 610, without re-extracting the key information in the first interactive information.

[0097] Similarly, with respect to the above step 230, in some other embodiments of the present disclosure, the implementation process of the method for acquiring the knowledge content related to the second interactive information from the knowledge base may be as follows: Figure 7 As shown, it mainly includes:

[0098] In step 710, key information of the first interaction information is extracted from the first interaction information.

[0099] In step 720, key information of the second interaction information is extracted from the second interaction information.

[0100] In step 730, the knowledge base is called to acquire relevant knowledge content based on the key information of the first interaction information and the key information of the second interaction information.

[0101] In step 740, the acquired knowledge content is used as knowledge content related to the second interactive information.

[0102] The implementation method of each step in the above method can refer to the above embodiments and will not be repeated here.

[0103] The knowledge content related to the second interactive information extracted from the knowledge base by the above method will be used as factual information to verify the accuracy of the second interactive information. In addition, by using the key information of the first interactive information as one of the bases for extracting the knowledge content in the process of obtaining the knowledge content related to the above second interactive information, in addition to verifying whether the second interactive information is consistent with the facts, it is also possible to verify whether the above second interactive information is consistent with the first interactive information. Therefore, the second interactive information verified by the above method is not only consistent with the facts but also consistent with the user's input, thereby effectively avoiding the problem of "hallucination" in the model.

[0104] With respect to the above step 240, in some embodiments of the present disclosure, the implementation process of the method for verifying the second interactive information based on the above knowledge content can be as follows: Figure 8 As shown, specifically including:

[0105] In step 810, the above-mentioned knowledge content is used as a premise.

[0106] In step 820, the second interaction information is used as a hypothesis.

[0107] In step 830, the trained reasoning model is called to determine the logical relationship between the premise and the assumption.

[0108] In step 840, it is determined whether the second interaction information passes verification based on the logical relationship between the premise and the assumption.

[0109] In the above information processing method, in the natural language interaction scenario, the logical relationship between the premise and the hypothesis can be judged by natural language inference (NLI), which can generally include: entailment, contradiction, and neutrality. In the embodiment of the present disclosure, the second interactive information that is inconsistent with the above knowledge content can generally be judged as not having passed the verification; while other situations are judged as having passed the verification. Of course, the embodiment of the present disclosure does not limit the specific determination conditions, and other determination conditions can also be used to determine whether the second interactive information has passed the verification.

[0110] With respect to the above step 240, in some other embodiments of the present disclosure, the above verification of the second interactive information based on the knowledge content may include: calling a trained accuracy prediction model based on the knowledge content and the second interactive information to determine the accuracy of the above second interactive information. The above accuracy prediction model may be implemented by LLM or other machine learning models, and is used to classify the second interactive information based on the knowledge content: consistent with the knowledge content or inconsistent with the knowledge content. In the embodiments of the present disclosure, the second interactive information whose classification result is inconsistent with the knowledge content can usually be judged as not having passed the verification; and the second interactive information whose classification result is consistent with the knowledge content can be judged as having passed the verification.

[0111] Furthermore, in some embodiments of the present disclosure, the information processing method may further include: feeding back to the user the second interaction information that has passed the verification; and processing the second interaction information that has not passed the verification for a second time. For example, the secondary processing may include: deleting or modifying based on the knowledge content, etc. The modified second interaction information may be fed back to the user.

[0112] In an embodiment of the present disclosure, in a natural language interaction scenario, the information processing system may feed back the second interaction information to the user through the LUI. Furthermore, if the first interaction information is in text form, the second interaction information may be displayed to the user through the LUI. If the first interaction information is in voice form, before feeding back the second interaction information, the information processing system may further convert the second interaction information into voice by calling a pre-set voice package, and play it to the user through the LUI.

[0113] It should be noted that, in a specific application, in order to ensure the accuracy of the verification, the second interactive information may be first segmented to obtain a plurality of sub-information; then, the operations of steps 230-240 are performed on the obtained multiple sub-information respectively. Then, in the above steps, one or more sub-information that has passed the verification or one or more sub-information that has been further processed are merged and then fed back to the above user. The above method enables the accuracy of the second interactive information to be correctly verified even when the content is large and the logic is complex, thereby making the accuracy of the second interactive information finally output higher and more effectively avoiding the problem of "hallucination" in the model.

[0114] In order to further ensure the security of the second interactive information output by the target generation model, the information processing method described in the embodiment of the present disclosure will further include: performing compliance verification on the verified second interactive information before feeding back the verified second interactive information to the above-mentioned user.

[0115] Specifically, the compliance verification is mainly used to verify whether the second interaction information satisfies one or more preset text output specifications.

[0116] In actual applications, when performing compliance verification, the information processing system may pre-set one or more text output specifications, and match the pre-set text output specifications with the second interactive information one by one. If the matching result meets the text output specification requirements, the compliance verification is passed; if the matching result does not meet one or more text output specification requirements, the compliance verification fails. For example, if a text output specification requires that the second interactive information must not contain one or more pre-set sensitive words, then the text recognition method can be used to verify whether the second interactive information includes the above sensitive words. If it does, the compliance verification fails.

[0117] In other embodiments, the compliance verification can be implemented through a trained LLM or other machine learning model. In these embodiments, the trained LLM or other machine learning model can be called a compliance verification model, which can essentially be a binary classification model for classifying the input second interaction information and determining whether the input second interaction information belongs to a verification pass type or a verification fail type. The generated second interaction information can be directly verified for compliance through the content verification model.

[0118] It can be seen that by performing compliance verification on the second interaction information, it can be basically ensured that the output second interaction information is compliant, thereby ensuring the user experience.

[0119] In order to further improve the accuracy of the second interactive information output by the target generation model, a decision-making capability can also be established for the target generation model during the training of the target generation model described in the embodiments of the present disclosure. In these embodiments, after receiving the prompt information, the trained target generation model can first decide whether it is necessary to obtain external auxiliary information based on the input prompt information; in response to determining that external auxiliary information needs to be obtained, the above-mentioned knowledge base can be called based on the input prompt information to obtain the knowledge content corresponding to the prompt information or to obtain the above-mentioned information processing auxiliary information; then, the above-mentioned second interactive information is generated based on the obtained external auxiliary information and the prompt information. It can be seen that the above-mentioned external auxiliary information will also include knowledge content or the above-mentioned information processing auxiliary information, etc.

[0120] It can be seen that through the above method, the trained target generation model can have decision-making ability. After receiving the first interaction information from the user, it can independently decide whether external auxiliary information is needed to generate the second interaction information, and can obtain corresponding external auxiliary information when necessary to assist in the generation of the second interaction information, so that the generated second interaction information can be more consistent with the first interaction information, which can better meet the needs of users and improve the user experience.

[0121] Corresponding to the above information processing method, an embodiment of the present disclosure further discloses an information processing system. Fig. 9 The internal structure of the information processing system described in the embodiment of the present disclosure is shown. Fig. 9 As shown, the above information processing system may include the following modules:

[0122] Knowledge base 910, including one or more databases for storing knowledge content;

[0123] User interface 920, used to receive first interaction information from a user;

[0124] An interaction module 930 is configured to call a trained target generation model based on the first interaction information to obtain the second interaction information;

[0125] A knowledge content extraction module 940, configured to obtain knowledge content related to the second interactive information from a knowledge base;

[0126] The risk control module 950 is used to verify the second interaction information based on the knowledge content.

[0127] It can be seen that the above-mentioned information processing system, on the basis of realizing human-computer interaction applications, can further verify the accuracy of the generated second interaction information based on the knowledge base, thereby ensuring the accuracy of the second interaction information and effectively avoiding the problem of "hallucination" in the target generation model.

[0128] The specific implementation of each of the above modules can refer to the above methods and drawings, and will not be repeated here. For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or more software and / or hardware. The device of the above embodiment is used to implement the corresponding information processing method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0129] As mentioned above, in addition to being implemented through hard coding methods such as computer code, the above-mentioned interaction module 930, knowledge content extraction module 940, and risk control module 950 and other modules can also be implemented through trained LLM or other machine learning models, thereby making full use of the performance advantages of LLM or other machine learning models.

[0130] In the embodiments of the present disclosure, the above-mentioned design method of dividing the information processing system into multiple functional modules can greatly improve the flexibility of the information processing system. Each module in the above-mentioned information processing system can be implemented by the same or different methods, and each module can be flexibly upgraded or replaced by other modules with the same function but better performance, thereby improving the performance of the entire information processing system.

[0131] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the information processing method described in any of the above embodiments is implemented.

[0132] Fig.10 A more specific hardware structure diagram of an electronic device provided in this embodiment is shown, and the device may include: a processor 2010, a memory 2020, an input / output interface 2030, a communication interface 2040, and a bus 2050. The processor 2010, the memory 2020, the input / output interface 2030, and the communication interface 2040 are connected to each other in communication within the device through the bus 2050.

[0133] The processor 2010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0134] The memory 2020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 2020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 2020 and are called and executed by the processor 2010.

[0135] The input / output interface 2030 is used to connect input / output devices to realize information input and output. The input / output devices can be configured in the device as components, or can be externally connected to the device to provide corresponding functions. The input devices can include microphones, various sensors, etc., and the output devices can include displays, speakers, vibrators, indicator lights, etc.

[0136] The communication interface 2040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0137] The bus 2050 includes a path that transmits information between the various components of the device (eg, the processor 2010, the memory 2020, the input / output interface 2030, and the communication interface 2040).

[0138] It should be noted that, although the above device only shows the processor 2010, the memory 2020, the input / output interface 2030, the communication interface 2040, and the bus 2050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0139] The electronic device of the above embodiment is used to implement the corresponding information processing method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0140] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the information processing method described in any of the above embodiments.

[0141] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0142] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the task processing method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0143] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0144] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0145] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0146] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. An information processing method, comprising: receiving first interaction information from a user; Based on the first interaction information, a trained target generation model is called to obtain second interaction information corresponding to the first interaction information; Acquire knowledge content related to the second interactive information from a knowledge base; as well as The second interaction information is verified based on the knowledge content.

2. The method according to claim 1, wherein: The acquiring knowledge content related to the second interactive information from the knowledge base includes: extracting key information of the second interaction information from the second interaction information; Based on the key information of the second interactive information, calling the knowledge base to obtain relevant knowledge content; and The acquired knowledge content is used as the knowledge content related to the second interactive information.

3. According to the method of claim 1, the acquiring knowledge content related to the second interactive information from a knowledge base comprises: extracting key information of the first interaction information from the first interaction information; as well as Based on the key information of the first interactive information, calling the knowledge base to obtain relevant knowledge content; as well as The acquired knowledge content is used as the knowledge content related to the second interactive information.

4. According to the method of claim 1, the acquiring knowledge content related to the second interactive information from a knowledge base comprises: extracting key information of the first interaction information from the first interaction information; extracting key information of the second interaction information from the second interaction information; as well as Based on the key information of the first interaction information and the key information of the second interaction information, calling the knowledge base to acquire relevant knowledge content; as well as The acquired knowledge content is used as the knowledge content related to the second interactive information.

5. The method according to claim 2 or 4, wherein: The extracting the key information of the second interaction information from the second interaction information includes: Acquire key information of the second interaction information from the target generation model; or, extracting key information of the second interaction information from the second interaction information based on a trained key information extraction model; or, Intent recognition and entity recognition are performed on the second interactive information, and the identified intent information and entity information are used as key information of the second interactive information.

6. The method according to claim 3 or 4, wherein: The extracting the key information of the first interaction information from the first interaction information includes: extracting key information of the first interaction information from the first interaction information based on a trained key information extraction model; or, Intent recognition and entity recognition are performed on the first interactive information, and the identified intent information and entity information are used as key information of the first interactive information.

7. The method according to claim 1, wherein: The verifying the second interactive information based on the knowledge content includes: Take the knowledge content as premise; taking the second interaction information as a hypothesis; Invoking a trained reasoning model to determine a logical relationship between the premise and the hypothesis; and Determining whether the second interaction information passes verification is based on a logical relationship between the premise and the assumption.

8. The method according to claim 1, wherein: The verifying the second interactive information based on the knowledge content includes: A trained accuracy prediction model is called based on the knowledge content and the second interaction information to determine whether the second interaction information passes verification.

9. The method according to claim 3 or 4, wherein: The calling a trained generation model based on the first interaction information to obtain second interaction information corresponding to the first interaction information includes: Based on the key information of the first interactive information, calling the knowledge base to acquire the knowledge content corresponding to the first interactive information; Constructing prompt information corresponding to the first interactive information based on the key information and the knowledge content; and The target generation model is called based on the prompt information to obtain second interaction information corresponding to the prompt information.

10. An information processing system comprising: A knowledge base, including one or more databases for storing knowledge content; A user interface, configured to receive first interaction information from a user; An interaction module, configured to call a trained target generation model based on the first interaction information to obtain the second interaction information; A knowledge content extraction module, used for acquiring knowledge content related to the second interactive information from a knowledge base; as well as A risk control module is used to verify the second interactive information based on the knowledge content.

11. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the information processing method according to any one of claims 1 to 4 and 7 to 8 is implemented.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the information processing method according to any one of claims 1 to 4 and 7 to 8.

13. A computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the information processing method according to any one of claims 1 to 4 and 7 to 8.

Images