Reference verification method, device, client and storage medium

Abstract

The embodiment of the present invention provides a reference verification method, device, client and storage medium, which are applied to a human-computer dialogue application based on generative artificial intelligence AIGC, the method comprising: in response to user input information in a dialogue interface, generating and displaying reply information of the user input information; the reply information comprises reply content and at least one reference badge control, the reply content is at least one of text content, picture content, audio and video content; in response to the user clicking the reference badge control, displaying a reference information display floating layer in the dialogue interface, the reference information display floating layer is used to display the reference verification information corresponding to the reference badge control. In the embodiment of the present invention, the reference verification information is clearly and intuitively displayed through the reference information display floating layer, which is convenient for users to verify the accuracy and integrity of the reply content, thereby improving the transparency and credibility of the reply and improving the user experience.

Description

[Technical field]

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a reference verification method, device, client, and storage medium. [Background technology]

[0002] With the continuous development of the field of artificial intelligence (AI) technology, AI-based human-computer dialogue systems are gradually being applied to various fields. In related technologies, AI-based human-computer dialogue systems usually use large language models (LLMs) as support. In the actual application of LLMs, hallucination problems may exist. The specific manifestation of the hallucination problem is that when the content generated by LLM requires knowledge beyond the training data, LLM may generate output that is inconsistent with the facts, that is, LLM may produce hallucinations, resulting in a poor user experience, and also resulting in low authenticity and credibility of the LLM output content. [Summary of the invention]

[0003] In view of this, the embodiments of the present invention provide a reference verification method, device, client and storage medium to solve the problem in the related art that LLM may produce hallucinations, resulting in poor user experience and low authenticity and credibility of LLM output content.

[0004] In a first aspect, an embodiment of the present invention provides a method for a human-computer dialogue application based on generative artificial intelligence AIGC, characterized in that the method includes:

[0005] In response to user input information in the dialogue interface, generate and display reply information of the user input information; the reply information includes reply content and at least one reference corner control, and the reply content is at least one of text content, picture content, audio and video content;

[0006] In response to the user clicking on the reference badge control, a reference information display floating layer is displayed in the dialog interface, and the reference information display floating layer is used to display the reference verification information corresponding to the reference badge control, so that the user can perform reference verification on the reply content according to the reference verification information.

[0007] In a possible implementation, the reply content includes at least one content to be verified, each content to be verified corresponds to at least one reference badge control, and the reference verification content includes key reference content, and the key reference content is used to indicate the content in the reference verification content that matches the content to be verified; the method further includes:

[0008] In response to the user clicking the reference badge control, the to-be-checked content corresponding to the reference badge control is highlighted, and the key reference content is highlighted on the reference information display floating layer.

[0009] In a possible implementation, in response to the user clicking the reference badge control, after displaying the reference information display floating layer in the dialog interface, the method further includes:

[0010] In response to a sliding operation input by the user on the reference information display floating layer, adjacent reference verification information of the reference verification information is displayed on the reference information display floating layer.

[0011] In a possible implementation, the method further includes:

[0012] In response to the user clicking on the reference badge control, the brightness of the dialog interface is reduced from a first brightness to a second brightness, and the reference information display floating layer is displayed at the first brightness.

[0013] In a possible implementation, the reference verification information includes a reference title control, reference verification content, publication time of the reference verification content, and site information of the reference verification content.

[0014] In a possible implementation, the reference verification content includes at least one piece of content determined from the site corresponding to the site information, or includes a reference description and summary information of the site, and the reference description is used to indicate that the reference verification content is not determined from the site.

[0015] In a possible implementation, in response to the user clicking on the reference badge control, after displaying the reference information display floating layer, the method further includes:

[0016] In response to a floating layer closing operation input by the user, the reference information display floating layer is closed, and the brightness of the dialog interface is restored from the second brightness to the first brightness.

[0017] In a second aspect, an embodiment of the present invention provides a reference verification device, the device comprising:

[0018] A generating module, configured to generate and display reply information of the user input information in response to the user input information in the dialogue interface; the reply information includes reply content and at least one reference corner control, and the reply content is at least one of text content, picture content, audio and video content;

[0019] The first display module is used to display a reference information display floating layer in the dialogue interface in response to the user clicking the reference badge control, and the reference information display floating layer is used to display the reference verification information corresponding to the reference badge control, so that the user can perform reference verification on the reply content according to the reference verification information.

[0020] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0021] A display screen; one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to perform the reference verification method in the first aspect or any possible implementation of the first aspect.

[0022] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the reference verification method in the first aspect or any possible implementation of the first aspect.

[0023] In the technical solution provided by the embodiment of the present invention, the reference verification information is clearly and intuitively displayed through the reference information display floating layer, which is convenient for users to verify the accuracy and integrity of the reply content, thereby improving the transparency and credibility of the reply and improving the user experience. At the same time, the illusion problem of LLM is alleviated, and the authenticity and reliability of LLM output content are improved.

[0024] In the embodiment of the present invention, by displaying the reference verification information, it is ensured that the intellectual property rights of the content creator are respected. At the same time, the user can obtain richer and deeper information and background knowledge by consulting the reference verification information, so as to better understand the background and context of the reply content, further improving the user experience.

Brief Description of the Drawings

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A flowchart of a reference verification method provided by an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of a dialogue interface provided by an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of another dialogue interface provided by an embodiment of the present invention;

[0029] Figure 4 A flowchart of a training method for a reference classification training model provided by an embodiment of the present invention;

[0030] Figure 5 A schematic diagram of a reference classification training model provided by an embodiment of the present invention;

[0031] Figure 6 A schematic diagram of another dialogue interface provided by an embodiment of the present invention;

[0032] Figure 7 A schematic diagram of the structure of a reference verification device provided by an embodiment of the present invention;

[0033] Figure 8 A schematic diagram of a client provided by an embodiment of the present invention. [Specific implementation method]

[0034] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] It should be clear that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0037] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0038] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0039] Figure 1 A flowchart of a reference verification method provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0040] Step 101: In response to user input information in a dialogue interface, generate and display reply information to the user input information.

[0041] Each step of the embodiment of the present invention may be performed by a client, wherein the electronic device includes but is not limited to a mobile phone, a tablet computer, a portable PC, a desktop computer, and the like.

[0042] The embodiment of the present invention is applied to a human-computer dialogue application based on generative artificial intelligence (AIGC). AIGC is a general artificial intelligence (AGI) that focuses on generating or creating new content. It generates new text, program code, images, videos, sounds and other data based on existing large-scale multimodal data sets through machine learning. It can handle various tasks and is applied to various scenarios, such as text summarization, text generation, machine translation, question and answer, etc., with a high level of generalization. The application of AIGC technology can improve the efficiency of content generation, save time and cost for creators and enterprises, and expand the possibility of content creation. The basic support of AIGC is the large language model LLM, which is a language model containing billions or more parameters. It generates human-like text by learning and understanding a large amount of text data and context data. For example, LLM includes Generative Pre-Trained Transformer (GPT)-3, GPT-4, Baichuan2-7B, Baichuan2-13B, etc. LLM has undergone comprehensive training including pre-training, supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF). In the pre-training stage, the model learns the language structure, grammar and factual knowledge in the text data through a large amount of text data. In the SFT stage, the pre-trained LLM is fine-tuned with labeled data to make it more suitable for performing a specific task. In the RLHF stage, the reinforcement learning method is used to directly optimize the language model using human feedback signals.

[0043] After the client obtains the user input information, it will process the user input information through the intention understanding module. The intention understanding module is used to determine whether it is necessary to call the retrieval-augmented generation technology (RAG for short) to process the user input information, that is, to achieve intention understanding. The intention understanding module can achieve intention understanding through relevant existing technologies. For example, the user input information is input into the trained input information classification model, and the classified user input information is output. The classified user input information includes the user input information and the category of the user input information. According to the category of the user input information, it is determined whether it is necessary to call the RAG technology to process the user input information to achieve intention understanding. In addition, intention understanding can also be achieved in other ways, which is not limited in the embodiments of the present invention. Specifically, the implementation of RAG technology mainly includes the following main steps: data extraction, data indexing, data retrieval, data sorting and data generation. Among them, the data retrieval link is the key link in obtaining effective information. Similarity retrieval and keyword retrieval can be used, and then sorting is performed to obtain data that is more relevant to the user input information.

[0044] In this step, the dialogue interface includes user input information and reply information, the reply information includes reply content and at least one reference badge control, the reply content includes at least one content to be verified, and the reply content is at least one of text content, picture content, audio and video content.

[0045] Figure 2 A schematic diagram of a dialogue interface provided by an embodiment of the present invention, such as Figure 2 As shown, the reply information includes reply content and five reference superscript controls, and the reply content includes text content. Among them, the five reference superscript controls are respectively a first superscript control, a second superscript control, a third superscript control, a fourth superscript control and a fifth superscript control.

[0046] like Figure 2 As shown, a nickname is displayed at the top of the dialogue interface, and a text input box and a voice input control are set at the bottom of the dialogue interface. Among them, the user can input the demand in text form through the text input box, or input the demand in voice form through the voice input control.

[0047] Step 102: In response to the user clicking on the reference badge control, a reference information display floating layer is displayed in the dialog interface.

[0048] In this step, the reference information display floating layer is used to display the reference verification information corresponding to the reference superscript control, so that the user can perform reference verification on the reply content according to the reference verification information. The reference verification information includes the reference title control, the reference verification content, the publication time of the reference verification content, and the site information of the reference verification content. Optionally, the reference verification information also includes the reference serial number. Among them, the site information of the reference verification content includes the site URL, site icon and site name. For example, the site can include official websites such as Zhihu, Baidu Encyclopedia, Sogou Q&A, Sohu News, etc., to ensure the authenticity and accuracy of the reference verification content. The reference verification content includes at least one section of content determined from the site corresponding to the site information; or, the reference verification content includes a reference description and summary information of the site, and the reference description is used to indicate that the reference verification content has not been determined from the site. For example, the reference verification content is a paragraph of text, the number of lines of a paragraph of text is limited to 3 lines, and the content exceeding 3 lines is represented by... For another example, the reference verification content is multiple paragraphs of text, the number of lines of multiple paragraphs of text is limited to 8 lines, and multiple paragraphs are separated by..., and the content exceeding 8 lines is represented by... For another example, the citation verification content includes the citation description and the summary information of the site. The citation description is a text description of "No direct content citation was found, the following is a site summary for reference." The number of lines of the summary information is limited to 3 lines, and information exceeding 3 lines is indicated by...

[0049] Further, the user clicks the reference badge control, and in response to the user clicking the reference badge control, the client reduces the brightness of the dialog interface from the first brightness to the second brightness, and displays the reference information display floating layer at the first brightness, thereby highlighting the reference information display floating layer. The second brightness is lower than the first brightness.

[0050] In the embodiment of the present invention, after step 102, the method may further include: in response to a floating layer closing operation input by the user, closing the reference information display floating layer, and restoring the brightness of the dialog interface from the second brightness to the first brightness.

[0051] Figure 3 A schematic diagram of another dialogue interface provided by an embodiment of the present invention, such as Figure 3 As shown, the user clicks on the first reference badge control, and the client responds to the user's operation of clicking on the first reference badge control by displaying a reference information display floating layer in the dialog interface, and displays the reference verification information corresponding to the first reference badge control on the reference information display floating layer. The reference information display floating layer is displayed at the bottom of the dialog interface, and the reference information display floating layer displays the words "reference verification information", the reference number is 1, and the reference verification content is "In a certain sports event in XXXX, Team A won 51 gold, 21 silver, and 28 bronze medals, a total of 100 medals. The number of gold medals ranked first, which is the best result of the team since XXXX." The publication time of the reference verification content is 2008-12-09.

[0052] Further, the user clicks the reference title control, and the client responds to the user's operation of clicking the reference title control, and jumps to the reference verification content display interface associated with the reference title control according to the site information of the reference verification content. The reference verification content display interface is a HyperText Markup Language 5 (HTML5) interface.

[0053] In an embodiment of the present invention, the reference verification content includes key reference content, and the key reference content is used to indicate the content in the reference verification content that matches the content to be verified. The user clicks the reference badge control, and the client highlights the content to be verified corresponding to the reference badge control in response to the user's operation of clicking the reference badge control, and highlights the key reference content on the reference information display floating layer. Specifically, the client inputs the content to be verified and the reference verification content into the reference classification training model, and outputs the key reference content.

[0054] like Figure 3 As shown, the reply content includes two contents to be verified, namely the first content to be verified and the second content to be verified, and each content to be verified corresponds to at least one reference badge control. Among them, the first content to be verified is "51 gold medals, 21 silver medals and 28 bronze medals", and the second content to be verified is "26 gold medals, 18 silver medals and 26 bronze medals". The first content to be verified corresponds to the first badge control, the second badge control, the third badge control and the fourth badge control, and the second content to be verified corresponds to the fifth badge control. The user clicks on the first reference badge control, and the client highlights the content to be verified corresponding to the first reference badge control in response to the user clicking on the first reference badge control, that is, the first content to be verified is highlighted, and the key reference content is highlighted on the reference information display floating layer. Specifically, the reference verification content corresponding to the first content to be verified is "In a sports event in XXXX, Team A won 51 gold, 21 silver, and 28 bronze medals, a total of 100 medals. The number of gold medals ranked first, which is the team's best result since XXXX." The client inputs the first content to be verified and the reference verification content into the reference classification training model, and outputs the key reference content. The key reference content is "51 gold, 21 silver, and 28 bronze medals, a total of 100 medals."

[0055] Figure 4 A flowchart of a training method for a reference classification training model provided by an embodiment of the present invention, such as Figure 4 As shown, the method includes:

[0056] Step S1: label the acquired random samples to generate label data.

[0057] In this step, the random sample includes the content to be verified and the reference verification content corresponding to the content to be verified, and the annotation data includes the first annotation data and the second annotation data. The training server divides the random sample into the first random sample and the second random sample according to the preset random sample ratio, and the random sample ratio is used to indicate the number ratio of the first random sample and the second random sample. For example, the random sample ratio is 1:2, that is, the number ratio of the first random sample and the second random sample is 1:2. The first random sample is annotated to generate the first annotation data; the longest common substring of the content to be verified and the reference verification content corresponding to the content to be verified in the second random sample is calculated; it is determined whether the length of the longest common substring is less than the length of the answer of the preset ratio; if the length of the longest common substring is greater than or equal to the length of the answer of the preset ratio, it is determined that the second random sample is a simple sample and the simple sample is filtered; if the length of the longest common substring is less than the length of the answer of the preset ratio, it is determined that the second random sample is a complex sample, and the complex sample is annotated to generate the second annotation data. For example, the preset ratio is 80%, and the size of the preset ratio can be adjusted according to the actual situation, and the embodiment of the present invention is not limited to this.

[0058] For simple samples, the model can determine the key reference content through simple word-by-word comparison without semantic understanding. If there are too many simple samples in the random sample, the model will not be able to learn the semantic correspondence between the content to be verified and the reference verification content, which will cause the model to be unable to accurately determine the key reference content in more complex semantic scenarios, resulting in a poor user experience.

[0059] In an embodiment of the present invention, by dividing the random samples into a first random sample and a second random sample, filtering the simple samples in the second random sample, and only annotating the complex samples in the second random sample, the model can learn the semantic correspondence in the complex samples and accurately determine the key reference content, thereby improving the user experience.

[0060] Step S2: input the annotated data into the established citation classification training model to train the citation classification training model and generate a trained citation classification training model.

[0061] In the embodiment of the present invention, the specific implementation method of reference verification is to generate reply information corresponding to the user input information based on RAG, input the reply information corresponding to the user input information into the trained reference verification classification model, and output the reference verification content. Among them, RAG provides additional information to LLM through external knowledge sources, so that LLM can generate more accurate and more contextual content, reduce the generation of erroneous information, alleviate the illusion problem of LLM, and improve the authenticity and reliability of LLM output content.

[0062] Figure 5 A schematic diagram of a reference classification training model provided by an embodiment of the present invention, such as Figure 5 As shown in FIG. 1 , the citation classification training model includes a Transformer module, a token classification layer, a BIO annotation layer, a citation classification layer, and a conditional random field (CRF) layer.

[0063] The Transformer module is used to process the input sequence based entirely on the self-attention mechanism. The Transformer module consists of an encoder and a decoder. The encoder is responsible for encoding the input sequence into a series of feature representations, and the decoder uses these feature representations to generate the output sequence. Each encoder and decoder consists of multiple layers, each of which contains a self-attention mechanism and a feedforward neural network.

[0064] The token classification layer is used to divide text data into at least one token, which can be understood as the smallest unit in the text. For the model, a token is a digital representation. Each token is associated with a unique digital ID, and the model uses these IDs to distinguish different tokens. During the training process, the model learns how to map text to these digital IDs so that it can encode and decode new text.

[0065] The BIO annotation layer is used to perform sequence annotation on text data using the BIO annotation method to mark entities in the text. In the BIO annotation method, B represents the beginning of an entity, I represents the inside of an entity, and O represents no entity. The BIO annotation layer automatically divides the text into units of meaning, such as vocabulary, part of speech, phrases, syntactic structures, etc., to help the model analyze and understand text data.

[0066] The citation classification layer is used to classify the text data as a whole according to whether there is a citation relationship between the text data, and divide the text data into two categories: those with citation relationships and those without citation relationships.

[0067] The CRF layer is used to automatically learn constraint information from the training data and add some constraints to the labels predicted by the model based on the constraint information to ensure the validity and legitimacy of the predicted labels.

[0068] In this step, the classification training model is referenced to learn the target loss function strategy, and the difference between the predicted value and the true value, that is, the loss value, is calculated. After obtaining the loss value, the model needs to update the model parameters to reduce the loss between the true value and the predicted value, that is, to make the predicted value generated by the model closer to the true value, so as to achieve the purpose of learning. In the embodiment of the present invention, the loss value is the sum of the loss of the BIO annotation layer, the loss of the CRF layer and the loss of the reference classification layer.

[0069] In the embodiment of the present invention, after the trained citation classification training model is generated, the trained citation classification training model can be verified by automatic evaluation or manual evaluation.

[0070] Step 103: In response to a sliding operation input by the user on the reference information display floating layer, adjacent reference verification information of the reference verification information is displayed on the reference information display floating layer.

[0071] Figure 6 A schematic diagram of another dialogue interface provided by an embodiment of the present invention, such as Figure 6 As shown, the user inputs a horizontal sliding operation on the reference information display floating layer, and the client responds to the horizontal sliding operation input by the user on the reference information display floating layer, and displays the adjacent reference verification information of the reference verification information on the reference information display floating layer. Among them, the adjacent verification information is the reference verification information corresponding to the second reference superscript control. The reference number of the adjacent verification information is 2, and the reference verification content is "In a certain sports event in XXXX, Team A won 51 gold medals, 21 silver medals, and 28 bronze medals, a total of 100 medals, ranking first in the gold medal list." The publication time of the reference verification content is 2009-09-09.

[0072] In the technical solution provided by the embodiment of the present invention, the reference verification information is clearly and intuitively displayed through the reference information display floating layer, which is convenient for users to verify the accuracy and integrity of the reply content, thereby improving the transparency and credibility of the reply and improving the user experience. At the same time, the illusion problem of LLM is alleviated, and the authenticity and reliability of LLM output content are improved.

[0073] In the embodiment of the present invention, by displaying the reference verification information, it is ensured that the intellectual property rights of the content creator are respected. At the same time, the user can obtain richer and deeper information and background knowledge by consulting the reference verification information, so as to better understand the background and context of the reply content, further improving the user experience.

[0074] Figure 7 A schematic diagram of a reference verification device provided by an embodiment of the present invention is shown in FIG. Figure 7As shown, the device includes a generation module 11 and a first display module 12, and the generation module 11 is connected to the first display module 12. The generation module 11 is used to generate and display reply information of the user input information in response to the user input information in the dialogue interface; the reply information includes reply content and at least one reference badge control, and the reply content is at least one of text content, picture content, and audio and video content. The first display module 12 is used to display a reference information display floating layer in the dialogue interface in response to the user clicking the reference badge control, and the reference information display floating layer is used to display the reference verification information corresponding to the reference badge control, so that the user can perform reference verification on the reply content according to the reference verification information.

[0075] In an embodiment of the present invention, the reply content includes at least one content to be verified, each content to be verified corresponds to at least one reference badge control, and the reference verification content includes key reference content, and the key reference content is used to indicate the content in the reference verification content that matches the content to be verified. The device also includes a highlight module 13, and the highlight module 13 is connected to the first display module 12. The highlight module 13 is used to highlight the content to be verified corresponding to the reference badge control in response to the user clicking the reference badge control, and highlight the key reference content on the reference information display floating layer.

[0076] In the embodiment of the present invention, the device further includes a second display module 14, which is connected to the first display module 12. The second display module 14 is used to display adjacent reference verification information of the reference verification information on the reference information display floating layer in response to a sliding operation input by a user on the reference information display floating layer.

[0077] In the embodiment of the present invention, the device further includes a third display module 15, which is connected to the first display module 12. The third display module 15 is used to reduce the brightness of the dialogue interface from the first brightness to the second brightness in response to the user clicking the reference badge control, and display the reference information display floating layer at the first brightness.

[0078] In the embodiment of the present invention, the device further includes a closing module 16, which is connected to the first display module 12 and the third display module 15. The closing module 16 is used to respond to the floating layer closing operation input by the user, close the reference information display floating layer, and restore the brightness of the dialogue interface from the second brightness to the first brightness.

[0079] In the technical solution provided by the embodiment of the present invention, the reference verification information is clearly and intuitively displayed through the reference information display floating layer, which is convenient for users to verify the accuracy and integrity of the reply content, thereby improving the transparency and credibility of the reply and improving the user experience. At the same time, the illusion problem of LLM is alleviated, and the authenticity and reliability of LLM output content are improved.

[0080] In the embodiment of the present invention, by displaying the reference verification information, it is ensured that the intellectual property rights of the content creator are respected. At the same time, the user can obtain richer and deeper information and background knowledge by consulting the reference verification information, so as to better understand the background and context of the reply content, further improving the user experience.

[0081] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute an embodiment of the reference verification method described above.

[0082] Figure 8 A schematic diagram of a client provided for an embodiment of the present invention includes: the client 3 of the embodiment includes: a processor 31, a memory 32, and a computer program 33 stored in the memory 32 and executable on the processor 31. When the computer program 33 is executed by the processor 31, the reference verification method in the embodiment is implemented. To avoid repetition, they are not described one by one here.

[0083] The client 3 includes, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will appreciate that Figure 8 This is only an example of the client 3 and does not constitute a limitation on the client 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the network device may also include input and output devices, network access devices, buses, etc.

[0084] The processor 31 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0085] The memory 32 may be an internal storage unit of the client 3, such as a hard disk or memory of the client 3. The memory 32 may also be an external storage device of the client 3, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the client 3. Further, the memory 32 may also include both an internal storage unit of the client 3 and an external storage device. The memory 32 is used to store computer programs and other programs and data required by the network device. The memory 32 may also be used to temporarily store data that has been output or is to be output.

[0086] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0087] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0088] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0089] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0090] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A reference verification method, applied to a human-computer dialogue application based on generative artificial intelligence AIGC on a mobile phone, characterized in that: The method comprises: In response to user input information in the dialogue interface, generate and display reply information of the user input information; the reply information includes reply content and at least one reference badge control, the reply content is at least one of text content, picture content, audio and video content, wherein the reply content includes at least one content to be verified, and each content to be verified corresponds to at least one reference badge control; In response to the user clicking the reference badge control, the brightness of the dialog interface is reduced from the first brightness to the second brightness, and a reference information display floating layer is displayed on the dialog interface at the first brightness, the reference information display floating layer is used to display reference verification information corresponding to the reference badge control, the reference verification information includes reference verification content, reference title control, and site information of the reference verification content; the reference verification content includes key reference content, and the key reference content is used to indicate the content in the reference verification content that matches the content to be verified; wherein the reference verification content is obtained by inputting reply information to the user input information into a reference verification classification model and then outputting it, the reference verification information and the reply information in the dialog interface are displayed on the screen at the same time, and the key reference content in the reference verification information and the content to be verified corresponding to the reference badge control are both highlighted; In response to a click operation on the reference title control, the interface jumps to a reference verification content display interface associated with the reference title control according to the site information of the reference verification content.

2. The method according to claim 1, characterized in that The reference verification content includes at least one piece of content determined from the site corresponding to the site information, or includes a reference description and summary information of the site, wherein the reference description is used to indicate that the reference verification content is not determined from the site.

3. The method according to claim 1, characterized in that Also includes: In response to a closing operation on the reference information display floating layer, the reference information display floating layer is closed, and the brightness of the dialog interface is restored from the second brightness to the first brightness.

4. The method according to claim 1, characterized in that: In response to a user's sliding operation on the reference information display floating layer, adjacent reference verification information of the reference verification information is displayed on the reference information display floating layer.

5. A reference verification device, characterized in that: The device comprises: A generating module, configured to generate and display reply information of the user input information in response to the user input information in the dialogue interface; the reply information includes reply content and at least one reference badge control, the reply content is at least one of text content, picture content, audio and video content, wherein the reply content includes at least one content to be verified, and each content to be verified corresponds to at least one reference badge control; A first display module is used to display a reference information display floating layer in the dialog interface in response to the user clicking the reference badge control, the reference information display floating layer is used to display the reference verification information corresponding to the reference badge control, so that the user can perform reference verification on the reply content according to the reference verification information, the reference verification information includes reference verification content, reference title control, and site information of the reference verification content; the reference verification content includes key reference content, and the key reference content is used to indicate the content in the reference verification content that matches the content to be verified; wherein the reference verification content is obtained by inputting the reply information of the user input information into the reference verification classification model and then outputting it, and the reference verification information and the reply information in the dialog interface are displayed on the screen at the same time; A highlight module, configured to highlight both the key reference content in the reference verification information and the content to be verified corresponding to the reference badge control in response to the user clicking the reference badge control; A third display module is used to reduce the brightness of the dialog interface from the first brightness to the second brightness in response to the user clicking the reference corner control, and display the reference information display floating layer at the first brightness; In response to a click operation on the reference title control, the interface jumps to a reference verification content display interface associated with the reference title control according to the site information of the reference verification content.

6. A client, characterized in that: include: Display screen; one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the client, enable the client to execute the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 4.

Images