Information processing method and apparatus, electronic device, and computer program product

Abstract

An information processing method and apparatus, an electronic device, and a computer program product, relating to the technical field of artificial intelligence. The method comprises: acquiring a plurality of first retrieval results for target information (S201); constructing prompt information on the basis of the target information and a first retrieval result among the plurality of first retrieval results to obtain a plurality of pieces of first prompt information (S202); processing the plurality of pieces of first prompt information by means of a target natural language model to obtain a probability value corresponding to first prompt information among the plurality of pieces of first prompt information (S203); and on the basis of the probability value, determining from the plurality of first retrieval results target reply information corresponding to the target information (S204). The method solves the technical problem in the related art of low accuracy of reply information of a large model due to the ranking of retrieved content relying on a manual supervision signal during question answering over document knowledge based on said model.

Description

Information processing method and device, electronic device, and computer program product

[0001] The present disclosure claims priority to Chinese Patent Application No. 202410961237.X, filed on July 17, 2024 with the Chinese Patent Office and entitled "Information processing method and device, electronic device, and computer program product", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present disclosure relates to the field of artificial intelligence, and in particular, to an information processing method and device, an electronic device, and a computer program product. BACKGROUND

[0003] With the development of artificial intelligence, a large model is often used to ask and answer knowledge in a document. To this end, the content of the document needs to be chunked first, and then the chunk content is roughly recalled for the question. However, in the collection of the rough recall stage, there are often a large number of non-precise matching items, and the information that is truly relevant to the question is often ranked last. Due to the limitation of the number of semantic units (tokens) of the large model, these effective information often cannot be sent into the large model, which directly affects the effect of asking and answering. In the prior art, artificial supervision signals are generally used to accurately sort the recalled chunks, which requires a large amount of manpower and has weak generalization ability. In addition, in different fields, the recalled information can be diverse, including not only semantic information of text, but also author information, time, document title, and other information.

[0004] In view of the problem in the prior art that the large model is used to ask and answer knowledge in a document, and artificial supervision signals are used to sort the recalled content, resulting in low accuracy of the model's reply information, no effective solution has been proposed so far. SUMMARY

[0005] Embodiments of the present disclosure provide an information processing method and device, an electronic device, and a computer program product to at least solve the technical problem in the prior art that the large model is used to ask and answer knowledge in a document, and artificial supervision signals are used to sort the recalled content, resulting in low accuracy of the model's reply information.

[0006] According to an aspect of an embodiment of the present disclosure, an information processing method is provided, including: obtaining a plurality of first recall results for target information, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information; constructing prompt information according to the target information and a first recall result in the plurality of first recall results to obtain a plurality of first prompt information; processing the plurality of first prompt information by a target natural language model to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, wherein the probability value represents a probability that a first recall result in the first prompt information contains information required by the target information; and determining target reply information corresponding to the target information from the plurality of first recall results according to the probability value.

[0007] Further, constructing prompt information according to the target information and the first recall result in the plurality of first recall results to obtain the plurality of first prompt information includes: splicing the target information and the first recall result in the plurality of first recall results to obtain a plurality of first question and answer pairs; constructing prompt information according to a pre-set binary classification text and a first question and answer pair in the plurality of first question and answer pairs to obtain a plurality of initial prompt information; and splicing a domain to which the target information belongs and the plurality of initial prompt information to obtain the plurality of first prompt information.

[0008] Further, processing the plurality of first prompt information by the target natural language model to obtain the probability value corresponding to the first prompt information in the plurality of first prompt information includes: processing the plurality of first prompt information by the target natural language model in parallel to obtain a logit value corresponding to the first prompt information in the plurality of first prompt information; and calculating the probability value according to the logit value.

[0009] Further, before processing the plurality of first prompt information by the target natural language model to obtain the probability value corresponding to the first prompt information in the plurality of first prompt information, the method further includes: obtaining a target training sample set, wherein the target training sample set includes a plurality of questions and a plurality of second recall results corresponding to a question in the plurality of questions; and training an initial natural language model according to the target training sample set to obtain the target natural language model.

[0010] Further, the obtaining the target training sample set comprises: obtaining the plurality of questions and the plurality of second recall results; splicing a question in the plurality of questions and a plurality of second recall results corresponding to the question to obtain a plurality of second question and answer pairs; and constructing a target training sample set according to the plurality of second question and answer pairs and a true label of a second question and answer pair in the plurality of second question and answer pairs, wherein the true label is used to represent whether the second recall result in the second question and answer pair includes information required by the question.

[0011] Further, the true label is obtained by the following steps: constructing prompt information according to the plurality of second question and answer pairs and a preset binary classification text to obtain a plurality of second prompt information; processing the plurality of second prompt information by a second natural language model to obtain reply information corresponding to the plurality of second prompt information, wherein the reply information is used to represent whether the recall result in the second prompt information includes information required by the question; and determining the true label according to the reply information.

[0012] Further, the obtaining the plurality of second recall results comprises: receiving the plurality of questions through a target interface and inputting the plurality of questions into a first natural language model; processing the plurality of questions by the first natural language model to obtain a plurality of reply information corresponding to the plurality of questions; and determining the plurality of second recall results from the plurality of reply information according to an order of the reply information.

[0013] Further, the determining the target reply information corresponding to the target information from the plurality of first recall results according to the probability value comprises: performing descending order arrangement processing on the plurality of first recall results according to the probability value to obtain a sorting result of the first recall results; and determining a first recall result with an arrangement order satisfying a preset requirement as the target reply information according to the sorting result.

[0014] According to another aspect of the embodiments of the present disclosure, an information processing method is also provided, comprising: obtaining a plurality of first recall results for target information sent by a client, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information; constructing prompt information in a cloud server according to the target information and a first recall result in the plurality of first recall results to obtain a plurality of first prompt information; processing the plurality of first prompt information by a target natural language model to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, wherein the probability value is used to represent a probability that the first recall result in the first prompt information contains information required by the target information; determining target reply information corresponding to the target information from the plurality of first recall results according to the probability value; and returning the target reply information to the client.

[0015] According to another aspect of the embodiments of the present disclosure, an information processing apparatus is also provided, comprising: a first obtaining unit configured to obtain a plurality of first recall results for target information, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information; a first constructing unit configured to construct prompt information according to the target information and a first recall result in the plurality of first recall results, to obtain a plurality of first prompt information; a processing unit configured to process the plurality of first prompt information by a target natural language model, to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, wherein the probability value is used to represent a probability that the first recall result in the first prompt information contains information required by the target information; and a sorting unit configured to determine target reply information corresponding to the target information from the plurality of first recall results according to the probability value.

[0016] Further, the constructing unit comprises: a splicing module configured to splice the target information and the first recall result in the plurality of first recall results, to obtain a plurality of first question-answer pairs; a constructing module configured to construct prompt information according to a pre-set binary classification text and a first question-answer pair in the plurality of first question-answer pairs, to obtain a plurality of initial prompt information; and a first determining module configured to splice a domain to which the target information belongs and the plurality of initial prompt information, to obtain the plurality of first prompt information.

[0017] Further, the processing unit comprises: a processing module configured to process the plurality of first prompt information concurrently by the target natural language model, to obtain a logit value corresponding to the first prompt information in the plurality of first prompt information; and a calculating module configured to calculate according to the logit value, to obtain the probability value.

[0018] Further, the apparatus further comprises: a second obtaining unit configured to obtain a target training sample set before processing the plurality of first prompt information by the target natural language model, to obtain the probability value corresponding to the first prompt information in the plurality of first prompt information, wherein the target training sample set comprises a plurality of questions and a plurality of second recall results corresponding to a question in the plurality of questions; and a training unit configured to train an initial natural language model according to the target training sample set, to obtain the target natural language model.

[0019] Further, the second obtaining unit comprises: an obtaining module, configured to obtain the plurality of questions and the plurality of second retrieval results; a splicing module, configured to splice a question in the plurality of questions and the plurality of second retrieval results corresponding to the question to obtain a plurality of second question-answer pairs; and a constructing module, configured to construct a target training sample set according to the plurality of second question-answer pairs and a true label of a second question-answer pair in the plurality of second question-answer pairs, wherein the true label is used to represent whether the second retrieval result in the second question-answer pair includes information required by the question.

[0020] Further, the true label is obtained by using the following device: a second constructing unit, configured to construct a plurality of second prompt information according to the plurality of second question-answer pairs and a preset binary classification text; a processing unit, configured to process the plurality of second prompt information by using a second natural language model to obtain reply information corresponding to the plurality of second prompt information, wherein the reply information is used to represent whether a retrieval result in the second prompt information includes information required by the question; and a determining unit, configured to determine the true label according to the reply information.

[0021] Further, the obtaining module comprises: a receiving submodule, configured to receive the plurality of questions through a target interface and input the plurality of questions into a first natural language model; a processing submodule, configured to process the plurality of questions by using the first natural language model to obtain a plurality of reply information corresponding to the plurality of questions; and a determining submodule, configured to determine the plurality of second retrieval results from the plurality of reply information according to an order of the reply information in the plurality of reply information.

[0022] Further, the sorting unit comprises: a sorting module, configured to perform descending order arrangement processing on the plurality of first retrieval results according to the probability value to obtain a sorting result of the first retrieval results; and a second determining module, configured to determine, according to the sorting result, a first retrieval result whose arrangement order meets a preset requirement as the target reply information.

[0023] According to another aspect of the embodiments of the present application, an electronic device is also provided, which comprises a memory storing an executable program and a processor configured to run the program, wherein the program is configured to implement the information processing method according to any one of the above embodiments.

[0024] According to another aspect of the embodiments of the present application, a computer program product is also provided, which comprises a stored computer program, and the computer program is configured to implement the information processing method according to any one of the above embodiments when the computer program is run by a processor.

[0025] In the embodiment of the present disclosure, the following steps are adopted: obtaining a plurality of first recall results for target information, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information; constructing prompt information according to the target information and a first recall result in the plurality of first recall results to obtain a plurality of first prompt information; processing the plurality of first prompt information by a target natural language model to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, wherein the probability value is used to represent a probability that a first recall result in the first prompt information contains information required by the target information; and determining target reply information corresponding to the target information from the plurality of first recall results according to the probability value, thereby solving the technical problem in the related art that the content of recall is sorted by relying on artificial supervision signals when a large model is used to ask questions and answer knowledge in a document, resulting in a relatively low accuracy of model reply information. In the present scheme, after obtaining a plurality of first recall results based on the target information by the first natural language model, a first prompt information is constructed by the target information and the first recall result, the first prompt information is used to guide the target natural language model to output a probability that the first recall result contains information required by the target information, and then the probability value is used to realize accurate sorting of the first recall result, thereby achieving the effect of improving the accuracy of model reply information. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings, which are included to provide a further understanding of the present disclosure and constitute a part of the present disclosure, illustrate the illustrative embodiments of the present disclosure and their description serve to explain the present disclosure, and do not limit the present disclosure in any way. In the drawings:

[0027] FIG. 1 is a hardware structure block diagram of a computer terminal according to an embodiment of the present disclosure;

[0028] FIG. 2 is a flowchart of an information processing method according to an embodiment of the present disclosure;

[0029] FIG. 3 is a schematic diagram of a target interface according to an embodiment of the present disclosure;

[0030] FIG. 4 is a flowchart of an information processing method according to an embodiment of the present disclosure;

[0031] FIG. 5 is a flowchart of an information processing method according to an embodiment of the present disclosure;

[0032] FIG. 6 is a schematic diagram of an information processing device according to an embodiment of the present disclosure;

[0033] FIG. 7 is a structure block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to enable personnel in the technical field to better understand the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in combination with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor should fall within the scope of protection of the present disclosure.

[0035] It should be noted that the terms "first", "second", and the like in the specification and claims of the present disclosure and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] First, some nouns or terms that appear in the description of the embodiments of the present disclosure are applicable to the following explanations:

[0037] Precise ranking: In search algorithms, a process often retrieves a part of content or goods that may be related to user interest from a large-scale candidate set. The goal of the recall stage is to cover as many objects that users may be interested in as possible. However, the results of recall often contain many non-precise matching items. Therefore, it is necessary to sort the recalled items more accurately after recall to determine the relevant and effective items in the front.

[0038] Large model, also known as large-scale pre-training model, refers to those machine learning models, especially deep learning models, with a large number of parameters (usually hundreds of millions to tens of billions of parameters). These models have learned rich language structures, semantic knowledge, and even cross-domain general knowledge through pre-training on large-scale unlabeled data sets. Due to its large size and rich prior knowledge, large models have shown superior performance in many natural language processing, image recognition, and even multi-modal tasks, and can achieve more complex language understanding and generation capabilities.

[0039] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards in the relevant region, and provide corresponding operation portal for user to choose authorization or refusal.

[0040] Embodiment 1

[0041] According to the embodiments of the present disclosure, an information processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from here.

[0042] The method provided by the embodiment of the present disclosure can be executed in a mobile terminal, a computer terminal or a similar computing device. FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an information processing method. As shown in FIG. 1, the computer terminal (or mobile device) 10 can include a processor set 102 (the processor set 102 can include but is not limited to a processing device such as a microprocessor MCU or a field programmable gate array (FPGA), and the processor set 102 can include a processor set, which is shown as 102a, 102b, …, 102n in FIG. 1), a memory 104 for storing data, and a transmission module 106 for communication function. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that the structure shown in FIG. 1 is only schematic, and it does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or fewer components than those shown in FIG. 1, or have a different configuration from that shown in FIG. 1.

[0043] It should be noted that the one or more processors 102 and / or other data processing circuitry described above can be referred to herein generally as "data processing circuitry". The data processing circuitry can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. Furthermore, the data processing circuitry can be a single standalone processing module, or incorporated in whole or in part within any of the other elements of the computer terminal 10 (or mobile device). As referred to in embodiments of the present disclosure, the data processing circuitry functions as a processor to control, for example, selection of a variable resistance terminal path connected to an interface.

[0044] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage means corresponding to the information processing method in embodiments of the present disclosure. The processor 102 can execute various functional applications and data processing, i.e., implement the information processing method described above, by running the software programs and modules stored in the memory 104. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory disposed remotely with respect to the processor 102, which can be connected to the computer terminal 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0045] The transmission device 106 is configured to receive or send data via a network. Examples of the network include, but are not limited to, a wireless network provided by a communication service provider of the computer terminal 10. In one example, the transmission device 106 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module configured to communicate with the Internet in a wireless manner.

[0046] The display can be a touch screen liquid crystal display that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0047] In the above operating environment, the present disclosure provides an information processing method as shown in FIG. 2. FIG. 2 is a flowchart I of the information processing method according to Embodiment One of the present disclosure. As shown in FIG. 2, the information processing method includes:

[0048] In step S201, a plurality of first recall results for target information are obtained, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information;

[0049] Optionally, the target information can be a question raised by a user. The user can input information that the user wants to ask through an interactive interface, and then input the target information into the first natural language model through the interactive interface. The target information is processed by the first natural language model, and then a plurality of first recall results are obtained from the corresponding database. It should be noted that the first recall result is information that the first natural language model considers to answer the target information, which is a rough recall result. It should be noted that the first natural language model can be a natural language model that can be used for text retrieval after training. The first natural language model can be a deep learning model.

[0050] In step S202, prompt information is constructed according to the target information and the first recall result in the plurality of first recall results, and a plurality of first prompt information is obtained.

[0051] Optionally, the prompt (i.e., the first prompt information) is constructed by the target information and the first recall result in the plurality of first recall results. For example, the target information and the first recall result are spliced to obtain the prompt. It should be noted that the prompt is used to guide the target natural language model to output the probability that the first recall result in the first prompt information contains the information required by the target information. For example, the target information includes N first recall results, and N first prompt information is constructed accordingly. The prompt word is optimized by the target information and the corresponding recall result, so that the model can score according to the amount of information containing the answer to the question in the paragraph text (i.e., the recall information), and improve the accuracy of subsequent accurate sorting of the recall result.

[0052] In step S203, the plurality of first prompt information is processed by the target natural language model to obtain a probability value corresponding to the first prompt information in the plurality of first prompt information, wherein the probability value represents the probability that the first recall result in the first prompt information contains the information required by the target information.

[0053] Optionally, the plurality of first prompt information is input into the target natural language model. The target natural language model outputs a probability value corresponding to the first prompt information. It should be noted that the output logic of the target natural language model has been modified to be able to output the probability that the first recall result in the first prompt information contains the information required by the target information. For example, N first prompt information is input into the target natural language model, and the target natural language model service is called to obtain corresponding N probability values. It should be noted that the target natural language model can be a natural language model that can be used to output the above-mentioned probability value after training. The target natural language model can be a deep learning model.

[0054] Step S204, determining the target reply information corresponding to the target information from the plurality of first recall results according to the probability value.

[0055] Optionally, the plurality of first recall results can be accurately sorted by the obtained probability value, and then more accurate target reply information corresponding to the target information can be obtained according to the sorted first recall results. In an optional embodiment, the target reply information corresponding to the target information can also be determined from the plurality of first recall results by direct comparison.

[0056] In summary, after obtaining the plurality of first recall results based on the target information by the first natural language model, the first prompt information is constructed by the target information and the first recall results, and then the probability value corresponding to the first recall results is determined by the target natural language model, which realizes accurate sorting of the first recall results and improves the accuracy of the model reply information.

[0057] In order to improve the accuracy of the natural language model output, in the information processing method provided in the embodiments of the present disclosure, the prompt information is constructed according to the target information and the first recall results in the plurality of first recall results, and the plurality of first prompt information is obtained, including: splicing the target information and the first recall results in the plurality of first recall results to obtain a plurality of first question and answer pairs; constructing prompt information according to the first question and answer pair of the plurality of first question and answer pairs and the preset binary classification text to obtain a plurality of initial prompt information; splicing the domain to which the target information belongs and the plurality of initial prompt information to obtain the plurality of first prompt information.

[0058] Optionally, the target information and the first recall results in the plurality of first recall results are spliced, for example, the target information includes N first recall results, and the target information and the N first recall results are spliced to obtain N first question and answer pairs. For example, the form of the first question and answer pair is question: '{query}'\nanswer: '{text}', wherein {query} is the target information described above, and {text} is the first recall result described above.

[0059] After obtaining the first question and answer pair described above, the prompt information is constructed according to the first question and answer pair of the plurality of first question and answer pairs and the preset binary classification text, for example, the initial prompt information is "question: '{query}'\nanswer: '{text}'\nDoes the answer contain the required information to answer the question? Please directly output yes or no." It should be noted that the preset binary classification text can be please directly output yes or no.

[0060] In order to enable better results in the professional field of chunk accurate sorting, the above initial prompt information is added with the corresponding domain information prefix, for example, in the power field, the prefix can be "you are a power field expert, you need to combine the relevant knowledge in the power field to complete the following task.", in the financial field, the prefix can be "you are a financial field expert, you need to combine the relevant knowledge in the financial field to complete the following task."

[0061] By constructing the above prompt, more effective data can be provided for the target natural language model, thereby improving the model question and answer effect.

[0062] In order to improve the accuracy of the output probability value, in the information processing method provided by the embodiments of the present disclosure, the target natural language model processes the plurality of first prompt information to obtain the probability value corresponding to the first prompt information in the plurality of first prompt information, comprising: processing the plurality of first prompt information by the target natural language model to obtain the logit value corresponding to the first prompt information in the plurality of first prompt information; and calculating the logit value to obtain the probability value.

[0063] Optionally, the output logic of the target natural language model is modified so that it outputs the logits (i.e. the above-mentioned logit value) corresponding to each token while outputting the tokens. logits is a concept in the field of machine learning, which is the original score generated by the output layer of a neural network, used to represent the confidence of a certain category.

[0064] The target natural language model processes the plurality of first prompt information concurrently to obtain the logit value corresponding to the first prompt information in the plurality of first prompt information, that is, based on the characteristics of the task prompt, the model service logic is modified, the logits values corresponding to "yes" and "no" in the logits output by the model are calculated, and the softmax score (i.e. the above-mentioned probability value) is calculated through the logits value, as part of the natural language model output. The score represents the degree of information required to answer the question in the answer, that is, the probability value of including the required information of the target information in the first recall result. In an optional embodiment, processing the plurality of first prompt information by the target natural language model comprises extracting features from the first prompt information by the target natural language model, and then outputting the logits values corresponding to "yes" and "no" according to the extracted feature information, and then obtaining the above-mentioned probability value.

[0065] The target natural language model is a natural language model that can be used to output the above-mentioned probability value after training, therefore, through the target natural language model, the probability value of including the required information of the target information in the recall result can be accurately obtained, and the accuracy of the reply to the target information is improved.

[0066] To improve the question and answer performance of the target natural language model, in the information processing method provided in the embodiments of the present disclosure, before the plurality of first prompt information is processed by the target natural language model to obtain the probability value corresponding to the first prompt information in the plurality of first prompt information, the method further comprises: obtaining a target training sample set, wherein the target training sample set comprises a plurality of questions and a plurality of second recall results corresponding to the questions in the plurality of questions; and training an initial natural language model according to the target training sample set to obtain the target natural language model.

[0067] Optionally, the plurality of questions and the plurality of second recall results corresponding to the questions in the plurality of questions are collected according to a field to be used (for example, the field of electric power), and the target training sample set is constructed according to the plurality of questions and the recall results, and the initial natural language model is trained according to the target training sample set to obtain the target natural language model.

[0068] For example, based on the above-mentioned constructed sample set, the initial natural language model is fully fine-tuned, the learning rate is 5e-7, and a total of 3 epochs are trained to obtain the target natural language model. It should be noted that one epoch refers to that the training sample set is completely traversed once in the training process. For example, if a training sample set has 1000 samples, and 10 are used each time, then one epoch is 100 iterations.

[0069] Through the above steps, full-parameter fine-tuning of the natural language model can be realized, and the question and answer performance of the target natural language model can be improved.

[0070] To improve the effect of model training, in the information processing method provided in the embodiments of the present disclosure, obtaining the target training sample set comprises: obtaining a plurality of questions and obtaining a plurality of second recall results; splicing the questions in the plurality of questions and the plurality of second recall results corresponding to the questions to obtain a plurality of second question and answer pairs; and constructing the target training sample set according to the plurality of second question and answer pairs and the true labels of the second question and answer pairs in the plurality of second question and answer pairs, wherein the true label is used to represent whether the second recall result in the second question and answer pair includes the information required by the question.

[0071] Optionally, the plurality of questions and the plurality of second recall results corresponding to the questions in the field to be used (for example, the field of electric power) are obtained, and then the questions and the recall results are matched two by two to form the above-mentioned plurality of second question and answer pairs, and then the true labels between the questions and the recall results are obtained, the true labels are used to represent whether the second recall result in the second question and answer pair includes the information required by the question, for example, the questions and the recall results can be labeled by artificial means. Finally, the target training sample set is constructed by the second question and answer pairs and the true labels.

[0072] To improve the efficiency of obtaining the real label, in the information processing method provided in the embodiments of the present disclosure, the following steps are adopted to obtain the real label: constructing prompt information according to the plurality of second question-answer pairs and the preset binary classification text to obtain a plurality of second prompt information; processing the plurality of second prompt information through a second natural language model to obtain reply information corresponding to the plurality of second prompt information, wherein the reply information is used to represent whether the recall result in the second prompt information includes the information required by the question; and determining the real label according to the reply information.

[0073] Optionally, the real label between the question and the recall result is obtained by the following steps: first, constructing a prompt by using the second question-answer pair and the binary classification text task, for example, the second prompt information is in the form of "question: '${query}'\nanswer: '${text}'\nDoes the answer contain the required information to answer the question? Please output yes or no directly."

[0074] Then, the second prompt information is input into the second natural language model, and the plurality of second prompt information is processed in batches by the second natural language model, that is, the second natural language model processes the plurality of second prompt information to obtain reply information corresponding to the plurality of second prompt information, and finally the real label is determined through the reply information.

[0075] The natural language model is used to process the question and the recall result in batches, thereby improving the accuracy of obtaining the real label.

[0076] To improve the accuracy of the recall result, in the information processing method provided in the embodiments of the present disclosure, obtaining the plurality of second recall results includes: receiving a plurality of questions through a target interface, and inputting the plurality of questions into a first natural language model; processing the plurality of questions through the first natural language model to obtain a plurality of reply information corresponding to the plurality of questions; and determining the plurality of second recall results from the plurality of reply information according to the order of the reply information in the plurality of reply information.

[0077] Optionally, the target interface can be used to receive a plurality of questions input by a user, and the target interface can input the plurality of questions into the first natural language model by calling an external interface of the first natural language model. The first natural language model performs search and semantic understanding based on the plurality of questions to obtain a plurality of reply information corresponding to the questions, and determines the plurality of second recall results from the plurality of reply information according to the order of the reply information, for example, determines the second recall information according to the top 10 recall text of the first natural language model.

[0078] In the information processing method provided in the embodiments of the present disclosure, determining the target reply information corresponding to the target information from the plurality of first recall results according to the probability value comprises: performing descending arrangement processing on the plurality of first recall results according to the probability value to obtain a sorting result of the first recall results; and determining the first recall result whose arrangement order meets a preset requirement as the target reply information according to the sorting result.

[0079] Optionally, after obtaining the probability value, the following steps are adopted to obtain the target reply information corresponding to the target information: performing descending arrangement processing on the plurality of first recall results according to the probability value output by the target natural language model to obtain a precise sorting result (i.e., the above-mentioned sorting result) of the first recall results, and determining the first recall result whose arrangement order meets a preset requirement as the target reply information according to the sorting result. For example, the first recall result with a high ranking can be taken as the target reply result of the target information.

[0080] In an optional embodiment, the processing of information can be implemented through the flowchart shown in FIG. 4, mainly including the following steps: 1) inputting N recalls corresponding to a query question (i.e., a query question) and a query (i.e., a query);

[0081] 2) concatenating the question and the corresponding recall to form a question and answer pair prompt (i.e., a prompt word), and adding a domain knowledge prefix in front. A total of N question and answer pairs are formed, for example: you are an expert in the power field, you need to combine the relevant knowledge in the power field to complete the following task. Question: '{query}'\nAnswer: '{text}'\nDoes the answer contain the required information to answer the question? Please output yes or no directly.

[0082] 3) concurrently calling the natural language model to perform precise sorting processing on the N question and answer pairs;

[0083] Based on the fine-tuned model, the output logic of the large model is modified so that the logits corresponding to each token are output at the same time as the tokens; based on the characteristics of the task prompt, the model service logic is modified to calculate the logit values corresponding to "yes" and "no" in the logits output by the model, and the softmax score is calculated therefrom as part of the model service output. The score represents the degree to which the answer contains the required information to answer the question. The N question and answer pairs constructed for the query are concurrently called to the large model service to obtain the corresponding N scores. According to the N scores, descending arrangement is performed to obtain the precise sorting result of the recalls.

[0084] By combining professional knowledge and task requirements, the prompt is designed and constructed, the large model is fine-tuned based on professional field knowledge, the model and service output logic are modified, and the large model service is called concurrently, so that the chunk accurate sorting reasoning speed and the model question and answer reply accuracy are improved.

[0085] In the information processing method provided in the embodiments of the present disclosure, a plurality of first recall results for target information are obtained, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information; prompt information is constructed according to the target information and a first recall result in the plurality of first recall results, to obtain a plurality of first prompt information; a target natural language model is used to concurrently process the plurality of first prompt information, to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, wherein the probability value represents a probability that the first recall result in the first prompt information contains information required by the target information; and target reply information corresponding to the target information is determined from the plurality of first recall results according to the probability value, thereby solving the technical problem in the related art that the content of recall is sorted by relying on artificial supervision signals when a large model is used to ask questions and answer knowledge in a document, resulting in relatively low accuracy of model reply information. In the present solution, after obtaining a plurality of first recall results based on the target information by using the first natural language model, the first prompt information is constructed according to the target information and the first recall result, the first prompt information is used to guide the target natural language model to output the probability that the first recall result contains information required by the target information, and then the probability value is used to accurately sort the first recall result, thereby achieving the effect of improving the accuracy of model reply information.

[0086] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present disclosure is not limited to the action sequence described, because according to the present disclosure, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the present disclosure.

[0087] Those skilled in the art can clearly understand the method according to the above-mentioned embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the methods of various embodiments of the present disclosure.

[0088] Embodiment 2

[0089] According to the embodiments of the present disclosure, an information processing method is also provided, as shown in FIG. 5, the information processing method comprises:

[0090] In step S501, a plurality of first recall results for target information sent by a client are acquired, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information;

[0091] In step S502, prompt information is constructed in a cloud server according to the target information and the first recall results in the plurality of first recall results, to obtain a plurality of first prompt information; the plurality of first prompt information is processed concurrently by a target natural language model, to obtain a probability value corresponding to the first prompt information in the plurality of first prompt information, wherein the probability value is used to represent a probability that the first recall result in the first prompt information contains information required by the target information; and the target reply information corresponding to the target information is determined from the plurality of first recall results according to the probability value;

[0092] In step S503, the target reply information is returned to the client.

[0093] It should be noted that the specific method of processing the target information in the cloud server is the same as that in the implementation, and will not be described here.

[0094] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a combination of a series of actions, but those skilled in the art should know that the present disclosure is not limited to the order of the actions described, because according to the present disclosure, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the present disclosure.

[0095] From the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the methods of various embodiments of the present disclosure.

[0096] Embodiment 3

[0097] According to the embodiments of the present disclosure, an information processing apparatus for implementing the information processing method is also provided, as shown in FIG. 6, the apparatus comprises: a first obtaining unit 601, a first constructing unit 602, a processing unit 603 and a sorting unit 604.

[0098] The first obtaining unit 601 is configured to obtain a plurality of first recall results for target information, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information;

[0099] The first constructing unit 602 is configured to construct prompt information according to the target information and a first recall result in the plurality of first recall results, to obtain a plurality of first prompt information;

[0100] The processing unit 603 is configured to process the plurality of first prompt information by a target natural language model, to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, wherein the probability value is used to represent a probability that the first recall result in the first prompt information contains information required by the target information;

[0101] The sorting unit 604 is configured to determine target reply information corresponding to the target information from the plurality of first recall results according to the probability value.

[0102] In the information processing apparatus provided by the embodiments of the present disclosure, the first obtaining unit 601 obtains a plurality of first recall results for target information, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information; the first constructing unit 602 constructs prompt information according to the target information and a first recall result in the plurality of first recall results, to obtain a plurality of first prompt information; the processing unit 603 processes the plurality of first prompt information by a target natural language model, to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, wherein the probability value is used to represent a probability that the first recall result in the first prompt information contains information required by the target information; and the sorting unit 604 determines target reply information corresponding to the target information from the plurality of first recall results according to the probability value, thereby solving the technical problem in the related art that the content of recall is sorted by relying on artificial supervision signals when a large model is used to ask questions and answer knowledge in a document, resulting in a relatively low accuracy of model reply information. In the present solution, after obtaining a plurality of first recall results based on the target information by the first natural language model, the first prompt information is constructed according to the target information and the first recall result, the first prompt information is used to guide the target natural language model to output the probability that the first recall result contains information required by the target information, and then the probability value is used to realize accurate sorting of the first recall result, thereby achieving the effect of improving the accuracy of model reply information.

[0103] Optionally, in the information processing apparatus provided by the embodiment of the present disclosure, the constructing unit comprises: a splicing module configured to splice the target information and the first recall result in the plurality of first recall results to obtain a plurality of first question-answer pairs; a constructing module configured to construct prompt information according to the plurality of first question-answer pairs and the preset binary classification text to obtain a plurality of initial prompt information; and a first determining module configured to splice the domain to which the target information belongs and the plurality of initial prompt information to obtain a plurality of first prompt information.

[0104] Optionally, in the information processing apparatus provided by the embodiment of the present disclosure, the processing unit comprises: a processing module configured to process the plurality of first prompt information concurrently by using the target natural language model to obtain a logit value corresponding to the first prompt information in the plurality of first prompt information; and a calculating module configured to calculate according to the logit value to obtain a probability value.

[0105] Optionally, in the information processing apparatus provided by the embodiment of the present disclosure, the apparatus further comprises: a second obtaining unit configured to obtain a target training sample set before processing the plurality of first prompt information by using the target natural language model to obtain the probability value corresponding to the first prompt information in the plurality of first prompt information, wherein the target training sample set comprises a plurality of questions and a plurality of second recall results corresponding to the question in the plurality of questions; and a training unit configured to train the initial natural language model according to the target training sample set to obtain the target natural language model.

[0106] Optionally, in the information processing apparatus provided by the embodiment of the present disclosure, the second obtaining unit comprises: an obtaining module configured to obtain the plurality of questions and the plurality of second recall results; a splicing module configured to splice the question in the plurality of questions and the plurality of second recall results corresponding to the question to obtain a plurality of second question-answer pairs; and a constructing module configured to construct the target training sample set according to the plurality of second question-answer pairs and a true label of the second question-answer pair in the plurality of second question-answer pairs, wherein the true label is used to represent whether the second recall result in the second question-answer pair includes the information required by the question.

[0107] Optionally, in the information processing apparatus provided by the embodiment of the present disclosure, the true label is obtained by using the following apparatus: a second constructing unit configured to construct prompt information according to the plurality of second question-answer pairs and the preset binary classification text to obtain a plurality of second prompt information; a processing unit configured to process the plurality of second prompt information by using a second natural language model to obtain reply information corresponding to the plurality of second prompt information, wherein the reply information is used to represent whether the recall result in the second prompt information includes the information required by the question; and a determining unit configured to determine the true label according to the reply information.

[0108] Optionally, in the information processing apparatus provided in the embodiments of the present disclosure, the obtaining module comprises: a receiving sub-module, configured to receive the plurality of questions through the target interface, and input the plurality of questions into the first natural language model; a processing sub-module, configured to process the plurality of questions through the first natural language model to obtain a plurality of reply information corresponding to the plurality of questions; and a determining sub-module, configured to determine a plurality of second recall results from the plurality of reply information according to the order of the reply information in the plurality of reply information.

[0109] Optionally, in the information processing apparatus provided in the embodiments of the present disclosure, the sorting unit comprises: a sorting module, configured to perform descending arrangement processing on the plurality of first recall results according to the probability values to obtain a sorting result of the first recall results; and a second determining module, configured to determine the first recall result whose arrangement order meets the preset requirement as the target reply information according to the sorting result.

[0110] It should be noted that the first obtaining unit 601, the first constructing unit 602, the processing unit 603 and the sorting unit 604 correspond to steps S201 to S204 in Embodiment 1, and the four units have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules can run in the computer terminal 10 provided in Embodiment 1 as part of the apparatus.

[0111] It should be noted that the preferred embodiments involved in the above embodiments of the present disclosure have the same scheme, application scenario and implementation process as those provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0112] Embodiment 4

[0113] The embodiments of the present disclosure can provide an electronic device, which can be any one of the electronic devices in the electronic device terminal group. Optionally, in the present embodiment, the electronic device can also be replaced by a terminal device such as a mobile terminal.

[0114] Optionally, in the present embodiment, the electronic device can be located in at least one network device of a plurality of network devices of a computer network.

[0115] ​​​​​​​In the embodiment, the computer terminal can execute program codes of the following steps in the information processing method: obtaining a plurality of first recall results for target information, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information; constructing prompt information according to the target information and a first recall result in the plurality of first recall results to obtain a plurality of first prompt information; processing the plurality of first prompt information by a target natural language model to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, wherein the probability value represents a probability that the first recall result in the first prompt information contains information required by the target information; and determining target reply information corresponding to the target information from the plurality of first recall results according to the probability value.

[0116] The computer terminal can execute program codes of the following steps in the information processing method: constructing prompt information according to the target information and a first recall result in the plurality of first recall results to obtain a plurality of first prompt information, including: splicing the target information and the first recall result in the plurality of first recall results to obtain a plurality of first question and answer pairs; constructing prompt information according to a pre-set binary classification text and a first question and answer pair in the plurality of first question and answer pairs to obtain a plurality of initial prompt information; and splicing a domain to which the target information belongs and the plurality of initial prompt information to obtain the plurality of first prompt information.

[0117] The computer terminal can execute program codes of the following steps in the information processing method: processing the plurality of first prompt information by the target natural language model to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, including: processing the plurality of first prompt information by the target natural language model in parallel to obtain a logit value corresponding to the first prompt information in the plurality of first prompt information; and calculating the probability value according to the logit value.

[0118] The computer terminal can execute program codes of the following steps in the information processing method: before processing the plurality of first prompt information by the target natural language model to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, the method further includes: obtaining a target training sample set, wherein the target training sample set includes a plurality of questions and a plurality of second recall results corresponding to a question in the plurality of questions; and training an initial natural language model according to the target training sample set to obtain the target natural language model.

[0119] The computer terminal can execute program codes of the following steps in the information processing method: obtaining the target training sample set comprises: obtaining a plurality of questions and obtaining a plurality of second recall results; splicing the questions in the plurality of questions and the plurality of second recall results corresponding to the questions to obtain a plurality of second question and answer pairs; and constructing the target training sample set according to the plurality of second question and answer pairs and a true label of a second question and answer pair in the plurality of second question and answer pairs, wherein the true label is used to represent whether the second recall result in the second question and answer pair includes information required by the question.

[0120] The computer terminal can execute program codes of the following steps in the information processing method: obtaining the true label comprises: constructing prompt information according to the plurality of second question and answer pairs and a preset binary classification text to obtain a plurality of second prompt information; processing the plurality of second prompt information through a second natural language model to obtain reply information corresponding to the plurality of second prompt information, wherein the reply information is used to represent whether the recall result in the second prompt information includes information required by the question; and determining the true label according to the reply information.

[0121] The computer terminal can execute program codes of the following steps in the information processing method: obtaining the plurality of second recall results comprises: receiving a plurality of questions through a target interface, and inputting the plurality of questions into a first natural language model; processing the plurality of questions through the first natural language model to obtain a plurality of reply information corresponding to the plurality of questions; and determining the plurality of second recall results from the plurality of reply information according to an order of the reply information in the plurality of reply information.

[0122] The computer terminal can execute program codes of the following steps in the information processing method: determining the target reply information corresponding to the target information from the plurality of first recall results according to the probability value comprises: performing descending arrangement processing on the plurality of first recall results according to the probability value to obtain a sorting result of the first recall results; and determining the first recall result whose arrangement order meets a preset requirement as the target reply information according to the sorting result.

[0123] Optionally, FIG. 7 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 7, the electronic device 20 can include one or more (only one is shown in FIG. 7) processors 702, a memory 704. The electronic device 20 can further include a storage controller for controlling and managing the memory 704, and can further include a peripheral interface for connecting a radio frequency module, an audio module, and a display screen, etc.

[0124] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the information processing method and device in the embodiments of the present disclosure. The processor executes various functions and data processing by running the software programs and modules stored in the memory, that is, implements the information processing method described above. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the electronic device 20 through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0125] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: obtaining a plurality of first recall results for target information, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information; constructing prompt information according to the target information and the first recall result in the plurality of first recall results to obtain a plurality of first prompt information; processing the plurality of first prompt information through a target natural language model to obtain a probability value corresponding to the first prompt information in the plurality of first prompt information, wherein the probability value is used to represent the probability that the first recall result in the first prompt information contains information required by the target information; and determining target reply information corresponding to the target information from the plurality of first recall results according to the probability value.

[0126] Optionally, the processor can further execute program codes of the following steps: constructing prompt information according to the target information and the first recall result in the plurality of first recall results to obtain a plurality of first prompt information includes: splicing the target information and the first recall result in the plurality of first recall results to obtain a plurality of first question and answer pairs; constructing prompt information according to a pre-set binary classification text and the first question and answer pair of the plurality of first question and answer pairs to obtain a plurality of initial prompt information; and splicing the domain to which the target information belongs and the plurality of initial prompt information to obtain the plurality of first prompt information.

[0127] Optionally, the processor can further execute program codes of the following steps: processing the plurality of first prompt information through the target natural language model to obtain a probability value corresponding to the first prompt information in the plurality of first prompt information includes: processing the plurality of first prompt information through the target natural language model concurrently to obtain a logit value corresponding to the first prompt information in the plurality of first prompt information; and calculating the probability value according to the logit value.

[0128] Optionally, the processor can further execute program codes of the following steps: before obtaining the probability value corresponding to the first prompt information in the plurality of first prompt information by processing the plurality of first prompt information through the target natural language model, the method further comprises: obtaining a target training sample set, wherein the target training sample set comprises a plurality of questions and a plurality of second retrieval results corresponding to the question in the plurality of questions; and training the initial natural language model according to the target training sample set to obtain the target natural language model.

[0129] Optionally, the processor can further execute program codes of the following steps: obtaining the target training sample set comprises: obtaining a plurality of questions and obtaining a plurality of second retrieval results; splicing the question in the plurality of questions and the plurality of second retrieval results corresponding to the question to obtain a plurality of second question and answer pairs; and constructing the target training sample set according to the plurality of second question and answer pairs and a true label of the second question and answer pair in the plurality of second question and answer pairs, wherein the true label is used to represent whether the second retrieval result in the second question and answer pair includes information required by the question.

[0130] Optionally, the processor can further execute program codes of the following steps: the true label is obtained by the following steps: constructing prompt information according to the plurality of second question and answer pairs and a preset binary classification text to obtain a plurality of second prompt information; processing the plurality of second prompt information through a second natural language model to obtain reply information corresponding to the plurality of second prompt information, wherein the reply information is used to represent whether the retrieval result in the second prompt information includes information required by the question; and determining the true label according to the reply information.

[0131] Optionally, the processor can further execute program codes of the following steps: obtaining the plurality of second retrieval results comprises: receiving a plurality of questions through a target interface, and inputting the plurality of questions into the first natural language model; processing the plurality of questions through the first natural language model to obtain a plurality of reply information corresponding to the plurality of questions; and determining the plurality of second retrieval results from the plurality of reply information according to the order of the reply information in the plurality of reply information.

[0132] Optionally, the processor can further execute program codes of the following steps: determining the target reply information corresponding to the target information from the plurality of first retrieval results according to the probability value comprises: performing descending arrangement processing on the plurality of first retrieval results according to the probability value to obtain a sorting result of the first retrieval result; and determining the first retrieval result whose arrangement order meets a preset requirement as the target reply information according to the sorting result.

[0133] By adopting the embodiment of the disclosure, a scheme for information processing is provided. The probability first prompt information required for guiding the target natural language model to output the information contained in the first recall result is constructed by the target information and the first recall result, and then the first recall result is accurately sorted by the probability value corresponding to the target natural language model first prompt information, so as to improve the accuracy of the model reply information, and solve the technical problem that the related art depends on artificial supervision signals to sort the content of the recall when the large model answers the knowledge in the document, resulting in low accuracy of the model reply information.

[0134] Those skilled in the art can understand that the structure shown in FIG. 7 is only schematic, and the electronic device 20 can also be a terminal device such as a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a mobile Internet device (MID), a PAD, etc. FIG. 7 does not limit the structure of the above-mentioned electronic device. For example, the electronic device 20 can further include more or less components (such as a network interface, a display device, etc.) than those shown in FIG. 7, or have a different configuration from that shown in FIG. 7.

[0135] Those skilled in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing the relevant hardware of the terminal device by a program, and the program can be stored in a computer readable storage medium, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0136] Embodiment 5

[0137] The embodiment of the disclosure also provides a computer program product. Optionally, in the embodiment, the above-mentioned computer program product can be used to save the program code executed by the information processing method provided in embodiment one.

[0138] Optionally, in the embodiment, the above-mentioned computer program product can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0139] The serial numbers of the above-mentioned embodiments of the disclosure are only for description, not representing the advantages and disadvantages of the embodiments.

[0140] In the above-mentioned embodiments of the disclosure, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0141] In several embodiments provided by the present disclosure, it should be understood that the disclosed technology can be implemented in other manners. For example, the described embodiments of the apparatus are merely schematic, and the division of units is merely logical function division, and there can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, and electrical or other forms.

[0142] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0143] In addition, each functional unit in the various embodiments of the present disclosure can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0144] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present disclosure, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, etc.

[0145] The above description is merely the preferred embodiments of the present disclosure, and it should be pointed out that, for those skilled in the art, without departing from the principles of the present disclosure, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present disclosure.

Claims

1. An information processing method, wherein, The method comprises the following steps: obtaining a plurality of first recall results for target information, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information; constructing prompt information according to the target information and a first recall result in the plurality of first recall results to obtain a plurality of first prompt information; processing the plurality of first prompt information by a target natural language model to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, wherein the probability value represents a probability that a first recall result in the first prompt information contains information required by the target information; determining target reply information corresponding to the target information from the plurality of first recall results according to the probability value.

2. The method of claim 1, wherein, The method comprises the following steps: splicing the target information and a first recall result in the plurality of first recall results to obtain a plurality of first question and answer pairs; constructing prompt information according to a preset binary classification text and a first question and answer pair in the plurality of first question and answer pairs to obtain a plurality of initial prompt information; splicing a domain to which the target information belongs and the plurality of initial prompt information to obtain the plurality of first prompt information.

3. The method of claim 1 or 2, wherein, The method comprises the following steps: concurrently processing the plurality of first prompt information by the target natural language model to obtain a logit value corresponding to a first prompt information in the plurality of first prompt information; calculating the probability value according to the logit value.

4. The method of any one of claims 1 to 3, wherein, Before the processing of the plurality of first prompt information by the target natural language model to obtain the probability value corresponding to the first prompt information in the plurality of first prompt information, the method further comprises the following steps: obtaining a target training sample set, wherein the target training sample set comprises a plurality of questions and a plurality of second recall results corresponding to a question in the plurality of questions; training an initial natural language model according to the target training sample set to obtain the target natural language model.

5. The method of claim 4, wherein, The method comprises the following steps: obtaining the plurality of questions and the plurality of second recall results; splicing a question in the plurality of questions and a plurality of second recall results corresponding to the question to obtain a plurality of second question and answer pairs; constructing a target training sample set according to the plurality of second question and answer pairs and a true label of a second question and answer pair in the plurality of second question and answer pairs, wherein the true label represents whether a second recall result in the second question and answer pair contains information required by the question.

6. The method of claim 5, wherein, The true label is obtained by the following steps: constructing prompt information according to the plurality of second question and answer pairs and a preset binary classification text to obtain a plurality of second prompt information; The second prompt information is processed by a second natural language model to obtain reply information corresponding to the second prompt information, wherein the reply information is used to represent whether the recall result in the second prompt information includes information required by the question; The real label is determined according to the reply information.

7. The method of claim 5 or 6, wherein, The plurality of second recall results are obtained by: The plurality of questions are received through a target interface, and the plurality of questions are input into a first natural language model; The plurality of questions are processed by the first natural language model to obtain a plurality of reply information corresponding to the plurality of questions; The plurality of second recall results are determined from the plurality of reply information according to the order of the reply information in the plurality of reply information.

8. The method of any one of claims 1 to 7, wherein, The target reply information corresponding to the target information is determined from the plurality of first recall results according to the probability value, including: The plurality of first recall results are arranged in descending order according to the probability value to obtain a sorting result of the first recall result; The first recall result whose arrangement order meets a preset requirement is determined as the target reply information according to the sorting result.

9. An information processing method, wherein, Including: A plurality of first recall results for target information sent by a client are obtained, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information; Prompt information is constructed in a cloud server according to the target information and a first recall result in the plurality of first recall results to obtain a plurality of first prompt information; a target natural language model is used to process the plurality of first prompt information to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, wherein the probability value is used to represent a probability that the first recall result in the first prompt information contains information required by the target information; a target reply information corresponding to the target information is determined from the plurality of first recall results according to the probability value; The target reply information is returned to the client.

10. An information processing apparatus, comprising: Including: A first acquisition unit is configured to obtain a plurality of first recall results for target information, wherein the plurality of first recall results are reply information output by a first natural language model based on the target information; A first construction unit is configured to construct prompt information according to the target information and a first recall result in the plurality of first recall results to obtain a plurality of first prompt information; A processing unit is configured to process the plurality of first prompt information by a target natural language model to obtain a probability value corresponding to a first prompt information in the plurality of first prompt information, wherein the probability value is used to represent a probability that the first recall result in the first prompt information contains information required by the target information; A sorting unit is configured to determine a target reply information corresponding to the target information from the plurality of first recall results according to the probability value.

11. An electronic device, comprising: Including: A memory stores an executable program; A processor is configured to run the program, wherein the program performs the information processing method in any one of claims 1 to 9 when running.

12. A computer program product, wherein, The computer program product comprises a stored computer program which, when executed by a processor, implements the information processing method according to any one of claims 1 to 9.

Images