Information processing method and device based on large language model, product and equipment

By judging the attribute parameters of the input information and performing knowledge base retrieval in the large language model, the problem of inaccurate information output in the large language model is solved, and more accurate information responses and reduced misleading are achieved.

CN118897883APending Publication Date: 2024-11-05CHINA TELECOM CORP LTD TECHNOLOGY INNOVATION CENTER +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411081063.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

Large language models may generate erroneous or unnecessary retrieval information during information processing, leading to inaccurate output and risks.

Method used

If the input information's specific attribute parameters meet the preset knowledge base retrieval conditions, the matching target information is retrieved from the knowledge base and processed in conjunction with a large language model to obtain the input response information.

Benefits of technology

It improved the accuracy of information responses, reduced unnecessary knowledge base retrieval operations, and avoided the introduction of misleading information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118897883A_ABST
    Figure CN118897883A_ABST
Patent Text Reader

Abstract

The invention provides an information processing method and device based on a large language model, a computer program product and electronic equipment, and relates to the technical field of information processing. The information processing method based on the large language model comprises the steps that first input information is acquired, and whether specific attribute parameters corresponding to the first input information meet preset knowledge base retrieval conditions or not is judged; if the specific attribute parameter corresponding to the first input information meets a preset knowledge base retrieval condition, performing retrieval from a knowledge base based on the first input information to obtain target retrieval information matched with the first input information; and determining second input information based on the target retrieval information, and processing the second input information through the large language model to obtain input reply information. According to the method, the output accuracy of the large language model can be ensured, meanwhile, unnecessary knowledge base retrieval operation is reduced, and introduction of information possibly generating misleading is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of information processing, and in particular to an information processing method based on a large language model, an information processing device based on a large language model, a computer program product, and an electronic device. Background Art

[0002] The continuous development of deep learning has brought artificial intelligence into the era of large language models. The so-called large models refer to a type of language models based on neural networks, which are trained using huge data sets and have billions or more parameters. Compared with models with smaller parameter scales, large language models have better migration capabilities, generalization capabilities, and stability. Large language models perform well in various general natural language tasks, but they also have some limitations, such as the possibility of generating content containing incorrect information.

[0003] In related technologies, in order to improve the output accuracy of a large language model, an input question and retrieved knowledge base information related to the input information are input into the large language model to obtain input answer information. However, the retrieved information is not always helpful for the output of the model, and erroneous or unnecessary retrieval information may introduce risks.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a large language model information processing method, a large language model information processing device, a computer program product and an electronic device, so as to at least to some extent solve the problem that erroneous or unnecessary retrieval information may be introduced in the related art.

[0006] According to a first aspect of the present disclosure, there is provided an information processing method based on a large language model, the method comprising: obtaining first input information, and determining whether a specific attribute parameter corresponding to the first input information satisfies a preset knowledge base retrieval condition; if the specific attribute parameter corresponding to the first input information satisfies the preset knowledge base retrieval condition, based on the first input information, searching from the knowledge base to obtain target retrieval information matching the first input information; based on the target retrieval information, determining second input information, and processing the second input information through a large language model to obtain input reply information.

[0007] According to a second aspect of the present disclosure, there is provided an information processing device based on a large language model, the device comprising: an attribute parameter judgment model, used to obtain first input information, and judge whether a specific attribute parameter corresponding to the first input information satisfies a preset knowledge base retrieval condition; a retrieval information determination module, used to search from a knowledge base based on the first input information if the specific attribute parameter corresponding to the first input information satisfies the preset knowledge base retrieval condition, and obtain target retrieval information matching the first input information; an input reply determination module, used to determine second input information based on the target retrieval information, and process the second input information through a large language model to obtain input reply information.

[0008] According to a third aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the information processing method based on a large language model according to the first aspect and possible implementation methods thereof are implemented.

[0009] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the large language model-based information processing method and its possible implementation method of the above-mentioned first aspect by executing the executable instructions.

[0010] The technical solution disclosed in this disclosure has the following beneficial effects:

[0011] In the above-mentioned information processing process based on the large language model, the first input information is obtained, and it is determined whether the specific attribute parameters corresponding to the first input information meet the preset knowledge base retrieval conditions; if the specific attribute parameters corresponding to the first input information meet the preset knowledge base retrieval conditions, based on the first input information, a search is performed from the knowledge base to obtain the target retrieval information matching the first input information; based on the target retrieval information, the second input information is determined, and the second input information is processed through the large language model to obtain input reply information. On the one hand, the present disclosure can improve the accuracy of information replies to a certain extent by combining the target retrieval information matching the first input information and using a large language model to obtain input reply information. On the other hand, the present disclosure obtains the target retrieval information matching the first input information when the specific attribute parameters corresponding to the first input information meet the preset knowledge base retrieval conditions, which can reduce unnecessary knowledge base retrieval operations while ensuring the accuracy of the large language model output and avoid introducing information that may be misleading. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A flowchart showing an information processing method based on a large language model in this exemplary embodiment is shown;

[0013] Figure 2 A flowchart of determining whether a specific attribute parameter corresponding to first input information satisfies a preset knowledge base search condition in this exemplary embodiment is shown;

[0014] Figure 3 A flowchart of determining target retrieval information in this exemplary embodiment is shown;

[0015] Figure 4 A flowchart of determining second input information in this exemplary embodiment is shown;

[0016] Figure 5 A flowchart of adaptive retrieval knowledge enhancement of a large language model in this exemplary embodiment is shown;

[0017] Figure 6 A structural block diagram of an information processing device based on a large language model in this exemplary embodiment is shown;

[0018] Figure 7 An electronic device for implementing the above-mentioned information processing method based on a large language model in this exemplary embodiment is shown. DETAILED DESCRIPTION

[0019] Exemplary embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings.

[0020] The accompanying drawings are schematic diagrams of the present disclosure and are not necessarily drawn to scale. Some of the block diagrams shown in the accompanying drawings may be functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in hardware modules or integrated circuits, or in networks, processors or microcontrollers. The embodiments can be implemented in various forms and should not be construed as being limited to the examples set forth herein. The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the description below, many specific details are provided to provide a full description of the embodiments of the present disclosure. However, those skilled in the art should appreciate that one or more specific details may be omitted when implementing the technical solution of the present disclosure, or one or more specific details may be replaced by other methods, components, devices, steps, etc.

[0021] In the related art, the retrieved knowledge base information related to the input information is not always helpful for the output of the model, and erroneous or unnecessary retrieval information may introduce risks.

[0022] In view of the above problems, the exemplary embodiments of the present disclosure provide an information processing method based on a large language model, an information processing device based on a large language model, a computer program product and an electronic device, which can be applied to a large language model that needs to be retrieved to enhance knowledge, and have a relatively broad application prospect. Exemplarily, it can be widely used in news generation scenarios, telecommunications customer service question and answer scenarios, legal consultation scenarios, etc., and the present disclosure does not specifically limit this.

[0023] In an optional embodiment, reference Figure 1 The figure provides an information processing method based on a large language model, which specifically includes the following steps S110 to S130:

[0024] Step S110, obtaining first input information, and determining whether a specific attribute parameter corresponding to the first input information satisfies a preset knowledge base search condition;

[0025] Step S120, if the specific attribute parameter corresponding to the first input information meets the preset knowledge base search condition, based on the first input information, search from the knowledge base to obtain target search information matching the first input information;

[0026] Step S130, based on the target search information, second input information is determined, and the second input information is processed by the large language model to obtain input reply information.

[0027] Figure 1 In the method shown, by combining the target retrieval information that matches the first input information and using a large language model to obtain input reply information, the accuracy of the information reply can be improved to a certain extent; when the specific attribute parameters corresponding to the first input information meet the preset knowledge base retrieval conditions, the target retrieval information that matches the first input information is obtained, which can reduce unnecessary knowledge base retrieval operations while ensuring the accuracy of the large language model output and avoid introducing information that may be misleading.

[0028] Below Figure 1 Provide detailed instructions for each step.

[0029] In step S110, first input information is obtained, and it is determined whether the specific attribute parameter corresponding to the first input information satisfies a preset knowledge base search condition.

[0030] The first input information refers to the initial information input by the user for which the user wants to seek a response result, and may be the original unprocessed input information. Optionally, the first input information may be information such as a question or a chat statement input by the user, and may include text information such as words and characters, or voice information, which is not specifically limited in the present disclosure.

[0031] After obtaining the first input information, the specific attribute parameter corresponding to the first input information may be determined, so as to judge whether the specific attribute parameter corresponding to the first input information satisfies the preset knowledge base search condition.

[0032] Optionally, the specific attribute parameter may include but is not limited to the popularity of the information. The popularity of the information may be used to characterize the popularity of the information or the popularity of the information. It should be noted that in actual application, the specific attribute parameter may also be other attribute parameters that are similar to or different from the popularity of the information, such as the field to which the information belongs, etc., and this disclosure does not specifically limit this.

[0033] The preset knowledge base search condition may be a pre-set condition that the first input information needs to satisfy for knowledge base search.

[0034] Optionally, the preset knowledge base search condition may be that the information popularity corresponding to the first input information is lower than the preset popularity, that is, the first input information is non-hot information. Exemplarily, the first input information may be obtained, and the information popularity corresponding to the first input information may be determined; after determining the information popularity corresponding to the first input information, it is determined whether the information popularity corresponding to the first input information is lower than the preset popularity.

[0035] Optionally, the preset knowledge base search condition may also be that the field to which the information corresponding to the first input information belongs belongs to a non-hot field. A list of hot fields may be preset, and fields to which information that is not on the list of hot fields belongs may be considered to be non-hot fields. Exemplarily, after determining the field to which the information corresponding to the first input information belongs, it is determined whether the field to which the information corresponding to the first input information belongs is not on the list of hot fields.

[0036] Exemplarily, when the specific attribute parameter includes information popularity, the information popularity corresponding to the first input information can be evaluated by one or more statistical parameters corresponding to the first input information. Specifically, the lower the statistical parameter value, the lower the information popularity; the higher the statistical parameter value, the higher the information popularity.

[0037] In an optional embodiment, when the specific attribute parameters include information popularity, the above-mentioned determination of the specific attribute parameters corresponding to the first input information can be achieved through the following steps: obtaining one or more statistical parameters corresponding to the first input information; determining the information popularity corresponding to the first input information based on the one or more statistical parameters corresponding to the first input information and the weights corresponding to each statistical parameter.

[0038] Exemplarily, the statistical parameters may include, but are not limited to: the number of views of the encyclopedia page associated with the first input information, the frequency of appearance of the knowledge associated with the first input information in the knowledge base, the number of searches for the first input information on search engines, the number of mentions and engagement of the first input information on social media platforms, etc.

[0039] Optionally, after obtaining one or more statistical parameters corresponding to the first input information, each statistical parameter may be normalized; based on each normalized statistical parameter and a weight corresponding to each statistical parameter, the popularity of the information corresponding to the first input information is determined.

[0040] Exemplarily, each statistical parameter may be normalized based on a z-score to scale each statistical parameter to a normal distribution interval with a mean of 0 and a standard deviation of 1.

[0041] Exemplarily, a weight can be assigned to each statistical parameter by using a trained machine learning model to obtain the weight corresponding to each statistical parameter. The weight value can reflect the importance of the corresponding statistical parameter in judging the popularity of information.

[0042] Exemplarily, each normalized statistical parameter and the weight corresponding to each statistical parameter may be multiplied and summed to obtain a popularity score corresponding to the first input information, and the popularity score is used as the popularity of the information.

[0043] Optionally, the popularity of the information may be reflected by a popularity score. The popularity of the information may be positively correlated with the popularity score. The higher the popularity score, the higher the popularity of the information; the lower the popularity score, the lower the popularity of the information.

[0044] After determining the popularity of the information corresponding to the first input information, the popularity score corresponding to the first input information can be compared with the preset popularity, and according to the comparison result, it is determined whether the popularity of the information corresponding to the first input information meets the preset knowledge base search condition. The preset popularity can be set according to business needs, and this disclosure does not specifically limit this.

[0045] like Figure 2 As shown, a flowchart is provided for determining whether a specific attribute parameter corresponding to the first input information satisfies a preset knowledge base search condition, which may specifically include the following steps:

[0046] Step S201, obtaining first input information and corresponding statistical parameters;

[0047] Step S202, normalizing each statistical parameter based on the z-score, multiplying each normalized statistical parameter and the weight corresponding to each statistical parameter, and summing them to obtain a popularity score corresponding to the first input information, and using the popularity score as the popularity of the information;

[0048] Step S203, determining whether the information popularity corresponding to the first input information is lower than a preset popularity; if so, executing step S204; if not, executing step S205;

[0049] Step S204, satisfying the preset knowledge base search condition;

[0050] Step S205: the preset knowledge base search condition is not met.

[0051] Since popular information usually appears multiple times in the model training data of the large language model, the large language model performs well for popular information. By judging whether the popularity of the information corresponding to the first input information is non-popular, it is convenient to screen out popular information, thereby reducing unnecessary retrieval operations and avoiding misleading and impact caused by the introduction of new information.

[0052] In step S120, if the specific attribute parameter corresponding to the first input information meets the preset knowledge base search condition, a search is performed from the knowledge base based on the first input information to obtain target search information matching the first input information.

[0053] The knowledge base refers to a collection that stores a large amount of knowledge information for retrieval, and can be a knowledge unit collection composed of multiple knowledge units.

[0054] The target retrieval information refers to retrieval information retrieved from the knowledge base that is similar or related to the first input information.

[0055] In an optional implementation, the above-mentioned searching from the knowledge base based on the first input information to obtain the target search information matching the first input information can be performed as follows: Figure 3 The steps shown are to achieve this:

[0056] Step S310, determining a first relevance score of each knowledge unit in the knowledge base relative to the first input information based on a first relevance evaluation algorithm;

[0057] Step S320, determining a second relevance score of each knowledge unit in the knowledge base relative to the first input information based on a second relevance evaluation algorithm;

[0058] Step S330: determining target retrieval information matching the first input information from the knowledge base based on the first relevance score and the second relevance score.

[0059] Figure 3 In the steps shown, by combining two different relevance evaluation algorithms to determine the target retrieval information matching the first input information from the knowledge base, the accuracy of the retrieval results can be enhanced to a certain extent. It should be noted that step S310 and step S320 can be executed synchronously or asynchronously, and the present disclosure does not specifically limit the execution order of step S310 and step S320.

[0060] Specifically, in step S310, based on a first relevance evaluation algorithm, a first relevance score of each knowledge unit in the knowledge base relative to the first input information is determined.

[0061] The first correlation score refers to a correlation score obtained based on a first correlation evaluation algorithm.

[0062] Optionally, the first relevance evaluation algorithm may be a BM25 algorithm. The BM25 algorithm is a probabilistic retrieval algorithm for information retrieval and text search, which is part of the BM series of algorithms and is intended to evaluate the relevance between documents and queries. It should be noted that, in actual application, the first relevance evaluation algorithm may also be implemented using other algorithms different from the second relevance evaluation algorithm, such as the GBM algorithm (Gradient Boosting Machines) based on machine learning, the SVM (Support Vector Machine) algorithm, etc., which is not specifically limited in the present disclosure.

[0063] Optionally, the knowledge unit can be divided by document, that is, a knowledge document is regarded as an indication unit. It should be noted that in actual application, the knowledge unit can also be divided in other ways, which is not specifically limited in the present disclosure.

[0064] Optionally, in step S310, based on the first relevance evaluation algorithm, determining the first relevance score of each knowledge unit in the knowledge base relative to the first input information can be achieved by the following steps: determining the relevance score of each word contained in the first input information relative to the current search knowledge unit; based on the relevance score of each word relative to the current search knowledge unit, obtaining the relevance score of the first input information relative to the current search knowledge unit. The current search knowledge unit can be any knowledge unit among the knowledge units in the knowledge base.

[0065] Optionally, the above determination of the relevance score of each word contained in the first input information relative to the current search knowledge unit can be achieved through the following steps: determine the frequency of occurrence of each word contained in the first input information in the current search knowledge unit, and use the frequency of occurrence of each word in the current search knowledge unit as the word frequency corresponding to each word; for each word, calculate the inverse of the number of knowledge units containing the corresponding word in the knowledge base, and determine the inverse document frequency corresponding to each word; based on the word frequency and inverse document frequency corresponding to each word contained in the first input information, determine the relevance score of each word contained in the first input information relative to the current search knowledge unit. Optionally, after determining the inverse document frequency corresponding to each word, it can be logarithmically transformed to reduce the impact of extreme values.

[0066] Exemplarily, the above-mentioned determination of the relevance score of each word contained in the first input information relative to the current search knowledge unit based on the word frequency and inverse document frequency corresponding to each word contained in the first input information can be achieved through the following steps: Determine the relevance score of each word contained in the first input information relative to the current search knowledge unit based on the word frequency and inverse document frequency corresponding to each word contained in the first input information, the length of the current search knowledge unit, and the average length of the knowledge unit set.

[0067] For example, it can be realized by the following formula (1).

[0068]

[0069] Among them, score represents the relevance score of a single word relative to the current retrieved knowledge unit, TF represents word frequency, IDF represents inverse document frequency, docLength represents the length of the current retrieved knowledge unit, avgDocLength represents the average length of the knowledge unit set in the knowledge base, k1 and b are control parameters. In actual application, k1 and b can be set according to actual application requirements, and the present disclosure does not specifically limit this. Exemplarily, the value range of k1 can be 1.2 to 2.0. When k1 is larger, the influence of word frequency on the score result will gradually decrease. When k1 is smaller, the influence of word frequency on the score result will be more significant. Exemplarily, the value range of b can be between 0 and 1. When b is close to 1, the influence of document length on the score result is greater; when b is close to 0, the influence of document length on the score result is smaller.

[0070] Considering the impact of length on relevance scores, long knowledge units may have higher word frequencies because they are longer. By calculating the average length of the knowledge unit set as the knowledge unit length standardization factor, the impact of long knowledge units due to length when calculating word frequencies can be reduced.

[0071] Optionally, the above-mentioned relevance score of the first input information relative to the current search knowledge unit is obtained based on the relevance score of each word relative to the current search knowledge unit. This can be achieved through the following steps: the relevance score of each word relative to the current search knowledge unit can be summed to obtain the relevance score of the first input information relative to the current search knowledge unit.

[0072] Specifically, in step S320, based on the second relevance evaluation algorithm, a second relevance score of each knowledge unit in the knowledge base relative to the first input information is determined.

[0073] The second correlation score refers to a correlation score obtained based on a second correlation evaluation algorithm.

[0074] Optionally, the second relevance evaluation algorithm can be a Contriever model algorithm. The Contriever model algorithm is a retriever trained without supervision, which searches for documents related to a given query in a large collection, takes the document set and the query as input, and outputs the relevance score of each document. It should be noted that in actual application, the second relevance evaluation algorithm can also be implemented using other algorithms different from the first relevance evaluation algorithm, for example, GBM algorithm (Gradient Boosting Machines) based on machine learning, SVM (Support Vector Machine) algorithm, etc., which is not specifically limited in this disclosure.

[0075] Optionally, in step S320, based on the second relevance evaluation algorithm, determining the second relevance score of each knowledge unit in the knowledge base relative to the first input information can be achieved by the following steps: inputting the first input information into the Contriever model to obtain a vector representation corresponding to the first input information; inputting each knowledge unit in the knowledge base into the Contriever model to obtain a vector representation of each knowledge unit in the knowledge base; and determining the second relevance score of each knowledge unit in the knowledge base relative to the first input information based on the vector representation corresponding to the first input information and the vector representation of each knowledge unit in the knowledge base.

[0076] Exemplarily, the vector representation of each knowledge unit in the knowledge base may be pre-stored in a vector database, such as Faiss, Annoy, etc., to improve ease of use.

[0077] Exemplarily, when determining the second correlation score of each knowledge unit in the knowledge base relative to the first input information based on the vector representation corresponding to the first input information and the vector representation of each knowledge unit in the knowledge base, cosine similarity and other methods can be used for calculation, and the present disclosure does not specifically limit this.

[0078] Exemplarily, the Contriever model can be pre-trained on large-scale text data to optimize the embedding representation between the first input information and the knowledge unit through contrastive learning, so that similar first input information and knowledge units are closer in the vector space.

[0079] Specifically, in step S330, based on the first relevance score and the second relevance score, target retrieval information matching the first input information is determined from the knowledge base.

[0080] Optionally, the total relevance score of each knowledge unit in the knowledge base relative to the first input information can be obtained based on the first relevance score and the second relevance score of each knowledge unit in the knowledge base relative to the first input information; and based on the total relevance score of each knowledge unit in the knowledge base relative to the first input information, target retrieval information matching the first input information is determined from the knowledge base.

[0081] Exemplarily, the first relevance score and the second relevance score of each knowledge unit in the knowledge base relative to the first input information may be summed to obtain a total relevance score of each knowledge unit in the knowledge base relative to the first input information.

[0082] Exemplarily, the knowledge information corresponding to the knowledge unit with the highest total relevance score may be used as the target retrieval information matching the first input information.

[0083] Optionally, a preset number of similar knowledge units may be selected from each knowledge unit in the knowledge base according to the vector representation corresponding to the first input information and the vector representation of each knowledge unit in the knowledge base to obtain a similar knowledge unit set. Based on the first relevance score and the second relevance score corresponding to each knowledge unit in the similar knowledge unit set, a knowledge unit with the highest total relevance score may be determined from the similar knowledge unit set.

[0084] In step S130, second input information is determined based on the target retrieval information, and the second input information is processed by the large language model to obtain input reply information.

[0085] The second input information refers to input information constructed based on the target retrieval information.

[0086] Among them, the large language model refers to a type of language model based on neural networks, which can be trained using huge data sets and can handle a variety of natural language tasks, such as question and answer, dialogue, etc.

[0087] When it is detected that the specific attribute parameter corresponding to the first input information meets the preset knowledge base retrieval condition, a knowledge base retrieval is performed, and the input information of the large language model is constructed using the retrieved target retrieval information, which can enhance the knowledge of the large language model and thus improve the model performance.

[0088] In an optional implementation, in step S130, the second input information is determined based on the target search information, which can be done by: Figure 4 The steps shown are to achieve this:

[0089] Step S410, obtaining the latest update time corresponding to the target search information, and determining whether the latest update time is later than the construction time of the training data corresponding to the large language model;

[0090] Step S420, if the latest update time is later than the construction time of the training data corresponding to the large language model, concatenate the first input information and the target search information to obtain second input information;

[0091] Step S430: If the latest update time is not later than the construction time of the training data corresponding to the large language model, the first input information is used as the second input information.

[0092] Figure 4 In the steps shown, when the latest update time corresponding to the target retrieval information is later than the construction time of the corresponding training data through the large language model, it will not be covered by the training data of the large language model, and the large language model cannot learn such information during the training process. By considering the update time of the information retrieved from the knowledge base, it can be used to decide whether to provide this information to the large language model for knowledge enhancement, and at the same time avoid introducing outdated information to affect the performance of the large language model.

[0093] Specifically, in step S410, the latest update time corresponding to the target search information is obtained, and it is determined whether the latest update time is later than the construction time of the training data corresponding to the large language model.

[0094] The latest update time corresponding to the target retrieval information refers to the time when the corresponding target retrieval information in the knowledge base is last updated. The construction time of the training data refers to the time when the training data corresponding to the large language model is constructed.

[0095] Optionally, when knowledge information in the knowledge base is updated, the update time of the knowledge information may be recorded so as to provide the latest update time.

[0096] Specifically, in step S420, if the latest update time is later than the construction time of the training data corresponding to the large language model, the first input information and the target search information are concatenated to obtain the second input information.

[0097] Optionally, if the latest update time is later than the construction time of the training data corresponding to the large language model, the prompt word template can be used to splice the retrieved target search information and the first input information together through natural language to obtain the second input information. The prompt word template can be a reusable text string, and by replacing the parameter value in the template, a specific prompt word can be generated to obtain the second input information.

[0098] For example, when the first input information is a question type information, the prompt word template can be used to splice the retrieved target search information with the input question information. For example: "Please answer the question based on the following information: [question]. The relevant information is as follows: [target search information 1], [target search information 2]".

[0099] It should be noted that the target search information obtained by the present disclosure may be one or more pieces, and the present disclosure does not specifically limit this.

[0100] If the latest update time is later than the construction time of the training data corresponding to the large language model, the first input information and the target retrieval information are concatenated as the second input information, which can enhance the knowledge of the large language model and thus improve the model performance.

[0101] Specifically, in step S430, if the latest update time is not later than the construction time of the training data corresponding to the large language model, the first input information is used as the second input information.

[0102] If the latest update time is no later than the construction time of the training data corresponding to the large language model, the first input information is used as the second input information to avoid introducing outdated target retrieval information that affects the performance of the large language model. In addition, when it is detected that the latest update time is no later than the construction time of the training data corresponding to the large language model, there is no need to splice the first input information, which can shorten the input length of the large language model to a certain extent and reduce the processing cost of the large language model.

[0103] In an optional implementation, if the specific attribute parameter corresponding to the input information does not satisfy the preset knowledge base search condition, the first input information is processed by a large language model to obtain input reply information.

[0104] When it is detected that the specific attribute parameters corresponding to the input information do not meet the preset knowledge base retrieval conditions, there is no need to perform a knowledge base search. The first input information is directly processed through the large language model to obtain input response information, thereby reducing unnecessary retrieval operations and reducing time costs.

[0105] like Figure 5As shown, a flowchart of adaptive retrieval knowledge enhancement of a large language model is provided, which may specifically include the following steps:

[0106] Step S501, obtaining first input information;

[0107] Step S502, determining whether the popularity of the information corresponding to the first input information meets the preset knowledge base search condition; if so, executing step S503; if not, jumping to step S506;

[0108] Step S503, based on the first input information, searching from the knowledge base to obtain target search information matching the first input information;

[0109] Step S504, determining whether the latest update time corresponding to the target search information is later than the construction time of the training data corresponding to the large language model; if so, executing step S505; if not, jumping to step S506;

[0110] Step S505, concatenating the first input information and the target search information to obtain second input information;

[0111] Step S506, using the first input information as the second input information;

[0112] Step S507: Process the second input information through the large language model to obtain input response information.

[0113] Figure 5 In the steps shown, the popularity of the information is used to determine whether to introduce information from the knowledge base. Since popular related information will appear many times in the training data, the large language model performs well on this information, and there is no need to introduce retrieval operations, thus avoiding the misleading and impact caused by the introduction of new information. In addition, Figure 5 In the steps shown, by considering the update time of the information retrieved from the knowledge base to decide whether to provide this information to the large language model for knowledge enhancement, it is possible to avoid introducing outdated information that affects the performance of the large language model.

[0114] The exemplary embodiments of the present disclosure also provide an information processing device based on a large language model.

[0115] refer to Figure 6 As shown, the information processing device 600 based on the large language model may include the following program modules:

[0116] The attribute parameter judgment model 610 is used to obtain the first input information and judge whether the specific attribute parameter corresponding to the first input information satisfies the preset knowledge base search condition;

[0117] A search information determination module 620 is used to search from the knowledge base based on the first input information to obtain target search information matching the first input information if the specific attribute parameter corresponding to the first input information meets the preset knowledge base search condition;

[0118] The input reply determination module 630 is used to determine the second input information based on the target retrieval information, and process the second input information through the large language model to obtain input reply information.

[0119] In an optional implementation, based on the above solution, the information processing device 600 based on the large language model further includes: an attribute parameter determination module, used to determine specific attribute parameters corresponding to the first input information, wherein the specific attribute parameters include information popularity.

[0120] In an optional embodiment, based on the aforementioned scheme, the attribute parameter determination module can be configured to: obtain one or more statistical parameters corresponding to the first input information; determine the popularity of the information corresponding to the first input information based on the one or more statistical parameters corresponding to the first input information and the weights corresponding to each statistical parameter.

[0121] In an optional implementation, based on the above solution, a knowledge base search condition is preset, including: the information popularity corresponding to the first input information is lower than the preset popularity.

[0122] In an optional embodiment, based on the aforementioned scheme, the information processing device 600 based on the large language model also includes: another input reply determination module, which is used to process the first input information through the large language model to obtain input reply information if the specific attribute parameters corresponding to the input information do not meet the preset knowledge base retrieval conditions.

[0123] In an optional embodiment, based on the aforementioned scheme, the retrieval information determination module 620 can be configured to: determine a first relevance score of each knowledge unit in the knowledge base relative to the first input information based on a first relevance evaluation algorithm; determine a second relevance score of each knowledge unit in the knowledge base relative to the first input information based on a second relevance evaluation algorithm; and determine target retrieval information that matches the first input information from the knowledge base based on the first relevance score and the second relevance score.

[0124] In an optional implementation, based on the aforementioned scheme, the input reply determination module 630 determines the second input information based on the target retrieval information, and can be configured to: obtain the latest update time corresponding to the target retrieval information, and determine whether the latest update time is later than the construction time of the training data corresponding to the large language model; if the latest update time is later than the construction time of the training data corresponding to the large language model, concatenate the first input information and the target retrieval information to obtain the second input information; if the latest update time is not later than the construction time of the training data corresponding to the large language model, use the first input information as the second input information.

[0125] The specific details of each part of the above-mentioned information processing device 600 based on the large language model have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0126] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0127] The exemplary embodiments of the present disclosure also provide a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the information processing method based on the large language model is implemented.

[0128] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.

[0129] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0130] The code of the computer program can be written in one or more programming languages. Programming languages ​​such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0131] The computer program may be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. The electronic device may convert the signal carrying the computer program into a digital signal, thereby running the computer program. When the computer program runs on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device may execute) the method steps of various exemplary embodiments of the present disclosure, such as the above-mentioned information processing method based on a large language model, which includes the following steps:

[0132] Obtaining first input information, and determining whether a specific attribute parameter corresponding to the first input information satisfies a preset knowledge base search condition;

[0133] If the specific attribute parameter corresponding to the first input information meets the preset knowledge base search condition, based on the first input information, search from the knowledge base to obtain target search information matching the first input information;

[0134] Based on the target retrieval information, second input information is determined, and the second input information is processed through the large language model to obtain input response information.

[0135] In an optional implementation, based on the above solution, the following steps may also be performed: determining specific attribute parameters corresponding to the first input information, wherein the specific attribute parameters include information popularity.

[0136] In an optional embodiment, based on the aforementioned scheme, determining the specific attribute parameters corresponding to the first input information can be achieved through the following steps: obtaining one or more statistical parameters corresponding to the first input information; determining the popularity of the information corresponding to the first input information based on the one or more statistical parameters corresponding to the first input information and the weights corresponding to each statistical parameter.

[0137] In an optional implementation, based on the above solution, a knowledge base search condition is preset, including: the information popularity corresponding to the first input information is lower than the preset popularity.

[0138] In an optional implementation, based on the above solution, the following steps may also be performed: if the specific attribute parameters corresponding to the input information do not meet the preset knowledge base retrieval conditions, the first input information is processed by a large language model to obtain input reply information.

[0139] In an optional embodiment, based on the aforementioned scheme, based on the first input information, searching from the knowledge base to obtain target retrieval information matching the first input information can be achieved through the following steps: based on a first relevance evaluation algorithm, determining a first relevance score of each knowledge unit in the knowledge base relative to the first input information; based on a second relevance evaluation algorithm, determining a second relevance score of each knowledge unit in the knowledge base relative to the first input information; based on the first relevance score and the second relevance score, determining the target retrieval information matching the first input information from the knowledge base.

[0140] In an optional implementation, based on the aforementioned scheme, based on the target retrieval information, determining the second input information can be achieved through the following steps: obtaining the latest update time corresponding to the target retrieval information, and determining whether the latest update time is later than the construction time of the training data corresponding to the large language model; if the latest update time is later than the construction time of the training data corresponding to the large language model, concatenating the first input information and the target retrieval information to obtain the second input information; if the latest update time is not later than the construction time of the training data corresponding to the large language model, using the first input information as the second input information.

[0141] In the above-mentioned information processing process based on the large language model, by combining the target retrieval information matching the first input information and using the large language model to obtain the input reply information, the accuracy of the information reply can be improved to a certain extent; when the specific attribute parameters corresponding to the first input information meet the preset knowledge base retrieval conditions, the target retrieval information matching the first input information is obtained, which can reduce unnecessary knowledge base retrieval operations while ensuring the accuracy of the large language model output, and avoid introducing information that may be misleading.

[0142] The exemplary embodiment of the present disclosure also provides an electronic device capable of implementing the above-mentioned information processing method based on a large language model. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as program code. The processor executes the method in the exemplary embodiment by executing the executable instructions. In addition, the electronic device may also include a display for displaying a graphical user interface.

[0143] Reference below Figure 7 , the electronic device is exemplarily described in the form of a general-purpose computing device. It should be understood that Figure 7The electronic device 700 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0144] like Figure 7 As shown, the electronic device 700 may include: a processor 710 , a memory 720 , a bus 730 , an I / O (input / output) interface 740 , a network adapter 750 , and a display 760 .

[0145] The memory 720 may include a volatile memory, such as a RAM 721, a cache unit 722, and may also include a non-volatile memory, such as a ROM 723. The memory 720 may also include one or more program modules 724, such program modules 724 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof may include the implementation of a network environment. For example, the program module 724 may include each module in the above-mentioned device.

[0146] The processor 710 may include one or more processing units. For example, the processor 710 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc.

[0147] The processor 710 may be configured to execute executable instructions stored in the memory 720 , such as executing any one or more method steps in the exemplary embodiment.

[0148] Exemplarily, the processor 710 may perform the following steps:

[0149] Obtaining first input information, and determining whether a specific attribute parameter corresponding to the first input information satisfies a preset knowledge base search condition;

[0150] If the specific attribute parameter corresponding to the first input information meets the preset knowledge base search condition, based on the first input information, search from the knowledge base to obtain target search information matching the first input information;

[0151] Based on the target retrieval information, second input information is determined, and the second input information is processed through the large language model to obtain input response information.

[0152] In an optional implementation, based on the above solution, the following steps may also be performed: determining specific attribute parameters corresponding to the first input information, wherein the specific attribute parameters include information popularity.

[0153] In an optional embodiment, based on the aforementioned scheme, determining the specific attribute parameters corresponding to the first input information can be achieved through the following steps: obtaining one or more statistical parameters corresponding to the first input information; determining the popularity of the information corresponding to the first input information based on the one or more statistical parameters corresponding to the first input information and the weights corresponding to each statistical parameter.

[0154] In an optional implementation, based on the above solution, a knowledge base search condition is preset, including: the information popularity corresponding to the first input information is lower than the preset popularity.

[0155] In an optional implementation, based on the above solution, the following steps may also be performed: if the specific attribute parameters corresponding to the input information do not meet the preset knowledge base retrieval conditions, the first input information is processed by a large language model to obtain input reply information.

[0156] In an optional embodiment, based on the aforementioned scheme, based on the first input information, searching from the knowledge base to obtain target retrieval information matching the first input information can be achieved through the following steps: based on a first relevance evaluation algorithm, determining a first relevance score of each knowledge unit in the knowledge base relative to the first input information; based on a second relevance evaluation algorithm, determining a second relevance score of each knowledge unit in the knowledge base relative to the first input information; based on the first relevance score and the second relevance score, determining the target retrieval information matching the first input information from the knowledge base.

[0157] In an optional implementation, based on the aforementioned scheme, based on the target retrieval information, determining the second input information can be achieved through the following steps: obtaining the latest update time corresponding to the target retrieval information, and determining whether the latest update time is later than the construction time of the training data corresponding to the large language model; if the latest update time is later than the construction time of the training data corresponding to the large language model, concatenating the first input information and the target retrieval information to obtain the second input information; if the latest update time is not later than the construction time of the training data corresponding to the large language model, using the first input information as the second input information.

[0158] In the above-mentioned information processing process based on the large language model, by combining the target retrieval information matching the first input information and using the large language model to obtain the input reply information, the accuracy of the information reply can be improved to a certain extent; when the specific attribute parameters corresponding to the first input information meet the preset knowledge base retrieval conditions, the target retrieval information matching the first input information is obtained, which can reduce unnecessary knowledge base retrieval operations while ensuring the accuracy of the large language model output, and avoid introducing information that may be misleading.

[0159] The bus 730 is used to realize the connection between different components of the electronic device 700, and may include a data bus, an address bus, and a control bus.

[0160] The electronic device 700 can communicate with one or more external devices 800 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 740 .

[0161] The electronic device 700 can communicate with one or more networks through the network adapter 750. For example, the network adapter 750 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. The network adapter 750 can communicate with other modules of the electronic device 700 through the bus 730.

[0162] The electronic device 700 may display a graphical user interface and the like through the display 760 .

[0163] although Figure 7 Not shown, other hardware and / or software modules may also be provided in the electronic device 700, including but not limited to: a display, a microcode, a device driver, a redundant processor, an external disk drive array, a RAID (Redundant Arrays of Independent Disks) system, a tape drive, and a data backup storage system.

[0164] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0165] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module" or "system". Those skilled in the art will readily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and implementation are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0166] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An information processing method based on a large language model, characterized in that: The method comprises: Acquire first input information, and determine whether a specific attribute parameter corresponding to the first input information satisfies a preset knowledge base search condition; If the specific attribute parameter corresponding to the first input information meets the preset knowledge base search condition, based on the first input information, search from the knowledge base to obtain target search information matching the first input information; Based on the target retrieval information, second input information is determined, and the second input information is processed through a large language model to obtain input reply information.

2. The method according to claim 1, characterized in that The method further comprises: Determine a specific attribute parameter corresponding to the first input information, wherein the specific attribute parameter includes information popularity.

3. The method according to claim 2, characterized in that The determining the specific attribute parameter corresponding to the first input information includes: Acquire one or more statistical parameters corresponding to the first input information; Based on one or more statistical parameters corresponding to the first input information and weights corresponding to the statistical parameters, the popularity of information corresponding to the first input information is determined.

4. The method according to claim 2, characterized in that: The preset knowledge base search conditions include: The information popularity corresponding to the first input information is lower than the preset popularity.

5. The method according to claim 1, characterized in that The method further comprises: If the specific attribute parameter corresponding to the input information does not satisfy the preset knowledge base search condition, the first input information is processed by a large language model to obtain input reply information.

6. The method according to claim 1, characterized in that The step of searching from a knowledge base based on the first input information to obtain target search information matching the first input information includes: Determining, based on a first relevance evaluation algorithm, a first relevance score of each knowledge unit in the knowledge base relative to the first input information; Determining, based on a second relevance evaluation algorithm, a second relevance score of each knowledge unit in the knowledge base relative to the first input information; Based on the first relevance score and the second relevance score, target retrieval information matching the first input information is determined from the knowledge base.

7. The method according to claim 1, characterized in that The determining the second input information based on the target retrieval information includes: Obtaining the latest update time corresponding to the target retrieval information, and determining whether the latest update time is later than the construction time of the training data corresponding to the large language model; If the latest update time is later than the construction time of the training data corresponding to the large language model, concatenating the first input information and the target search information to obtain second input information; If the latest update time is not later than the construction time of the training data corresponding to the large language model, the first input information is used as the second input information.

8. An information processing device based on a large language model, characterized in that: The device comprises: An attribute parameter judgment model, used to obtain first input information and judge whether a specific attribute parameter corresponding to the first input information satisfies a preset knowledge base search condition; A search information determination module, configured to search from a knowledge base based on the first input information to obtain target search information matching the first input information if the specific attribute parameter corresponding to the first input information meets a preset knowledge base search condition; The input reply determination module is used to determine the second input information based on the target retrieval information, and process the second input information through the large language model to obtain input reply information.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

Citation Information

Cited By

  • Retrieval enhancement generation method and device, computer program product and electronic equipment

    CN119621933A