Information recommendation method and device, equipment and storage medium

Abstract

The invention provides an information recommendation method and device, equipment and a storage medium, relates to the technical field of computers, in particular to the technical field of artificial intelligence such as large models, intelligent recommendation and intelligent agents, and can be applied to the field of intelligent finance and the like. According to the specific implementation scheme, in response to the received current multi-round dialogue between a first object and a second object, the intention of the first object is determined based on the current multi-round dialogue, the first object is a served party, and the second object is a service provider; based on the intention of the first object, determining a plurality of candidate entries and reference multi-round conversations associated with the plurality of candidate entries from a database; determining one or more reference entries from the plurality of candidate entries based on the intention of the first object and the reference multi-round dialogues associated with the plurality of candidate entries; and taking the one or more reference entries as reference recommendation information, and sending the reference recommendation information.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the field of artificial intelligence technology such as large models, intelligent recommendations, and intelligent agents, and can be used in the fields of smart finance and the like. Background Art

[0002] In the related art, intelligent customer service determines the user's intention based on a question input by the user, and then retrieves the reference content with the highest similarity to the user's intention in the database. However, in this way, when the question input by the user is not accurate enough, the user's intention identified by the intelligent customer service is often not accurate enough, which leads to inaccurate reference content determined based on inaccurate intention, and thus it is impossible to provide accurate reference to the user through the reference content, affecting the quality of service provided to the user. Therefore, how to accurately identify the user's actual intention and then obtain reference content that matches the user's intention, so as to provide the user with quality service, becomes a technical problem to be solved. Summary of the invention

[0003] The present disclosure provides a method, apparatus, device and storage medium for information recommendation.

[0004] According to one aspect of the present disclosure, there is provided an information recommendation method, comprising:

[0005] In response to receiving a current multi-round dialogue between a first object and a second object, determining an intention of the first object based on the current multi-round dialogue, wherein the first object is a service recipient and the second object is a service provider;

[0006] Based on the intention of the first object, determining a plurality of candidate entries and a reference multi-round dialogue associated with the plurality of candidate entries from a database;

[0007] Determine one or more reference items from the multiple candidate items based on the intention of the first object and the reference multiple rounds of dialogue associated with the multiple candidate items;

[0008] The one or more reference items are used as reference recommendation information, and the reference recommendation information is sent.

[0009] According to another aspect of the present disclosure, there is provided an information recommendation device, comprising:

[0010] an intention analysis module, configured to, in response to receiving a current multi-round dialogue between a first object and a second object, determine the intention of the first object based on the current multi-round dialogue, wherein the first object is a service recipient and the second object is a service provider;

[0011] A retrieval module, configured to determine, from a database, a plurality of candidate entries and a plurality of reference rounds of dialogue associated with the plurality of candidate entries based on the intention of the first object;

[0012] a screening module, configured to determine one or more reference items from the plurality of candidate items based on the intention of the first object and the reference multi-round dialogues associated with the plurality of candidate items;

[0013] The reference recommendation information sending module is used to use the one or more reference items as reference recommendation information and send the reference recommendation information.

[0014] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0018] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0019] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0020] By adopting the method provided in this embodiment, the intention of the first object can be more accurately obtained based on the analysis of the current multiple rounds of dialogue between the first object as the service recipient and the second object as the service provider. Furthermore, through a more accurate intention of the first object, it can be ensured that the obtained candidate entries can provide questions and answers that are closer to the intention of the first object. In addition, in combination with the intention of the first object and the reference multiple rounds of dialogue associated with the multiple candidate entries, reference recommendation information that is closer to the intention of the first object can be more accurately obtained from the multiple candidate entries and sent, thereby obtaining reference information that matches the intention of the first object, thereby improving the quality of service provided to the first object.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0023] Figure 1 is a flowchart of an information recommendation method according to an embodiment of the present disclosure;

[0024] Figure 2 is a flowchart of an information recommendation method according to another embodiment of the present disclosure;

[0025] Figure 3 is a schematic block diagram of an information recommendation device according to an embodiment of the present disclosure;

[0026] Figure 4 is a schematic block diagram of an information recommendation device according to another embodiment of the present disclosure;

[0027] Figure 5 is a block diagram of an electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0029] Figure 1 It is a schematic flow chart of an information recommendation method provided according to an embodiment of the present disclosure. The method is applicable to situations in which information recommendation is performed, and is particularly applicable to situations in which information recommendation is performed in the financial field. The method can be performed by an information recommendation device, which can be implemented in software and / or hardware, and can be integrated into an electronic device that carries the function of recommending information, such as a server. It should be noted that the information recommendation method of the present disclosure can be performed by a large model-based intelligent agent in a server, wherein an intelligent agent refers to a computer program based on a large language model, which has planning and thinking capabilities, memory capabilities, and the ability to use tool functions, and can autonomously complete a given task.

[0030] Figure 1 As shown, the information recommendation method proposed in the embodiment of the present disclosure includes:

[0031] S110, in response to receiving a current multi-round dialogue between a first object and a second object, determining the intention of the first object based on the current multi-round dialogue, wherein the first object is a service recipient and the second object is a service provider.

[0032] Receiving the current multi-round dialogue between the first object and the second object may refer to the server receiving the current multi-round dialogue between the first object and the second object from the second device.

[0033] The first object may be a first user, and the first user may be a user who currently needs to be served. The first user may use a first device.

[0034] The second object is a service provider, that is, the second object may be a service provider that provides services for the first object; specifically, the second object may be a customer service, for example, the second object may be a manual customer service or an intelligent customer service.

[0035] Optionally, the second object may be a human customer service, and the second device is an electronic device used by the second object. Optionally, the second object may be an intelligent customer service, and the intelligent customer service may be provided as a module or function in the second device.

[0036] The device type of the first device may be any one of a mobile phone, a tablet computer, a desktop computer, a laptop computer, etc., and the device type of the second device may be any one of a mobile phone, a tablet computer, a desktop computer, a laptop computer, etc., which are not listed here for brevity.

[0037] The determining of the intention of the first object based on the current multiple rounds of dialogue may be: an agent in the server determines the intention of the first object based on the current multiple rounds of dialogue.

[0038] S120: Based on the intention of the first object, determine from a database a plurality of candidate entries and a reference multi-round dialogue associated with the plurality of candidate entries.

[0039] The reference multi-round dialogues associated with the multiple candidate entries include: one or more reference multi-round dialogues associated with each candidate entry in the multiple candidate entries.

[0040] Each candidate entry includes a candidate question and its corresponding candidate answer.

[0041] The method of determining multiple candidate entries and reference multi-round dialogues associated with the multiple candidate entries from a database based on the intention of the first object may be: the agent in the server determines multiple candidate entries and reference multi-round dialogues associated with the multiple candidate entries from a database based on the intention of the first object.

[0042] S130: Determine one or more reference items from the multiple candidate items based on the intention of the first object and the reference multiple rounds of dialogue associated with the multiple candidate items.

[0043] Among them, the determining one or more reference items from the multiple candidate items based on the intention of the first object and the reference multi-round dialogues associated with the multiple candidate items may be: the agent in the server determines one or more reference items from the multiple candidate items based on the intention of the first object and the reference multi-round dialogues associated with the multiple candidate items.

[0044] S140: Use the one or more reference items as reference recommendation information, and send the reference recommendation information.

[0045] The taking one or more reference items as reference recommendation information may be: the intelligent agent in the server takes the one or more reference items as reference recommendation information.

[0046] The sending of the reference recommendation information may be: the server sending the reference recommendation information to the second device, wherein the reference recommendation information is used for the second object corresponding to the second device to provide a service for the first object.

[0047] By adopting the method provided in this embodiment, the intention of the first object can be more accurately obtained based on the analysis of the current multiple rounds of dialogue between the first object as the service recipient and the second object as the service provider. Furthermore, through a more accurate intention of the first object, it can be ensured that the obtained candidate entries can provide questions and answers that are closer to the intention of the first object. In addition, in combination with the intention of the first object and the reference multiple rounds of dialogue associated with the multiple candidate entries, reference recommendation information that is closer to the intention of the first object can be more accurately obtained from the multiple candidate entries and sent, thereby obtaining reference information that matches the intention of the first object, thereby improving the quality of service provided to the first object.

[0048] In one embodiment, determining the intention of the first object based on the current multi-round conversations includes: determining multi-round conversations to be analyzed based on the current multi-round conversations; generating input information based on the multi-round conversations to be analyzed; inputting the input information into an intention analysis model to obtain analysis results corresponding to the multi-round conversations to be analyzed output by the intention analysis model, wherein the analysis results include the intention of the first object.

[0049] In one case, the current multi-round conversation between the first object and the second object received by the server may be the voice recognition result of all voice conversations between the first object and the second object recorded on the second device side before the triggering moment, which is received by the server from the second device. Wherein, in the case where the second object is a manual customer service, the triggering moment is the moment corresponding to the second object performing a triggering operation on the second device; in the case where the second object is an intelligent customer service, the triggering moment is the moment corresponding to the second device determining that the triggering condition is met. The triggering operation may be the operation of the second object clicking a corresponding physical trigger button or a virtual trigger button on the second device according to actual needs, which is not limited in this embodiment. The triggering condition may be preset in the second device, which is not limited in this embodiment.

[0050] The current multi-round dialogue may be a speech recognition result corresponding to the speech dialogue between the first object and the second object, and the speech recognition result may be a character recognition result or a text recognition result.

[0051] The current multiple rounds of conversations have a sequence. The method for determining the sequence of the current multiple rounds of conversations may include one of the following: sorting each round of conversations in the current multiple rounds of conversations based on the order of the timestamps of the speech recognition results corresponding to each round of conversation from early to late; sorting each round of conversations in the current multiple rounds of conversations based on the position of the speech recognition results corresponding to each round of conversation in the current multiple rounds of conversations.

[0052] Each round of the current multi-round dialogue includes speech recognition results corresponding to two voices, and the speech recognition results corresponding to the two voices included in each round of the dialogue also have a sequence. The method for determining the sequence of the speech recognition results corresponding to the two voices included in each round of the dialogue includes one of the following: sorting the speech recognition results corresponding to the two voices based on the order of the timestamps of the speech recognition results corresponding to the two voices from early to late; sorting the speech recognition results corresponding to the two voices based on the positions of the speech recognition results corresponding to the two voices.

[0053] Optionally, the determining of the multi-round dialogue to be analyzed based on the current multi-round dialogue may include: the agent in the server determines whether the number of rounds of the current multi-round dialogue is greater than Z; if the number of rounds of the current multi-round dialogue is greater than Z, taking the dialogue ranked as the last Z rounds in the current multi-round dialogue as the multi-round dialogue to be analyzed; if the number of rounds of the current multi-round dialogue is less than or equal to Z, taking the current multi-round dialogue as the multi-round dialogue to be analyzed. Wherein, Z may be a positive integer greater than or equal to 2 pre-set according to actual conditions. In one example, Z may be equal to 5.

[0054] Optionally, the determining the multi-round dialogues to be analyzed based on the current multi-round dialogues may include the agent in the server using the current multi-round dialogues as the multi-round dialogues to be analyzed. That is, in this case, it is not determined whether the number of rounds of the current multi-round dialogues is greater than Z, and the current multi-round dialogues are directly used as the multi-round dialogues to be analyzed.

[0055] In one case, the current multi-round conversation between the first object and the second object received by the server may be the speech recognition result of the last Z rounds of voice conversation between the first object and the second object recorded on the second device side before the triggering moment, which is received by the server from the second device.

[0056] In this case, determining the multi-round dialogues to be analyzed based on the current multi-round dialogues may include the agent in the server using the current multi-round dialogues as the multi-round dialogues to be analyzed.

[0057] It should be pointed out that if the number of rounds of speech recognition results corresponding to the voice conversation between the first object and the second object recorded before the triggering moment on the second device side is less than Z, the former situation can be adopted for processing, that is, the speech recognition results of all voice conversations between the first object and the second object recorded before the triggering moment are sent to the server, and no repetitive explanation is given here.

[0058] All the multiple rounds of conversations to be analyzed must include speech recognition results corresponding to one or more voices of the first object and speech recognition results corresponding to one or more voices of the second object. Each round of conversations in the multiple rounds of conversations to be analyzed includes speech recognition results corresponding to two voices. It should be understood that although all the multiple rounds of conversations to be analyzed must include speech recognition results corresponding to the voices corresponding to the two objects respectively, a certain round of conversations in the multiple rounds of conversations to be analyzed may include speech recognition results corresponding to two voices of the same object, or speech recognition results corresponding to two voices corresponding to two objects respectively.

[0059] In one example, generating input information based on the multiple rounds of conversations to be analyzed may include: an agent in the server using the multiple rounds of conversations to be analyzed as input information.

[0060] In one example, the generating of input information based on the multiple rounds of dialogue to be analyzed may include: the agent in the server uses the multiple rounds of dialogue to be analyzed and prompt words as input information. The prompt words are used to guide the intention analysis model to understand the task corresponding to the input information and supplement the background knowledge. The prompt words can be set according to the actual situation and are not limited here.

[0061] Optionally, the analysis result may include a summary of the intent and content of the first object.

[0062] The content summary is used to represent the question that the first subject wants to ask obtained by analyzing the multiple rounds of conversations to be analyzed. The content summary can also be called a conversation summary of the multiple rounds of conversations to be analyzed, or the question that the first subject wants to ask.

[0063] The purpose of the first object's intention is to express the question that the first object wants to ask in the form of a standard question. The first object's intention can be a standard question rewritten from a content summary or a question in the form (or format) of a standard question. The first object's intention can also be called a rewrite of a content summary, or a rewrite of a conversation summary, or a standard question, etc. Among them, the standard question form can be a standard question form for a specified business field, and the specified business field can include one of the following: financial field, medical field, education field, logistics field, etc.

[0064] Optionally, the analysis result may include only the intention of the first object.

[0065] In this way, the intention analysis model is used to analyze the multiple rounds of conversations to be analyzed in the current multiple rounds of conversations to obtain the intention of the first object, thereby improving the efficiency and accuracy of obtaining the intention of the first object.

[0066] In one embodiment, generating input information based on the multiple rounds of conversations to be analyzed includes: inputting the multiple rounds of conversations to be analyzed into a keyword extraction model to obtain target keywords output by the keyword extraction model; when there are one or more target reference questions matching the target keywords in multiple preset entries included in the database, using the multiple rounds of conversations to be analyzed and the one or more target reference questions as the input information.

[0067] Each of the plurality of preset items includes a preset question and a corresponding preset answer.

[0068] The keyword extraction model may be obtained by training the second preset model, wherein the second preset model may be a large language model. The specific method of training the second preset model is not limited in the present application.

[0069] The database includes multiple preset items, wherein each preset item in the multiple preset items includes a preset question and a preset answer corresponding to the preset question, and different preset items in the multiple preset items include different preset questions and / or different preset answers. The multiple preset items at least include multiple preset items in a specified business field; in addition, the multiple preset items may or may not include preset items in other business fields, which is not limited here.

[0070] In addition to multiple preset entries, the database also includes at least one of the following: a vector of each preset entry in the multiple preset entries, a preset business type associated with each preset entry, one or more reference multi-round conversations associated with each preset entry, and a vector of each reference multi-round conversation in the one or more reference multi-round conversations.

[0071] The vector acquisition method of each preset entry in the multiple preset entries may include one of the following: the intelligent agent in the server inputs each preset entry in the multiple preset entries into the vector extraction model to obtain the vector of each preset entry in the multiple preset entries output by the vector extraction model; the intelligent agent in the server calculates each preset entry using a preset vector calculation method to obtain the vector of each preset entry. The vector extraction model may be a BGE-large embedding (Bag of Global Embeddings-Large Embedding) model, and the vector calculation method is not limited in this embodiment.

[0072] The method for obtaining the one or more reference multi-round dialogues associated with each preset item includes: the agent in the server inputs each preset item into a dialogue generation model, and obtains the one or more reference multi-round dialogues associated with each preset item output by the dialogue generation model. The dialogue generation model can be obtained by training a third preset model, and the third preset model can be a large language model. The training method for training the third preset model is not limited in this application.

[0073] Determining whether there are reference questions matching the target keyword in multiple preset items included in the database may include: an intelligent agent in the server determines whether the preset questions of multiple preset items included in the data include the target keyword; in a case where the preset questions of one or more first items in the multiple preset items include the target keyword, using the preset questions in the one or more first items as the one or more target reference questions; in a case where the preset questions of each preset item in the multiple preset items do not include the target keyword, determining that there are no reference questions matching the target keyword in the multiple preset items included in the database.

[0074] In one example, taking the multiple rounds of conversations to be analyzed and the one or more target reference questions as the input information may include: the agent in the server taking the one or more target reference questions as a small sample (Few-shot) of prompt words (prompt); taking the multiple rounds of conversations to be analyzed and the prompt words containing the small sample as input information.

[0075] In addition, it may also include: when there is no reference question matching the target keyword in the multiple preset entries included in the database, the intelligent agent in the server uses the multiple rounds of dialogue to be analyzed as the input information.

[0076] In this way, through the target keywords corresponding to the multiple rounds of conversations to be analyzed, target reference questions related to the multiple rounds of conversations to be analyzed can be retrieved, and then through the target reference questions and the input information generated by the multiple rounds of conversations to be analyzed, the intention analysis model can obtain more reference information, thereby improving the accuracy of the intention analysis model in analyzing the intention of the first object.

[0077] In one embodiment, a training method for an intent analysis model may include: inputting a training sample into a first preset model to obtain a predictive analysis result output by the first preset model, wherein the training sample includes multiple rounds of historical conversations between a third object and a fourth object, the third object is a service recipient, and the fourth object is a service provider, and the predictive analysis result includes the predicted intent of the third object; obtaining a target loss based on the predictive analysis result and multiple sample annotations corresponding to the training sample, wherein the multiple sample annotations corresponding to the training sample are obtained based on multiple large models, the parameter amounts of the multiple large models are greater than the parameter amounts of the first preset model, and the parameter amounts of different large models in the multiple large models are different; training the first preset model based on the target loss to obtain the intent analysis model.

[0078] The parameter amount of the multiple large models is greater than the parameter amount of the first preset model, which may include: the parameter amount of each large model in the multiple large models is greater than the parameter amount of the first preset model.

[0079] The number of the training samples can be one or more.

[0080] Exemplarily, the number of training samples is one.

[0081] Optionally, the training sample may include historical multi-round conversations between the third object and the fourth object.

[0082] The third object may be a second user, and the second user may be a user who has been served in the past. The second user and the first user may be the same or different, and this application does not limit this.

[0083] The fourth object is a service provider, that is, the fourth object may be a service provider that provides services to the third object; specifically, the fourth object may be a customer service, and the fourth object and the second object may be the same customer service or different customer service, which is not limited in this application.

[0084] Optionally, the training sample may include a historical multi-round dialogue between the third object and the fourth object, and one or more historical reference questions associated with the historical multi-round dialogue.

[0085] The method for obtaining one or more historical reference questions associated with the historical multi-round dialogue may include: inputting the historical multi-round dialogue between the third object and the fourth object into a keyword extraction model to obtain keywords corresponding to the historical multi-round dialogue output by the keyword extraction model; judging whether there are reference questions matching the keywords corresponding to the historical multi-round dialogue in the multiple preset entries contained in the database; and extracting one or more historical reference questions associated with the historical multi-round dialogue when there are one or more historical reference questions matching the keywords corresponding to the historical multi-round dialogue in the multiple preset entries contained in the database. The method for judging whether there are reference questions matching the keywords corresponding to the historical multi-round dialogue in the multiple preset entries contained in the database is the same as the above-mentioned method for judging whether there are reference questions matching the target keywords in the multiple preset entries contained in the database, and will not be repeated here.

[0086] Exemplarily, the number of training samples is multiple.

[0087] Optionally, each of the multiple training samples may include a historical multi-round dialogue between the third object and the fourth object.

[0088] Optionally, each of the multiple training samples may include a historical multi-round dialogue between the third object and the fourth object, and one or more historical reference questions associated with the historical multi-round dialogue. Different training samples in the multiple training samples include different one or more historical reference questions associated with the historical multi-round dialogue. The method for obtaining the one or more historical reference questions associated with the historical multi-round dialogue included in each training sample is the same as the above example and will not be repeated here.

[0089] In a scenario of multiple training samples, different training samples among the multiple training samples include different historical rounds of conversations; and different training samples among the multiple training samples correspond to different third objects, and / or different training samples among the multiple training samples correspond to different fourth objects, wherein the relevant descriptions of the third object and the fourth object are similar to the above examples and are not repeated here.

[0090] Regardless of whether there are one or more training samples, the processing of determining the multiple sample annotations corresponding to the training sample is the same for each training sample. Taking any training sample as the qth training sample (q is an integer greater than or equal to 1) as an example, the method of determining the multiple sample annotations corresponding to the training sample may include: inputting the qth training sample into multiple large models respectively, and obtaining the sample annotations of the qth training sample output by the multiple large models respectively. The large model may also be called a large language model.

[0091] An exemplary explanation is given for the multiple large models: assuming that the number of the multiple large models is three, the parameter amount of the first large model may be a billion-parameter model, the parameter amount of the second large model may be a tens-billion-parameter model, the parameter amount of the third large model may be a hundred-billion-parameter model, and the parameter amount of the first preset model may be less than a billion parameters.

[0092] Regardless of whether there are one or more training samples, for each training sample, the process of inputting the training sample into the first preset model to obtain the prediction analysis result output by the first preset model is the same. Still taking any training sample as the qth training sample as an example, the process of inputting the training sample into the first preset model to obtain the prediction analysis result output by the first preset model includes: inputting the qth training sample and the prompt word into the first preset model to obtain the prediction analysis result of the qth training sample output by the first preset model. Among them, the prompt word is the same as above and will not be repeated here.

[0093] Optionally, the prediction analysis result of the qth training sample includes the predicted intention of the third object of the qth training sample and the predicted content summary of the qth training sample. The predicted content summary is similar to the relevant description of the above content summary, and the relevant description of the predicted intention of the third object is similar to the relevant description of the intention of the above first object, and no further description is given.

[0094] Optionally, the prediction analysis result of the qth training sample includes the predicted intention of the third object of the qth training sample.

[0095] In one embodiment, the target loss is obtained based on the prediction analysis result and multiple sample annotations corresponding to the training sample, including: calculating the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample; and obtaining the target loss based on the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample.

[0096] In one example, the number of training samples is one.

[0097] In a case where the prediction output result includes the prediction intention of the third object and the prediction content summary of the training sample, obtaining the target loss based on the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample may include: calculating the loss corresponding to each sample annotation in the multiple sample annotations based on the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample; and adding the loss corresponding to each sample annotation to obtain the target loss.

[0098] Among them, the multiple sample annotations are obtained by processing the training samples by each large model in the multiple large models, so each sample annotation can also be called the sample annotation corresponding to each large model.

[0099] Optionally, taking any one of the multiple sample annotations as the i-th sample annotation (or the sample annotation corresponding to the i-th large model) as an example, based on the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample, the loss corresponding to each sample annotation in the multiple sample annotations is calculated, including: based on the probability distribution between the predicted content summary in the prediction analysis result and the training sample, and the probability distribution between the content summary annotation included in the i-th sample annotation and the training sample, the i-th first sub-loss is obtained; based on the probability distribution between the predicted intent in the prediction analysis result and the training sample, and the probability distribution between the intent annotation included in the i-th sample annotation and the training sample, the i-th second sub-loss is obtained; the sum of the i-th first sub-loss and the i-th second sub-loss is used as the loss corresponding to the i-th sample annotation. Wherein, i is a positive integer.

[0100] The method of obtaining the i-th first sub-loss based on the probability distribution between the predicted content summary in the prediction analysis result and the training sample, and the probability distribution between the content summary annotation included in the i-th sample annotation and the training sample, includes: calculating the probability distribution between the predicted content summary in the prediction analysis result and the training sample, the probability distribution between the content summary annotation included in the i-th sample annotation in multiple sample annotations and the training sample, and the first weight corresponding to the i-th large model based on the divergence function to obtain the i-th first sub-loss. The divergence function can be a KL (Kullback-Leibler) divergence function. Among them, the first weights corresponding to different large models in the multiple large models are different, and the first weight corresponding to each large model in the multiple large models is related to the parameter amount of each large model. For example, the larger the parameter amount of a large model, the higher the corresponding first weight.

[0101] The method of obtaining the i-th second sub-loss based on the probability distribution between the predicted intent in the prediction analysis result and the training sample, and the probability distribution between the intent annotation contained in the i-th sample annotation and the training sample, includes: calculating the probability distribution between the predicted intent in the prediction analysis result and the training sample, the probability distribution between the intent annotation contained in the i-th sample annotation and the training sample, and the second weight corresponding to the i-th large model based on the divergence function to obtain the i-th second sub-loss. Among them, the second weights corresponding to different large models in the multiple large models are different, and the second weight corresponding to each large model in the multiple large models is related to the parameter amount of each large model, for example: the larger the parameter amount of a large model, the higher the corresponding second weight, anyway.

[0102] Optionally, taking any one of the multiple sample annotations as the i-th sample annotation as an example, the loss corresponding to each sample annotation in the multiple sample annotations is calculated based on the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample, including: obtaining the i-th second sub-loss based on the probability distribution between the predicted intent in the prediction analysis result and the training sample, and the probability distribution between the intent annotation included in the i-th sample annotation and the training sample; using the i-th second sub-loss as the loss corresponding to the i-th sample annotation. The calculation method of the i-th second sub-loss is the same as the above example, which will not be repeated here.

[0103] In one example, the number of training samples is multiple.

[0104] In a case where the prediction output result includes the prediction intention of the third object and the prediction content summary of the training sample, obtaining the target loss based on the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample may include: calculating the loss corresponding to each sample annotation in the multiple sample annotations of each training sample in the multiple training samples based on the probability distribution between the prediction analysis results corresponding to the multiple training samples and each training sample in the multiple training samples, and the probability distribution between the multiple sample annotations of each training sample and each training sample; adding the loss corresponding to each sample annotation in the multiple sample annotations of each training sample to obtain the total loss corresponding to each training sample; and adding the total losses corresponding to each training sample to obtain the target loss.

[0105] Among them, the multiple sample annotations of each training sample are obtained by processing the training sample by each large model in the multiple large models. Therefore, each sample annotation of each training sample can also be called the sample annotation corresponding to each large model under the training sample.

[0106] Optionally, taking any one of the multiple training samples as the qth training sample, and any one of the multiple sample annotations of the qth training sample as the ith sample annotation (or the sample annotation corresponding to the ith large model under the qth training sample) as an example, the probability distribution between the prediction analysis results corresponding to the multiple training samples and each training sample in the multiple training samples, and the probability distribution between the multiple sample annotations of each training sample and each training sample, to calculate the loss corresponding to each sample annotation in the multiple sample annotations of each training sample in the multiple training samples, includes: based on the prediction content summary in the prediction analysis result corresponding to the qth training sample and the probability distribution between the qth training sample The probability distribution of the i-th sample annotation of the q-th training sample and the probability distribution between the content summary annotation contained in the i-th sample annotation of the q-th training sample and the q-th training sample are used to obtain the i-th first sub-loss corresponding to the i-th sample annotation of the q-th training sample; based on the probability distribution between the predicted intention in the prediction analysis result corresponding to the q-th training sample and the q-th training sample, and the probability distribution between the intention annotation contained in the i-th sample annotation of the q-th training sample and the q-th training sample, the i-th second sub-loss corresponding to the i-th sample annotation of the q-th training sample is obtained; the sum of the i-th first sub-loss and the i-th second sub-loss is used as the loss corresponding to the i-th sample annotation of the q-th training sample. Where q is a positive integer. Wherein, the calculation method of the i-th first sub-loss and the i-th second sub-loss is similar to the above example and will not be repeated here.

[0107] Optionally, taking any one of the multiple training samples as the qth training sample and any one of the multiple sample annotations of the qth training sample as the ith sample annotation as an example, the loss corresponding to each sample annotation of the multiple sample annotations of each training sample in the multiple training samples is calculated based on the probability distribution between the prediction analysis results corresponding to the multiple training samples and each training sample in the multiple training samples, and the probability distribution between the multiple sample annotations of each training sample and each training sample, including: based on the probability distribution between the predicted intention in the prediction analysis result corresponding to the qth training sample and the qth training sample, and the probability distribution between the intention annotation included in the ith sample annotation of the qth training sample and the qth training sample, the ith second sub-loss corresponding to the ith sample annotation of the qth training sample is obtained; the ith second sub-loss is used as the loss corresponding to the ith sample annotation of the qth training sample. The calculation method of the ith second sub-loss is similar to the above example and will not be repeated here.

[0108] Still taking any one of the multiple training samples as the qth training sample as an example, the loss corresponding to each sample annotation in the multiple sample annotations of each training sample is added to obtain the total loss corresponding to each training sample, including: adding the loss corresponding to each sample annotation in the multiple sample annotations of the qth training sample to obtain the total loss corresponding to the qth training sample.

[0109] When there are three large models, the above total loss is explained in combination with the formula:

[0110]

[0111] in, is the target loss. i can be used to refer to different large models, i∈b,tb,hb respectively represent large models with billion parameters (b), ten billion parameters (tb), and hundred billion parameters (hb).

[0112] s q is the prediction content summary of the prediction analysis results corresponding to the qth training sample, x q is the qth training sample, s i q The content summary annotation of the qth training sample output by the ith large model, q ranges from 1 to N, and θ represents the parameters of the first preset model; P θ (s q |x q ) is the probability distribution between the content summary of the prediction analysis result corresponding to the qth training sample and the qth training sample, P i (s i q |x q ) is the content of the sample annotation corresponding to the i-th large model under the q-th training sample, summarizing the probability distribution between the annotation and the q-th training sample; D KL is the divergence function, α i is the first weight corresponding to the i-th large model, α i ·D KL (P θ (s q |x q )||P i (s i q |x q )) is the i-th first sub-loss corresponding to the i-th sample label of the q-th training sample.

[0113] r q is the prediction intention in the prediction analysis result corresponding to the qth training sample, ri q The intention annotation in the sample annotation corresponding to the i-th large model under the q-th training sample, β i is the second weight corresponding to the i-th large model; β i ·D KL (P θ (r q |x q )||P i (r i q |x q )) is the i-th second sub-loss corresponding to the i-th sample label of the q-th training sample.

[0114] When there is only one training sample (ie N = 1), we only need to calculate The sum of the losses corresponding to each sample annotation is added to obtain the target loss. When there are multiple training samples (i.e. N is a positive integer greater than 1), it is necessary to calculate The summation is performed, that is, the loss corresponding to each sample annotation in multiple sample annotations of N training samples is added to obtain the total loss corresponding to each training sample; the total losses corresponding to the N training samples are added to obtain the target loss.

[0115] The training of the first preset model based on the target loss to obtain the intention analysis model may include: adjusting the parameters of the first preset model based on the target loss reverse conduction; obtaining the trained first preset model when the convergence condition is met, and using the trained first preset model as the intention analysis model. The convergence condition may be at least one of the following: the target loss is zero; the target loss is less than or equal to a threshold; the number of training iterations reaches a preset number, etc.

[0116] In this way, the sample annotations output by multiple large models with larger parameters are used to train the first preset model with smaller parameters. Through such distillation training method, the first preset model with smaller parameters can learn the capabilities of the large models; and, by calculating the target loss through the probability distribution between the prediction analysis results and the training samples, and the probability distribution between the multiple sample annotations and the training samples, the accuracy of the target loss can be guaranteed, and further it can be guaranteed that the trained intent analysis model can more accurately learn the capabilities of multiple large models, thereby improving the accuracy of the trained intent analysis model in predicting intent.

[0117] In addition, the scenarios to which this application applies are particularly suitable for voice channels, so there are higher requirements for real-time performance. Although the model with a larger number of parameters is more accurate in summary, it cannot meet the real-time requirements in actual use. Although the first preset model with a relatively small number of parameters is fast, it lacks in the accuracy of the reply content and the standardization of the reply format. Therefore, by training the first preset model in the above manner, the above problems can be solved, taking into account speed while ensuring the accuracy of the predicted intention.

[0118] In one example, the method further includes: fine-tuning the intent analysis model based on the fine-tuning sample to obtain a fine-tuned intent analysis model. The fine-tuning method may be LoRA (Low-Rank Adaptation of Large Language Models), and the specific fine-tuning process is not limited in this application.

[0119] The fine-tuning sample includes at least one of the following: multiple preset entries in the database; multiple general domain question-answer pairs. The number of the multiple general domain question-answer pairs can be a specified multiple of the multiple preset entries, and the specified multiple can be set according to actual conditions, which is not limited in this application, for example, it can be 5 times. The method for obtaining the general domain question-answer pairs is not limited in this application.

[0120] Since there are many proper nouns in the designated business field, and for the intent analysis model, some professional vocabulary and expressions are not easily triggered and expressed in the reasoning process of the intent analysis model, in order to solve this problem, multiple preset items in the database can be used as fine-tuning samples. In this way, the summary content of the intent analysis model can be closer to the preset items being retrieved to improve the retrieval effect. In addition, in order to make the fine-tuned intent analysis model more specific and general, general domain question and answer pairs of specified multiples of multiple preset items are used together with multiple preset items as fine-tuning samples to fine-tune the intent analysis model, and the method can also reduce the cost of fine-tuning (time cost, computing resources).

[0121] In one embodiment, based on the intention of the first object, determining multiple candidate items and reference multi-round dialogues associated with the multiple candidate items from a database includes: calculating the similarity between multiple preset items included in the database and the intention of the first object, wherein the similarity includes text similarity and / or vector similarity; based on the similarity between the multiple preset items and the intention of the first object, selecting multiple candidate items from the multiple preset items; and extracting reference multi-round dialogues associated with the multiple candidate items from the database.

[0122] Each of the plurality of preset items includes a preset question and a corresponding preset answer.

[0123] The text similarity may be calculated using a BM25 (Best Match 25) algorithm, and the vector similarity may be calculated using one of the following methods: cosine distance, Manhattan algorithm, and the like.

[0124] Optionally, the selecting of multiple candidate entries from the multiple preset entries based on the similarity between the multiple preset entries and the intention of the first object includes: when the similarity includes text similarity, the intelligent agent in the server sorts the text similarity of the multiple preset entries included in the database with the intention of the first object from high to low, and obtains a first sorting result of the multiple preset entries; based on the first sorting result, selecting the first first number of second entries from the multiple preset entries, and using the first first number of second entries as the multiple candidate entries, the first number can be a positive integer greater than or equal to 2 set according to actual conditions, and is not limited in this application.

[0125] Optionally, based on the similarity between the multiple preset entries and the intention of the first object, multiple candidate entries are selected from the multiple preset entries, including: when the similarity includes a text vector, the intelligent agent in the server sorts the vector similarities of the multiple preset entries included in the database with the intention of the first object from high to low, and obtains a second sorting result of the multiple preset entries; based on the second sorting result, the first second number of third entries are selected from the multiple preset entries, and the first second number of third entries are used as the multiple candidate entries. The second number can be a positive integer greater than or equal to 2 set according to actual conditions, and is not limited in this application.

[0126] Optionally, the selecting multiple candidate entries from the multiple preset entries based on the similarity between the multiple preset entries and the intention of the first object includes: when the similarity includes text similarity and vector similarity, the intelligent agent in the server sorts the text similarity of the multiple preset entries included in the database with the intention of the first object from high to low, and obtains a first sorting result of the multiple preset entries; based on the first sorting result, selecting the first first number of second entries from the multiple preset entries; sorting the vector similarity of the intention of the first object from the multiple preset entries included in the database from high to low, and obtains a second sorting result of the multiple preset entries; based on the second sorting result, selecting the first second number of third entries from the multiple preset entries; and taking the union of the first first number of second entries and the second number of third entries as the multiple candidate entries.

[0127] In one example, before selecting multiple candidate items from the multiple preset items based on the similarity between the multiple preset items and the intention of the first object, the method further includes: the agent in the server obtains the target service type corresponding to the first object. The target service type corresponding to the first object may be the service type selected by the first object by pressing a button when the first object uses the first device to make a call, wherein the button is a physical button or a virtual button; or the target service type corresponding to the first object may be given by the second device or defaulted, etc., and the method for obtaining or determining the target service type corresponding to the first object is not limited here.

[0128] The selecting of multiple candidate entries from the multiple preset entries based on the similarity between the multiple preset entries and the intention of the first object comprises: the intelligent agent in the server selects multiple preset entries corresponding to the target business type from all preset entries in the database, and selects multiple candidate entries from the multiple preset entries corresponding to the target business type based on the similarity between each preset entry in the multiple preset entries corresponding to the target business type and the intention of the first object. The processing method of selecting multiple candidate entries from the multiple preset entries corresponding to the target business type based on the similarity between each preset entry in the multiple preset entries corresponding to the target business type and the intention of the first object is the same as the processing method of selecting multiple candidate entries from the multiple preset entries based on the similarity between each preset entry and the intention of the first object, and will not be repeated here.

[0129] In this way, multiple candidate items are selected from the multiple preset items according to the similarity between each preset item in the multiple preset items included in the database and the intention of the first object, and the reference multi-round dialogues associated with the multiple candidate items are extracted from the database. In this way, multiple candidate items similar to the intention of the first object and the reference multi-round dialogues associated therewith can be determined more accurately.

[0130] In one embodiment, the method of determining one or more reference items from the multiple candidate items based on the intention of the first object and the reference multi-round conversations associated with the multiple candidate items includes: determining initial ranking scores of the multiple candidate items based on the similarity between the intention of the first object and the multiple candidate items; adjusting the initial ranking scores of the multiple candidate items to obtain target ranking scores of the multiple candidate items based on the similarity between the intention of the first object and the reference multi-round conversations associated with the multiple candidate items; and determining one or more reference items from the multiple candidate items based on the target ranking scores of the multiple candidate items.

[0131] Optionally, the initial ranking scores of the multiple candidate entries are determined based on the similarity between the intention of the first object and the multiple candidate entries, including: when the similarity is text similarity, the intelligent agent in the server uses the text similarity between each candidate entry in the multiple candidate entries and the intention of the first object as the initial ranking score of each candidate entry.

[0132] Optionally, the initial ranking scores of the multiple candidate entries are determined based on the similarity between the intention of the first object and the multiple candidate entries, including: when the similarity is vector similarity, the intelligent agent in the server uses the vector similarity between each candidate entry in the multiple candidate entries and the intention of the first object as the initial ranking score of each candidate entry.

[0133] Optionally, the initial ranking scores of the multiple candidate entries are determined based on the similarity between the intention of the first object and the multiple candidate entries, including: when the similarity includes text similarity and vector similarity, the intelligent agent in the server determines the initial ranking score of each candidate entry based on the text similarity and / or vector similarity between each candidate entry in the multiple candidate entries and the intention of the first object.

[0134] In combination with the foregoing embodiments, when the similarity includes text similarity and vector similarity, multiple candidate entries may include a first number of second entries selected based on the highest text similarity, a second number of third entries selected based on the highest vector similarity, any second entry may be the same as or different from one of the second number of third entries, and any third entry may be the same as or different from one of the first number of second entries. Therefore, the composition of the multiple candidate entries may include at least one of the following situations: some candidate entries only have corresponding text similarities, some candidate entries only have corresponding vector similarities, and some candidate entries have both text similarity and vector similarity.

[0135] Taking the kth candidate entry among multiple candidate entries as an example, the determination of the initial ranking score of each candidate entry based on the text similarity and / or vector similarity between each candidate entry among the multiple candidate entries and the intention of the first object may include one of the following: when the kth candidate entry has the corresponding vector similarity and the text similarity, the agent in the server determines the initial ranking score of the kth candidate entry based on the vector similarity and the text similarity corresponding to the kth candidate entry; when the kth candidate entry only has the corresponding vector similarity, the agent in the server uses the vector similarity corresponding to the kth candidate entry as the initial ranking score of the kth candidate entry; when the kth candidate entry only has the corresponding text similarity, the text similarity corresponding to the kth candidate entry is used as the initial ranking score of the kth candidate entry. Wherein, k is a positive integer greater than or equal to 1.

[0136] Determining the initial ranking score of the kth candidate entry based on the vector similarity and the text similarity corresponding to the kth candidate entry may include one of the following: the intelligent agent in the server takes the average value of the vector similarity and the text similarity corresponding to the kth candidate entry as the initial ranking score of the kth candidate entry; the intelligent agent in the server takes the maximum value of the vector similarity and the text similarity corresponding to the kth candidate entry as the initial ranking score of the kth candidate entry.

[0137] In one example, the initial ranking scores of the multiple candidate entries are adjusted based on the similarity between the intention of the first object and the reference multi-round conversations associated with the multiple candidate entries to obtain the target ranking scores of the multiple candidate entries, including: determining adjustment parameters corresponding to the multiple candidate entries based on the similarity between the intention of the first object and the reference multi-round conversations associated with the multiple candidate entries, weight values ​​of the reference multi-round conversations associated with the multiple candidate entries, and influence factors of the reference multi-round conversations associated with the multiple candidate entries on the multiple candidate entries; and adjusting the initial ranking scores of the multiple candidate entries based on the adjustment parameters corresponding to the multiple candidate entries to obtain the target ranking scores of the multiple candidate entries.

[0138] The number of reference multi-round conversations associated with each of the multiple candidate entries may be one or more. The similarity between the intention of the first object and the reference multi-round conversations associated with the multiple candidate entries may be: the similarity between the vector of the intention of the first object and each vector of the reference multi-round conversations in one or more reference multi-round conversations associated with each of the multiple candidate entries.

[0139] The method for determining the weight value of the reference multi-round dialogue associated with the multiple candidate entries includes: the intelligent agent in the server determines the weight value of the reference multi-round dialogue associated with each candidate entry in the multiple candidate entries based on the specified features of each reference multi-round dialogue in one or more reference multi-round dialogues associated with each candidate entry in the multiple candidate entries. Among them, the specified features of any reference multi-round dialogue may include at least one of the following: the length of any reference multi-round dialogue, the number of key information contained in any reference multi-round dialogue, and the key information may be the relevant information in the candidate entry. The specific calculation method is not limited in this application, and it can be understood that the longer the length of any reference multi-round dialogue, the greater its weight, and the more key information contained in any reference multi-round dialogue, the greater its weight.

[0140] The influence factor of the reference multi-round dialogue associated with each candidate entry among the multiple candidate entries on the each candidate entry is used to indicate the degree of association between the reference multi-round dialogue associated with each candidate entry and the each candidate entry, which can be set according to actual conditions and is not limited in this application.

[0141] Taking the kth candidate entry among multiple candidate entries as an example, determining the adjustment parameters corresponding to the multiple candidate entries based on the similarity between the intention of the first object and the reference multi-round conversations associated with the multiple candidate entries, the weight values ​​of the reference multi-round conversations associated with the multiple candidate entries, and the influence factors of the reference multi-round conversations associated with the multiple candidate entries on the multiple candidate entries may include: the intelligent agent in the server obtains one or more first values ​​based on the similarity between the intention of the first object and one or more reference multi-round conversations associated with the kth candidate entry, the weight value of each reference multi-round conversation in one or more reference multi-round conversations associated with the kth candidate entry, and the influence factor of each reference multi-round conversation associated with the kth candidate entry on the kth candidate entry; and determining the adjustment parameter corresponding to the kth candidate entry based on the one or more first values ​​and the first adjustment weight.

[0142] In the case that there is one reference multi-round dialogue associated with the k-th candidate entry, one or more first values ​​are obtained based on the similarity between the intention of the first object and one or more reference multi-round dialogues associated with the k-th candidate entry, the weight value of each reference multi-round dialogue in the one or more reference multi-round dialogues associated with the k-th candidate entry, and the influence factor of each reference multi-round dialogue associated with the k-th candidate entry on the k-th candidate entry, including: the intelligent agent in the server multiplies the similarity between the intention of the first object and the reference multi-round dialogue associated with the k-th candidate entry, the weight value of the reference multi-round dialogue associated with the k-th candidate entry, and the influence factor of the reference multi-round dialogue associated with the k-th candidate entry to obtain the first value.

[0143] Determining the adjustment parameter corresponding to the kth candidate entry based on one or more first values ​​and the first adjustment weight includes: the agent in the server multiplies the first value and the first adjustment weight to obtain the adjustment parameter corresponding to the kth candidate entry.

[0144] In the case where there are multiple reference multi-round conversations associated with the kth candidate entry, taking any one of the reference multi-round conversations as the jth reference multi-round conversation as an example, one or more first values ​​are obtained based on the similarity between the intention of the first object and one or more reference multi-round conversations associated with the kth candidate entry, the weight value of each reference multi-round conversation in the one or more reference multi-round conversations associated with the kth candidate entry, and the influence factor of each reference multi-round conversation associated with the kth candidate entry on the kth candidate entry, including: the intelligent agent in the server multiplies the similarity between the intention of the first object and the jth reference multi-round conversation associated with the kth candidate entry, the weight value of the jth reference multi-round conversation associated with the kth candidate entry, and the influence factor of the jth reference multi-round conversation associated with the kth candidate entry on the kth candidate entry to obtain the jth first value.

[0145] The method of determining the adjustment parameter corresponding to the kth candidate entry based on one or more first values ​​and a first adjustment weight includes: the agent in the server sums the multiple first values ​​to obtain a second value; and multiplying the second value and the first adjustment weight to obtain the adjustment parameter corresponding to the kth candidate entry.

[0146] Still taking the kth candidate entry among multiple candidate entries as an example, the initial ranking scores of the multiple candidate entries are adjusted based on the adjustment parameters corresponding to the multiple candidate entries to obtain the target ranking scores of the multiple candidate entries, including: the intelligent agent in the server multiplies the initial ranking score of the kth candidate entry by the second adjustment weight to obtain the intermediate ranking score of the kth candidate entry; and adds the intermediate ranking score of the kth candidate entry and the adjustment parameter corresponding to the kth candidate entry to obtain the target ranking score of the kth candidate entry.

[0147] Among them, the first adjustment weight and the second adjustment weight are related, and are both used to adjust the degree of influence of the reference multi-round dialogue on the initial ranking score. Among them, the second adjustment weight can be equal to 1 minus the first adjustment weight, and the value of the first adjustment weight can be set according to the actual situation, as long as it is within the range of 0 to 1 (including 0 and 1). For example: Assume that the first adjustment weight is expressed as λ, when λ=0, the second adjustment weight is equal to 1, and at this time, the ranking is completely based on the initial ranking score (that is, the initial ranking score is used as the target ranking score), and the influence of the reference multi-round dialogue is not considered. When λ=1, the second adjustment weight is equal to 0, and at this time, the target ranking score is determined completely based on the similarity between the reference multi-round dialogue and the intention of the first object, and the initial ranking score is ignored. When 0<λ<1, the first adjustment weight and the second adjustment weight are both greater than 0 and less than 1. At this time, both the similarity between the reference multi-round dialogue and the intention of the first object and the initial ranking score are considered.

[0148] Taking the kth candidate entry among the multiple candidate entries as an example, the initial ranking scores of the multiple candidate entries are adjusted based on the adjustment parameters corresponding to the multiple candidate entries to obtain the target ranking scores of the multiple candidate entries, which is described in combination with the formula:

[0149]

[0150] Among them, λ is the first adjustment weight, (1-λ) is the second adjustment weight, b k is the kth candidate entry among multiple candidate entries; OriginalScore(b k )(Original score(b k )) is the initial ranking score of the kth candidate entry, d j k is the jth reference multi-round dialogue associated with the kth candidate entry (the value of j ranges from 1 to m); S(d j k ) is the similarity between the first object's intention and the j-th reference multi-round dialogue associated with the k-th candidate item; W(d j k ) is the weight value of the jth reference multi-round dialogue corresponding to the kth candidate item; Influence(b k ,d j k )(Influence(b k ,d j k )) is the influence factor of the j-th reference multi-round dialogue corresponding to the k-th candidate item on the k-th candidate item; Score(b k )(score(b k )) is the target ranking score of the kth candidate entry.

[0151] Determining one or more reference items from the multiple candidate items based on the target ranking scores of the multiple candidate items may include: the agent in the server sorts the target ranking scores of the multiple candidate items from high to low to obtain a ranking result of the multiple candidate items; and taking the first H candidate items in the ranking result of the multiple candidate items as one or more reference items, where H may be a positive integer greater than or equal to 1 set according to actual conditions. For example, the first three items in the target ranking scores of the multiple candidate items may be selected as three reference items.

[0152] In this way, the reference multi-round conversation can represent the relevant scenarios of the associated candidate items, so the initial ranking score is adjusted according to the intention of the first object and the similarity of the reference multi-round conversations associated with multiple candidate items. In addition, the parameters related to the reference multi-round conversation are also combined in the adjustment process, so that the adjusted target ranking score can be more accurate.

[0153] In one embodiment, it also includes: the agent in the server inputs the current multi-round dialogue, the intention of the first object, and one or more reference items into the enhanced retrieval model, obtains the reference answer to the question output by the enhanced retrieval model, and sends the reference answer to the second device. The enhanced retrieval model can be obtained by training the third preset model, and the specific training method is not limited in this application. The third preset model can be RAG (Retrieval Augmented Generation).

[0154] Combination Figure 2 The above embodiments are described, including:

[0155] S201, the second device sends the current multi-round dialogue between the first object and the second object to the server.

[0156] The agent side in the server performs the processing from S202 to S210, which is as follows:

[0157] S202: receiving the current multi-round dialogue between the first object and the second object.

[0158] S203: Determine multiple rounds of dialogue to be analyzed based on the current multiple rounds of dialogue.

[0159] S204: Input the multiple rounds of dialogue to be analyzed into a keyword extraction model to obtain target keywords output by the keyword extraction model.

[0160] S205: using the multiple rounds of dialogue to be analyzed and the one or more target reference questions as the input information; or using the multiple rounds of dialogue to be analyzed as the input information. The one or more target reference questions are target reference questions that match the target keyword in multiple preset items included in the database.

[0161] S206: Input the input information into an intention analysis model to obtain analysis results corresponding to the multiple rounds of conversations to be analyzed output by the intention analysis model, wherein the analysis results include the intention of the first object.

[0162] S207: Extract the intention of the first object from the analysis result.

[0163] S208, obtaining the target service type corresponding to the first object from the second device. It can be understood that as long as the service type corresponding to the first object is obtained before S209, it is within the scope of this application.

[0164] S209, selecting multiple preset entries corresponding to the target business type from all preset entries in the database, and selecting multiple candidate entries from the multiple preset entries corresponding to the target business type based on the similarity between the multiple preset entries corresponding to the target business type and the intention of the first object; determining one or more reference entries from the multiple candidate entries based on the intention of the first object and a reference multi-round dialogue associated with the multiple candidate entries; using the one or more reference entries as reference recommendation information, and sending the reference recommendation information to the second device.

[0165] S210, optionally, inputting the current multi-round conversation, the intention of the first object, and one or more reference items into an enhanced retrieval model, obtaining a reference answer to the question output by the enhanced retrieval model, and sending the reference answer to the question to the second device.

[0166] Figure 3 FIG. 1 is a schematic block diagram of an information recommendation device provided by an embodiment of the present disclosure. Figure 3 As shown, including:

[0167] The intention analysis module 301 is used to determine the intention of the first object based on the current multi-round dialogue in response to receiving the current multi-round dialogue between the first object and the second object, wherein the first object is a service recipient and the second object is a service provider;

[0168] A retrieval module 302, configured to determine, from a database, a plurality of candidate items and a plurality of reference rounds of dialogue associated with the plurality of candidate items based on the intention of the first object;

[0169] A screening module 303, configured to determine one or more reference items from the multiple candidate items based on the intention of the first object and the reference multiple rounds of dialogue associated with the multiple candidate items;

[0170] The reference recommendation information sending module 304 is configured to use the one or more reference items as reference recommendation information and send the reference recommendation information.

[0171] The screening module is used to determine the initial ranking scores of the multiple candidate entries based on the similarity between the intention of the first object and the multiple candidate entries; adjust the initial ranking scores of the multiple candidate entries based on the similarity between the intention of the first object and the reference multi-round dialogues associated with the multiple candidate entries to obtain the target ranking scores of the multiple candidate entries; and determine one or more reference entries from the multiple candidate entries based on the target ranking scores of the multiple candidate entries.

[0172] The screening module is used to determine the adjustment parameters corresponding to the multiple candidate entries based on the similarity between the intention of the first object and the reference multi-round conversations associated with the multiple candidate entries, the weight values ​​of the reference multi-round conversations associated with the multiple candidate entries, and the influence factors of the reference multi-round conversations associated with the multiple candidate entries on the multiple candidate entries; based on the adjustment parameters corresponding to the multiple candidate entries, adjust the initial ranking scores of the multiple candidate entries to obtain the target ranking scores of the multiple candidate entries.

[0173] The retrieval module is used to calculate the similarity between multiple preset items included in the database and the intention of the first object, wherein the similarity includes text similarity and / or vector similarity; based on the similarity between the multiple preset items and the intention of the first object, select multiple candidate items from the multiple preset items; and extract reference multi-round dialogues associated with the multiple candidate items from the database.

[0174] The intention analysis module is used to determine multiple rounds of conversations to be analyzed based on the current multiple rounds of conversations; generate input information based on the multiple rounds of conversations to be analyzed; input the input information into the intention analysis model to obtain analysis results corresponding to the multiple rounds of conversations to be analyzed output by the intention analysis model, wherein the analysis results include the intention of the first object.

[0175] The intention analysis module is used to input the multiple rounds of conversations to be analyzed into a keyword extraction model to obtain a target keyword output by the keyword extraction model; when there are one or more target reference questions matching the target keyword in the multiple preset entries contained in the database, the multiple rounds of conversations to be analyzed and the one or more target reference questions are used as the input information.

[0176] like Figure 4 The device further comprises:

[0177] The model training module 401 is used to input the training sample into the first preset model to obtain the prediction analysis result output by the first preset model, wherein the training sample includes multiple rounds of historical conversations between a third object and a fourth object, the third object is the service recipient, and the fourth object is the service provider, and the prediction analysis result includes the predicted intention of the third object; based on the prediction analysis result and multiple sample annotations corresponding to the training sample, a target loss is obtained, wherein the multiple sample annotations corresponding to the training sample are obtained based on multiple large models, the parameter amounts of the multiple large models are greater than the parameter amounts of the first preset model, and the parameter amounts of different large models in the multiple large models are different; the first preset model is trained based on the target loss to obtain the intention analysis model.

[0178] The model training module is used to calculate the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample; based on the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample, the target loss is obtained.

[0179] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0180] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0181] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0182] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0183] like Figure 5As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0184] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0185] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above. For example, in some embodiments, the above method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, at least one step of the method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the above method in any other appropriate manner (e.g., by means of firmware).

[0186] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0187] The program code for implementing the method of the present disclosure may be written in any combination of at least one programming language. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0188] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on at least one line, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0189] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0190] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0191] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0192] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0193] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An information recommendation method, comprising: In response to receiving a current multi-round dialogue between a first object and a second object, determining an intention of the first object based on the current multi-round dialogue, wherein the first object is a service recipient and the second object is a service provider; Based on the intention of the first object, determining a plurality of candidate entries and a reference multi-round dialogue associated with the plurality of candidate entries from a database; Determine one or more reference items from the multiple candidate items based on the intention of the first object and the reference multiple rounds of dialogue associated with the multiple candidate items; The one or more reference items are used as reference recommendation information, and the reference recommendation information is sent.

2. The method according to claim 1, wherein: The determining one or more reference items from the multiple candidate items based on the intention of the first object and the reference multiple rounds of dialogue associated with the multiple candidate items includes: Determining initial ranking scores of the plurality of candidate entries based on similarities between the intent of the first object and the plurality of candidate entries; Based on the similarity between the intention of the first object and the reference multi-round dialogues associated with the multiple candidate entries, adjusting the initial ranking scores of the multiple candidate entries to obtain target ranking scores of the multiple candidate entries; One or more reference items are determined from the plurality of candidate items based on the target ranking scores of the plurality of candidate items.

3. The method according to claim 2, wherein: The adjusting the initial ranking scores of the multiple candidate entries based on the similarity between the intention of the first object and the reference multi-round dialogues associated with the multiple candidate entries to obtain the target ranking scores of the multiple candidate entries includes: Determine adjustment parameters corresponding to the multiple candidate entries based on similarities between the intention of the first object and the reference multi-round conversations associated with the multiple candidate entries, weight values ​​of the reference multi-round conversations associated with the multiple candidate entries, and influence factors of the reference multi-round conversations associated with the multiple candidate entries on the multiple candidate entries; Based on the adjustment parameters corresponding to the multiple candidate entries, the initial ranking scores of the multiple candidate entries are adjusted to obtain the target ranking scores of the multiple candidate entries.

4. The method according to any one of claims 1 to 3, wherein: The determining, based on the intention of the first object, a plurality of candidate entries and a reference multi-round dialogue associated with the plurality of candidate entries from a database includes: Calculating similarities between a plurality of preset entries included in the database and the intention of the first object, wherein the similarities include text similarity and / or vector similarity; Selecting a plurality of candidate items from the plurality of preset items based on similarities between the plurality of preset items and the intention of the first object; A reference multi-turn dialogue associated with the plurality of candidate entries is extracted from the database.

5. The method according to claim 1, wherein: The determining the intention of the first object based on the current multi-round dialogue includes: Based on the current multiple rounds of dialogue, determining multiple rounds of dialogue to be analyzed; Generating input information based on the multiple rounds of dialogue to be analyzed; The input information is input into an intention analysis model to obtain analysis results corresponding to the multiple rounds of conversations to be analyzed output by the intention analysis model, wherein the analysis results include the intention of the first object.

6. The method according to claim 5, wherein: The generating input information based on the multiple rounds of dialogue to be analyzed includes: Inputting the multiple rounds of dialogue to be analyzed into a keyword extraction model to obtain target keywords output by the keyword extraction model; In the case that there are one or more target reference questions matching the target keyword in the multiple preset entries included in the database, the multiple rounds of dialogue to be analyzed and the one or more target reference questions are used as the input information.

7. The method according to claim 5 or 6, further comprising: Inputting a training sample into a first preset model to obtain a prediction analysis result output by the first preset model, wherein the training sample includes a historical multi-round conversation between a third object and a fourth object, the third object is a service recipient, the fourth object is a service provider, and the prediction analysis result includes a prediction intention of the third object; Based on the prediction analysis result and the multiple sample annotations corresponding to the training samples, a target loss is obtained, wherein the multiple sample annotations corresponding to the training samples are obtained based on multiple large models, the parameter amounts of the multiple large models are greater than the parameter amount of the first preset model, and the parameter amounts of different large models in the multiple large models are different; The first preset model is trained based on the target loss to obtain the intention analysis model.

8. The method according to claim 7, wherein: The obtaining of the target loss based on the prediction analysis result and the multiple sample annotations corresponding to the training sample includes: Calculating the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample; The target loss is obtained based on the probability distribution between the prediction analysis result and the training samples, and the probability distribution between the multiple sample annotations and the training samples.

9. An information recommendation device, comprising: an intention analysis module, configured to, in response to receiving a current multi-round dialogue between a first object and a second object, determine the intention of the first object based on the current multi-round dialogue, wherein the first object is a service recipient and the second object is a service provider; A retrieval module, configured to determine, from a database, a plurality of candidate entries and a plurality of reference rounds of dialogue associated with the plurality of candidate entries based on the intention of the first object; a screening module, configured to determine one or more reference items from the plurality of candidate items based on the intention of the first object and the reference multi-round dialogues associated with the plurality of candidate items; The reference recommendation information sending module is used to use the one or more reference items as reference recommendation information and send the reference recommendation information.

10. The device according to claim 9, wherein: The screening module is configured to determine an initial ranking score of the plurality of candidate entries based on the similarity between the intention of the first object and the plurality of candidate entries; Based on the similarity between the intention of the first object and the reference multi-round dialogues associated with the multiple candidate entries, adjusting the initial ranking scores of the multiple candidate entries to obtain target ranking scores of the multiple candidate entries; One or more reference items are determined from the plurality of candidate items based on the target ranking scores of the plurality of candidate items.

11. The device according to claim 10, wherein: The screening module is configured to determine adjustment parameters corresponding to the multiple candidate entries based on the similarity between the intention of the first object and the reference multi-round dialogues associated with the multiple candidate entries, the weight values ​​of the reference multi-round dialogues associated with the multiple candidate entries, and the influence factors of the reference multi-round dialogues associated with the multiple candidate entries on the multiple candidate entries; Based on the adjustment parameters corresponding to the multiple candidate entries, the initial ranking scores of the multiple candidate entries are adjusted to obtain the target ranking scores of the multiple candidate entries.

12. The device according to any one of claims 9 to 11, wherein: The retrieval module is used to calculate the similarity between multiple preset items included in the database and the intention of the first object, wherein the similarity includes text similarity and / or vector similarity; based on the similarity between the multiple preset items and the intention of the first object, select multiple candidate items from the multiple preset items; and extract reference multi-round dialogues associated with the multiple candidate items from the database.

13. The device according to claim 9, wherein: The intention analysis module is used to determine the multiple rounds of dialogue to be analyzed based on the current multiple rounds of dialogue; Based on the multiple rounds of conversations to be analyzed, input information is generated; the input information is input into an intention analysis model to obtain analysis results corresponding to the multiple rounds of conversations to be analyzed output by the intention analysis model, wherein the analysis results include the intention of the first object.

14. The device according to claim 13, wherein: The intention analysis module is used to input the multiple rounds of conversations to be analyzed into a keyword extraction model to obtain a target keyword output by the keyword extraction model; when there are one or more target reference questions matching the target keyword in the multiple preset entries contained in the database, the multiple rounds of conversations to be analyzed and the one or more target reference questions are used as the input information.

15. The device according to claim 13 or 14, further comprising: A model training module is used to input training samples into a first preset model to obtain a prediction and analysis result output by the first preset model, wherein the training samples include multiple rounds of historical conversations between a third object and a fourth object, the third object is a service recipient, and the fourth object is a service provider, and the prediction and analysis result includes the predicted intention of the third object; based on the prediction and analysis result and multiple sample annotations corresponding to the training samples, a target loss is obtained, wherein the multiple sample annotations corresponding to the training samples are obtained based on multiple large models, the parameter amounts of the multiple large models are greater than the parameter amounts of the first preset model, and the parameter amounts of different large models in the multiple large models are different; the first preset model is trained based on the target loss to obtain the intention analysis model.

16. The device according to claim 15, wherein: The model training module is used to calculate the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample; based on the probability distribution between the prediction analysis result and the training sample, and the probability distribution between the multiple sample annotations and the training sample, the target loss is obtained.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

19. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images