A method and system for sorting conversational questions based on self-determination of central sentences

Abstract

The present invention belongs to the field of natural language processing, and provides a method and system for sorting conversational questions based on self-determination of central sentences. The method includes obtaining a conversation context and a question library, using a trained question sorting model to obtain a question sorting score, selecting the question with the highest score and the central sentence identified when the question is used as the target question. In terms of model design, the present invention first identifies the central sentence that can ensure consistency and coherence, and then randomly deletes other sentences for fine-tuning; in terms of model training, an unsupervised learning strategy is used to optimize the identification of the central sentence. The present invention identifies the central sentence and uses it to guide the design of the fine-tuning task, thereby improving the consistency and coherence of the conversation context and the clarified questions during the fine-tuning process. The unsupervised training method for the central sentence prediction task reduces the manpower and material resources for data labeling, making the method widely applicable.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and in particular relates to a method and system for sorting conversational questions based on self-determination of central sentences. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Building conversational information seeking systems (CIS) has become a new trend and has garnered widespread attention in recent years. Compared to traditional information acquisition systems, such as search engines, a key feature of conversational information seeking systems is their hybrid drive. This involves proactively asking clarifying questions to help users clarify their intent when it's unclear.

[0004] Existing work on explicit question generation falls into two categories: conversational question generation (CQG) and conversational question ranking (CQR). Given a conversational context, conversational question generation generates explicit questions by generating each word in the question from beginning to end. This approach often generates very flexible and fluent questions. However, the quality of the questions generated by this method is often not guaranteed, and it is prone to generating aggressive and offensive questions.

[0005] Conversational question ranking selects an appropriate question from a predefined question library as a clarification question. Question selection is performed by scoring each question and selecting the highest-scoring question as the final answer. Because questions in the predefined question library are often manually assigned or selected, their quality is generally guaranteed. Therefore, this method is widely used in commercial conversational systems, such as Microsoft XiaoIce.

[0006] Existing work on conversational question ranking has achieved promising results by designing various fine-tuning strategies to improve model performance. However, these efforts often perform random operations on sentences in the conversation context during the fine-tuning process. For example, fine-tuning involves randomly deleting sentences from the conversation context. This disrupts the consistency and coherence between the conversation context and the explicit question, resulting in a decrease in model performance. Summary of the Invention

[0007] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a method and system for sorting conversational questions based on self-determination of central sentences, which can ensure the consistency and coherence of the conversation context and the clear questions.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] The first aspect of the present invention provides a method for sorting conversational questions based on self-determination of central sentences.

[0010] A method for sorting conversational questions based on self-determination of central sentences, comprising:

[0011] Obtain the conversation context and question library, use the trained question ranking model to obtain the question ranking score, select the question with the highest score and the central sentence identified when the question is used as the target question;

[0012] The training process of the question ranking model includes: using a training set and combining a loss function to train the question ranking model, obtaining the model with the best performance on the validation set as the original model, and the model with the second best performance as the shadow model; wherein both the training set and the validation set include: dialogue context, question library and question relevance labels; using a heuristic method and combining a given weight factor to calculate the score of each sentence in the dialogue context, selecting the sentence with the highest score in the dialogue context obtained by the original model as the identified central sentence, and selecting the sentence with the highest score in the dialogue context obtained by the shadow model as the identified another central sentence; based on the principle of making the identified central sentence and the identified another central sentence as similar as possible, optimizing the weight factor in combination with the loss function until the weight factor converges; based on the original model and the converged weight factor, using the training set, designing a fine-tuning task to train the original model to obtain a trained question ranking model.

[0013] Furthermore, before training the question ranking model, the process of building the question ranking model is included:

[0014] Based on the conversation context matrix and question base matrix, the BERT encoder is used to obtain the aggregated features of the conversation context matrix and question base matrix;

[0015] Based on the aggregated features, a fully connected layer is used to obtain a score for each question and a central sentence identified when the question is used as a target question.

[0016] Furthermore, the loss function includes a binary cross entropy loss function:

[0017]

[0018] Among them, y i ∈{0,1} is the question relevance label given in the dataset, f(U,Qi ) represents the question score, and the dialogue context U=[U1,U2,…,U l ], the question Q in the question bank = [Q1,Q2,…,Q i ].

[0019] Furthermore, the process of using a heuristic method, combined with a given weight factor, to calculate the score of each sentence in the conversation context, selecting the sentence with the highest score in the conversation context obtained by the original model as the identified central sentence, and selecting the sentence with the highest score in the conversation context obtained by the shadow model as the identified another central sentence specifically includes:

[0020] Based on the original model, a heuristic method is used, combined with a given weight factor, to calculate the score of each sentence in the conversation context, and the sentence with the highest score in the conversation context is selected as the central sentence;

[0021] Based on the shadow model, a heuristic method is used, combined with a given weight factor, to calculate the score of each sentence in the dialogue context, and the sentence with the highest score in the dialogue context is selected as the other central sentence to be identified.

[0022] Furthermore, the heuristic method includes:

[0023] Using the leave-one-out method, for the jth sentence in the dialogue context, by leaving out the sentence, calculate the sentence's response to the target question Q in the question ranking * Score:

[0024]

[0025] Among them, f θ (·,·) is the sorting model in step 1, with parameter θ; UU j Indicates leaving U j The context after

[0026] Furthermore, the heuristic method also includes: using the independence method, for the j-th sentence in the dialogue context, by individually measuring the effect of the sentence, calculating the sentence's effect on the target question Q in the question ranking * Score:

[0027]

[0028] Here the conversation context only contains the jth sentence, f θ (·,·) is the sorting model in step 1, with parameter θ;

[0029] Furthermore, the heuristic method also includes: using the context method, for the jth sentence in the dialogue context, by measuring the role of the sentence in the known context, calculating the sentence's effect on the target question Q in the question sorting * Score:

[0030]

[0031] Here U 1:j represents the first j sentences in the conversation context, f θ (·,·) is the ranking model in step 1, and the parameter is θ; since this method cannot measure the role of the first sentence U1 in the conversation context,

[0032] Furthermore, the heuristic method also includes: using the following method, for the jth sentence in the dialogue context, by measuring the role of the sentence in the known following situation, calculating the sentence's effect on the target question Q in the question sorting * Score:

[0033]

[0034] Here U j:l Indicates the jth sentence to the last sentence in the conversation context, f θ (·,·) is the ranking model in step 1, and the parameter is θ; since this method cannot measure the last sentence U in the conversation context l The effect of

[0035] Furthermore, the process until the weight factor converges includes:

[0036] For the original model, based on the given weight factor, the score of each sentence in the conversation context is calculated. The score for the j-th sentence is:

[0037]

[0038] And select the sentence with the highest score in the conversation context as the identified central sentence:

[0039]

[0040] Where c is the index of the identified central sentence and W is the weight factor; represents the score of sentence j calculated using the heuristic method;

[0041] For the shadow model, based on the given weight factor, another score is calculated for each sentence in the conversation context. The score for the j-th sentence is:

[0042]

[0043] And identify another central sentence:

[0044]

[0045] sc is the index of the identified another central sentence; Denote the another score of sentence j calculated by the heuristic method;

[0046] Taking the principle that the identified central sentence and the identified another central sentence are as identical as possible, optimize the weight factor W:

[0047]

[0048] Where N is the number of training samples, c n and sc n are the central sentence predictions of the original model f θ and the shadow model f φ for the nth sample, is the indicator function, which is 1 only when c n = c n and 0 otherwise;

[0049] Optimize the weight factor W using the binary cross-entropy loss function:

[0050]

[0051]

[0052] Where α is a coefficient to ensure the priority;

[0053] Optimize the weight factor W until the weight factor W converges.

[0054] Furthermore, the process of designing a fine-tuning task to train the original model using the training set according to the original model and the converged weight factor to obtain the trained question ranking model specifically includes:

[0055] Randomly delete the first k sentences in the conversation context of the training samples in the training set to construct new samples ( ′ ,); where k ∈ Possion(λ), and 1 ≤ k < c, c is the index of the identified central sentence, U ′ = [U k+1 ,…, c ,…, l , λ is a hyperparameter; and the constructed new samples contain the identified central sentence;

[0056] Use the new samples and combine with the binary cross-entropy loss function to train the original model; ]>

[0057]

[0058] Among them, y i ∈{0,1} is the question relevance label given in the dataset, f(U′,Q i ) indicates the question score;

[0059] Get the trained question ranking model.

[0060] A second aspect of the present invention provides a conversational question ranking system based on self-determination of central sentences.

[0061] A conversational question ranking system based on self-determination of central sentences, comprising:

[0062] A ranking module is configured to: obtain the conversation context and question library, use the trained question ranking model to obtain question ranking scores, select the question with the highest score, and the central sentence identified when the question is used as the target question;

[0063] The question ranking model training module is configured as follows: using the training set and combining the loss function to train the question ranking model, obtaining the model with the best performance on the validation set as the original model, and the model with the second best performance as the shadow model; wherein both the training set and the validation set include: dialogue context, question library and question relevance label; using a heuristic method and combining the given weight factor to calculate the score of each sentence in the dialogue context, selecting the sentence with the highest score in the dialogue context obtained by the original model as the identified central sentence, and selecting the sentence with the highest score in the dialogue context obtained by the shadow model as the identified another central sentence; based on the principle that the identified central sentence and the identified another central sentence are as similar as possible, combining the loss function, optimizing the weight factor until the weight factor converges; according to the original model and the converged weight factor, using the training set, designing a fine-tuning task to train the original model, and obtaining a trained question ranking model.

[0064] A third aspect of the present invention provides a computer-readable storage medium.

[0065] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for sorting conversational questions based on self-determination of central sentences as described in the first aspect above.

[0066] A fourth aspect of the present invention provides a computer device.

[0067] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for sorting conversational questions based on self-determination of central sentences as described in the first aspect above are implemented.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] In model design, this invention first identifies the central sentence that ensures consistency and coherence, and then randomly deletes other sentences for fine-tuning. In model training, an unsupervised learning strategy is used to optimize the recognition of the central sentence. Compared with existing conversational question ranking methods, this invention has the following advantages:

[0070] 1. This paper proposes to identify the central sentence in the conversation context, which can ensure the consistency and coherence between the conversation context and the question during fine-tuning, thereby improving model performance.

[0071] 2. The present invention designs an unsupervised training strategy for central sentence recognition, which can avoid the consumption of manpower and material resources for data labeling, reduce the model's dependence on data labeling, and improve the application ability of the method.

[0072] 3. The model designed by the present invention can identify the central sentence as a reference for question ranking prediction, providing explainability for the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0074] Figure 1 It is a flow chart of the method for sorting conversational questions based on self-determination of the central sentence of the present invention;

[0075] Figure 2 It is a schematic diagram of the present invention in optimizing weight factors and identifying central sentences;

[0076] Figure 3 It is a schematic diagram of the present invention when applied in a model. DETAILED DESCRIPTION

[0077] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0078] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0079] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0080] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to the various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code can include one or more executable instructions for implementing the logical functions specified in the various embodiments. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of the boxes in the flowchart and / or block diagram, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0081] Example 1

[0082] like Figure 1 As shown, this embodiment provides a method for sorting conversational questions based on self-determination of the central sentence. This embodiment uses the method applied to the server as an example for illustration. It is understandable that the method can also be applied to the terminal, and can also be applied to a system including the terminal, the server, and the terminal, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0083] Obtain the conversation context and question library, use the trained question ranking model to obtain the question ranking score, select the question with the highest score and the central sentence identified when the question is used as the target question;

[0084] The training process of the question ranking model includes: using a training set and combining a loss function to train the question ranking model, obtaining the model with the best performance on the validation set as the original model, and the model with the second best performance as the shadow model; wherein both the training set and the validation set include: dialogue context, question library and question relevance labels; using a heuristic method and combining a given weight factor to calculate the score of each sentence in the dialogue context, selecting the sentence with the highest score in the dialogue context obtained by the original model as the identified central sentence, and selecting the sentence with the highest score in the dialogue context obtained by the shadow model as the identified another central sentence; based on the principle of making the identified central sentence and the identified another central sentence as similar as possible, optimizing the weight factor in combination with the loss function until the weight factor converges; based on the original model and the converged weight factor, using the training set, designing a fine-tuning task to train the original model to obtain a trained question ranking model.

[0085] In order to ensure the consistency and coherence between the conversation context and the clarified questions, the present invention proposes a new fine-tuning scheme. In terms of model design, the fine-tuning is performed by first identifying the central sentence that can ensure consistency and coherence, and then randomly deleting other sentences; in terms of model training, an unsupervised learning strategy is adopted to optimize the identification of the central sentence. Compared with existing methods, the central sentence is identified and used to guide the design of the fine-tuning task, which improves the consistency and coherence between the conversation context and the clarified questions during the fine-tuning process. The unsupervised training method for the central sentence prediction task reduces the manpower and material resources for data labeling, making this method widely used. At the same time, the identified central sentence can be used as a reference for question sorting, providing explainability for the model. The present invention is described in detail below, based on the conversational question sorting determined by the central sentence, such as Figure 1 As shown, it specifically includes the following steps:

[0086] Step 1: Build a model for question sorting. Given a conversation context U = [U1, U2, ..., U l ], the question Q in the question bank = [Q1,Q2,…,Q m ], the task of the model is to predict a score for each question in the question library and select the one with the highest score as the output.

[0087] For the i-th question Q i , the model will U and Q i Spliced together and decomposed into a series of words, a BERT encoder is used to obtain the aggregate representation of U and Q:

[0088] h i=BERT(U,Q i ),

[0089] Then, through a fully connected layer Linear, predict the score of the question:

[0090] f(U,Q i )=Linear(h i ).

[0091] In this way, the model can predict a score for each question, thus having the ability to rank questions.

[0092] Step 2: Identify the central sentence in the conversation context. Based on the model in step 1, an unsupervised approach is used to identify the central sentence in the conversation context. Figure 2 A schematic diagram showing the present invention in weight factor optimization and central sentence identification.

[0093] This step is achieved through the following process.

[0094] Step 1: Train the conversational question ranking model. Use a public conversational question ranking dataset and input the conversation context, question library, and question relevance labels, using binary cross entropy loss for training:

[0095]

[0096] Among them, y i ∈{0,1} is the question relevance label given in the dataset. During the training process, the model with the best performance on the validation set is used as the original model f θ , the suboptimal model is used as the shadow model f φ .

[0097] Step 2: Identify the central sentence.

[0098] For the original model f θ , a score is calculated for each sentence in the conversation context using the following four parameter-free heuristic methods:

[0099] 1) Leave-one-out method: For the j-th sentence in the dialogue context, leave out the sentence and calculate the sentence's response to the target question Q in the question ranking. * Score:

[0100]

[0101] Among them, f θ (·,·) is the sorting model in step 1, with parameters θ, UU j Indicates leaving U j The context after.

[0102] 2) Independent method: For the jth sentence in the dialogue context, the effect of the sentence is measured separately and the effect of the sentence on the target question Q in the question ranking is calculated. * Score:

[0103]

[0104] Here the conversation context only contains the j-th sentence.

[0105] 3) Grammar method: For the jth sentence in the conversation context, by measuring the role of the sentence in the known context, calculate the sentence's role in the target question Q in the question ranking * Score:

[0106]

[0107] Here U 1: Represents the first j sentences in the conversation context. Since this method cannot measure the role of the first sentence U1 in the conversation context,

[0108] 4) Context method: For the jth sentence in the dialogue context, by measuring the role of the sentence in the known context, calculate the sentence's role in the target question Q in the question ranking * Score:

[0109]

[0110] Here U j:l Represents the jth sentence to the last sentence in the conversation context. Since this method cannot measure the last sentence U in the conversation context l The effect of

[0111] Finally, the score of each sentence in the conversation context is calculated through a trainable parameter W. Specifically, for the j-th sentence:

[0112]

[0113] And select the sentence with the highest score in the conversation context as the identified central sentence:

[0114]

[0115] Where c is the index of the identified central sentence.

[0116] Step 3: Optimize the weight factor in Step 2. Based on the shadow model f φ , calculate another score for each sentence in the conversation context in the same way as in step 2:

[0117]

[0118] And identify another central sentence:

[0119]

[0120] By promoting the original model f θ and the shadow model f φ to make the obtained central sentences as identical as possible, optimize the weight factor W:

[0121]

[0122] where N is the number of training samples, c n and sc n are the central sentence predictions of the original model f θ and the shadow model f φ for the nth sample, is the indicator function, which is 1 only when c n = c n and 0 otherwise.

[0123] Meanwhile, we also use binary cross-entropy loss to optimize the weight factor W to make it refer to the scores calculated by all four methods as much as possible:

[0124]

[0125] Since is not differentiable, we use the sequential model-based global optimization (SMBO) algorithm provided by the hyperopt library to solve it:

[0126]

[0127] We set α = 0.001 to ensure that has absolute priority.

[0128] Finally, after the optimization of the weight factor W converges, for the original model, we identify the central sentences in the conversation context of each sample based on the method in step 2.

[0129] Step 3: Design a fine-tuning task and train a question ranking model.

[0130] Design a central sentence retention fine-tuning task. Randomly delete the first k sentences in the conversation context to construct new samples ( ′ ,). Among them, k ∈ Possion(λ), and 1 ≤ k < c, where c is the index of the central sentence identified in step 3, U ′=[U k+1 ,…, c ,…, l ], λ is a hyperparameter, which we set to 1.0. In this way, the constructed new sample must contain the recognized central sentence, thus ensuring the consistency between the conversation context and the target question.

[0131] Input new samples and use the same binary cross entropy loss as in step 1 to train the problem sorting model:

[0132]

[0133] Step 4: Actual conversational information acquisition application.

[0134] After training is complete, the weight factors and model parameters are fixed, and the model can then be applied to actual conversational information acquisition scenarios.

[0135] like Figure 3 As shown in the figure, given a conversation context and a question library, the conversation context and candidate questions are fed into the model one by one to predict a score. The question with the highest score is selected as the final output of the model. The central sentence identified when this question is used as the target question is also output as a reference for the model's prediction.

[0136] Example 2

[0137] This embodiment provides a conversational question ranking system based on self-determination of central sentences.

[0138] A conversational question ranking system based on self-determination of central sentences, comprising:

[0139] A ranking module is configured to: obtain the conversation context and question library, use the trained question ranking model to obtain question ranking scores, select the question with the highest score, and the central sentence identified when the question is used as the target question;

[0140] The question ranking model training module is configured as follows: using the training set and combining the loss function to train the question ranking model, obtaining the model with the best performance on the validation set as the original model, and the model with the second best performance as the shadow model; wherein both the training set and the validation set include: dialogue context, question library and question relevance label; using a heuristic method and combining the given weight factor to calculate the score of each sentence in the dialogue context, selecting the sentence with the highest score in the dialogue context obtained by the original model as the identified central sentence, and selecting the sentence with the highest score in the dialogue context obtained by the shadow model as the identified another central sentence; based on the principle that the identified central sentence and the identified another central sentence are as similar as possible, combining the loss function, optimizing the weight factor until the weight factor converges; according to the original model and the converged weight factor, using the training set, designing a fine-tuning task to train the original model, and obtaining a trained question ranking model.

[0141] It should be noted that the examples and application scenarios implemented by the above-mentioned ranking module and question ranking model training module are the same as those in the steps of Example 1, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0142] Example 3

[0143] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method for sorting conversational questions based on self-determination of a central sentence as described in the first embodiment above are implemented.

[0144] Example 4

[0145] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for sorting conversational questions based on self-determination of a central sentence as described in the first embodiment are implemented.

[0146] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0147] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0148] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0150] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0151] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for sorting conversational questions based on self-determination of central sentences, characterized by: include: Obtain the conversation context and question library, use the trained question ranking model to obtain the question ranking score, select the question with the highest score and the central sentence identified when the question is used as the target question; The training process of the question ranking model includes: using a training set and combining a loss function to train the question ranking model, obtaining the model with the best performance on the validation set as the original model, and the model with the second best performance as the shadow model; wherein both the training set and the validation set include: dialogue context, question library and question relevance labels; using a heuristic method and combining a given weight factor to calculate the score of each sentence in the dialogue context, selecting the sentence with the highest score in the dialogue context obtained by the original model as the identified central sentence, and selecting the sentence with the highest score in the dialogue context obtained by the shadow model as the identified another central sentence; based on the principle of making the identified central sentence and the identified another central sentence as similar as possible, optimizing the weight factor in combination with the loss function until the weight factor converges; based on the original model and the converged weight factor, using the training set, designing a fine-tuning task to train the original model to obtain a trained question ranking model.

2. The method for sorting conversational questions based on self-determination of central sentences according to claim 1 is characterized in that: Before training the question ranking model, the process of building the question ranking model includes: Based on the conversation context matrix and question base matrix, the BERT encoder is used to obtain the aggregated features of the conversation context matrix and question base matrix; Based on the aggregated features, a fully connected layer is used to obtain a score for each question and a central sentence identified when the question is used as a target question.

3. The method for sorting conversational questions based on self-determination of central sentences according to claim 1 is characterized in that: The loss function includes a binary cross entropy loss function: Among them, t i ∈{0,1} is the question relevance label given in the dataset, f(U,Q i ) represents the question score, and the dialogue context U=[U1,U2,…,U l ], the question Q in the question bank = [Q1,Q2,…,Q i ].

4. The method for sorting conversational questions based on self-determination of central sentences according to claim 1 is characterized in that: The process of using a heuristic method and a given weight factor to calculate the score of each sentence in the conversation context, selecting the sentence with the highest score in the conversation context obtained by the original model as the central sentence to be identified, and selecting the sentence with the highest score in the conversation context obtained by the shadow model as the other central sentence to be identified specifically includes: Based on the original model, a heuristic method is used, combined with a given weight factor, to calculate the score of each sentence in the conversation context, and the sentence with the highest score in the conversation context is selected as the central sentence; Based on the shadow model, a heuristic method is used, combined with a given weight factor, to calculate the score of each sentence in the dialogue context, and the sentence with the highest score in the dialogue context is selected as the other central sentence to be identified.

5. The method for sorting conversational questions based on self-determination of central sentences according to claim 1 is characterized in that: The heuristics include: Using the leave-one-out method, for the jth sentence in the dialogue context, by leaving out the sentence, calculate the sentence's response to the target question Q in the question ranking * Score: Among them, f θ (·,·) is the sorting model in step 1, with parameter θ; UU j Indicates leaving U j The context after Alternatively, the heuristic method further includes: using an independent method, for the jth sentence in the dialogue context, by individually measuring the effect of the sentence, calculating the sentence's effect on the target question Q in the question ranking * Score: Here the conversation context only contains the jth sentence, f θ (·,·) is the sorting model in step 1, with parameter θ; Alternatively, the heuristic method further includes: using the above context method, for the jth sentence in the dialogue context, by measuring the role of the sentence in the known above context, calculating the sentence's effect on the target question Q in the question sorting * Score: Here U 1: represents the first j sentences in the conversation context, f θ (·,·) is the ranking model in step 1, and the parameter is θ; since this method cannot measure the role of the first sentence U1 in the conversation context, Alternatively, the heuristic method further includes: using the following method, for the jth sentence in the dialogue context, by measuring the effect of the sentence under the known following situation, calculating the sentence's effect on the target question Q in the question sorting * Score: Here U j:l Indicates the jth sentence to the last sentence in the conversation context, f θ (·,·) is the ranking model in step 1, and the parameter is θ; since this method cannot measure the last sentence U in the conversation context l The effect of 6. The method for sorting conversational questions based on self-determination of central sentences according to claim 1 is characterized in that: The process until the weight factor converges includes: For the original model, based on the given weight factor, the score of each sentence in the conversation context is calculated. The score for the j-th sentence is: And select the sentence with the highest score in the conversation context as the identified central sentence: Where c is the index of the identified central sentence and W is the weight factor; represents the score of sentence j calculated using the heuristic method; For the shadow model, based on the given weight factor, another score is calculated for each sentence in the conversation context. The score for the j-th sentence is: And identify another central sentence: sc is the index of another identified central sentence; represents another score of sentence j calculated using the heuristic method; Based on the principle that identifying the central sentence and identifying another central sentence are as similar as possible, the weight factor W is optimized: Where N is the number of training samples, c n and sc n The original model f θ and shadow model f φ Regarding the central sentence prediction of the nth sample, is an indicator function, only when c n =sc n 1 when , 0 otherwise; The binary cross entropy loss function is used to optimize the weight factor W: Among them, α is the coefficient, ensuring priority; The weight factor W is optimized until the weight factor W converges.

7. The method for sorting conversational questions based on self-determination of central sentences according to claim 1 is characterized in that: The process of training the original model based on the original model and the converged weight factors using new samples and designing fine-tuning tasks to obtain the trained question ranking model specifically includes: Randomly delete the first k sentences in the dialogue context of the training samples in the training set to construct new samples (U′, Q); where k ∈ Possion(λ), and 1 ≤ k < c, c is the index of the identified central sentence, U′ = [U k+1 , …, U c , …, U l , λ is a hyperparameter; and the constructed new samples contain the identified central sentences; Use new samples and combine them with the binary cross entropy loss function to train the original model; Among them, y i ∈{0,1} is the question relevance label given in the dataset, f(U′,Q i ) indicates the question score; Get the trained question ranking model.

8. A conversational question ranking system based on self-determination of central sentences, characterized by: include: A ranking module is configured to: obtain the conversation context and question library, use the trained question ranking model to obtain question ranking scores, select the question with the highest score, and the central sentence identified when the question is used as the target question; The question ranking model training module is configured as follows: using the training set and combining the loss function to train the question ranking model, obtaining the model with the best performance on the validation set as the original model, and the model with the second best performance as the shadow model; wherein both the training set and the validation set include: dialogue context, question library and question relevance label; using a heuristic method and combining the given weight factor to calculate the score of each sentence in the dialogue context, selecting the sentence with the highest score in the dialogue context obtained by the original model as the identified central sentence, and selecting the sentence with the highest score in the dialogue context obtained by the shadow model as the identified another central sentence; based on the principle that the identified central sentence and the identified another central sentence are as similar as possible, combining the loss function, optimizing the weight factor until the weight factor converges; according to the original model and the converged weight factor, using the training set, designing a fine-tuning task to train the original model, and obtaining a trained question ranking model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for sorting conversational questions based on self-determination of a central sentence as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps in the method for sorting conversational questions based on self-determination of central sentences are implemented as described in any one of claims 1 to 7.

Images