Sentence ranking method and device based on information entropy and electronic equipment

Abstract

The disclosure provides a sentence ranking method and device based on information entropy and electronic equipment, and belongs to the computer field. The sentence ranking method comprises: obtaining a plurality of sentences to be ranked; determining the probability of each sentence in the plurality of sentences at each position in the paragraph; determining the information entropy of each ranking path based on the probability of each sentence in the plurality of sentences at each position in the paragraph, wherein the information entropy of the ranking path is negatively correlated with the semantic coherence corresponding to the ranking path; and determining the ranking result of the plurality of sentences based on the information entropy of each ranking path. By using the disclosure, the sentence ranking can be realized to obtain a semantically coherent paragraph.

Description

Technical Field

[0001] This disclosure relates to the field of natural language processing technology, and in particular to a sentence ranking method, apparatus and electronic device based on information entropy. Background Technology

[0002] Semantic coherence is a fundamental aspect of natural language processing tasks. Typically, in a coherent discourse, sentences should follow a chronological order of events, and the correct logical sequence of parts within the discourse facilitates comprehension. The order of sentences within a paragraph determines its coherence; therefore, sentence ordering is an important task in natural language processing, attempting to organize jumbled sentences into a semantically coherent text. Summary of the Invention

[0003] According to one aspect of this disclosure, a sentence sorting method is provided, comprising:

[0004] Get multiple sentences to be sorted;

[0005] Determine the probability of each sentence in a paragraph at each position;

[0006] The information entropy of each sorting path is determined based on the probability of each sentence in each position in the paragraph.

[0007] The sorting results of multiple sentences are determined based on the information entropy of each sorting path.

[0008] According to another aspect of this disclosure, a sentence sorting apparatus is provided, comprising:

[0009] The retrieval module is used to retrieve multiple sentences to be sorted;

[0010] The first determining module is used to determine the probability of each sentence in each position in the paragraph among the plurality of sentences;

[0011] The second determining module is used to determine the information entropy of each sorting path based on the probability of each sentence in each position in the paragraph.

[0012] The third determining module is used to determine the sorting result of the multiple sentences based on the information entropy of each sorting path.

[0013] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0014] Processor; and

[0015] A memory that stores a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the methods of this disclosure.

[0016] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods of this disclosure.

[0017] One or more technical solutions provided in this disclosure determine the probability of each sentence in each position in a paragraph among multiple sentences to be processed, determine the information entropy of each sorting path based on the probability of each sentence in each position in the paragraph, and use the information entropy to represent the disorder of a sorted sentence sequence, so as to realize sentence sorting to obtain a semantically coherent paragraph. Attached Figure Description

[0018] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A flowchart of a sentence sorting method according to an exemplary embodiment of the present disclosure is shown;

[0020] Figure 2 A flowchart of a method for constructing a disordered sentence sequence according to an exemplary embodiment of the present disclosure is shown;

[0021] Figure 3 A schematic block diagram of a sentence sorting model according to an exemplary embodiment of the present disclosure is shown;

[0022] Figure 4 A schematic block diagram of an exemplary BERT model is shown;

[0023] Figure 5 A schematic block diagram of an exemplary self-attention unit is shown;

[0024] Figure 6 A schematic block diagram of an exemplary Transformer encoder network is shown;

[0025] Figure 7 A schematic block diagram of an exemplary fully connected layer network for a decoding module is shown.

[0026] Figure 8 A flowchart is shown illustrating a method for determining the probability of each sentence in a paragraph at various positions using the sentence ranking model of this disclosure, according to an exemplary embodiment of this disclosure;

[0027] Figure 9 A schematic diagram illustrating exemplary sentences, positions, and probabilities is shown;

[0028] Figure 10 A flowchart of a sentence sorting question processing method according to an exemplary embodiment of the present disclosure is shown;

[0029] Figure 11 A schematic block diagram of a sentence sorting apparatus according to an exemplary embodiment of the present disclosure is shown;

[0030] Figure 12 A schematic block diagram illustrating the construction of a disordered sentence sequence according to an exemplary embodiment of the present disclosure is shown;

[0031] Figure 13 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0032] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0033] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0034] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0035] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0036] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0037] Traditional methods include sentence ranking by identifying word chains, calculating transition probabilities between sentences and greedily decoding sentence order, using Hidden Markov Models (HMMs) for topic and topic transition modeling, and extracting entities and learning entity transition probabilities for sentence ranking. These methods have limitations, namely, their reliance on linguistic domain knowledge.

[0038] In the field of deep learning, some researchers use Bi-LSTM networks to encode sentences, followed by Transformer encoders that remove positional encoding to encode paragraphs, and finally Pointer Networks to decode the sentence order. Other researchers have replaced the sentence encoder with a Transformer-based Bi-directional Encoder Representation (BERT) and the decoder with output sentence order scores. Still others use BERT-based sentence pair encoding to capture local and global information between sentences and incorporate relative order information into PointerNetworks for sentence order decoding. The main difference between these methods lies in the decoding method of sentence order. Decoding using Pointer Networks suffers from error propagation, meaning that a decoding error at a certain position can lead to an incorrect decoding order. Decoding by outputting sentence order scores introduces errors in loss calculation. Still others use a Bi-directional Autoregressive Transformer (BART) to solve sentence ordering tasks by generating sentence order tags; however, this method suffers from the problem of generating duplicate sentence order tags.

[0039] To address this, this disclosure provides a scheme for sentence sorting and constructing disordered sentence sequences. Information entropy is used to represent the degree of disorder in a sorted sentence sequence. The smaller the information entropy of the sentence sequence, the lower its disorder level, and the more semantically coherent the arrangement; conversely, the larger the information entropy, the higher the disorder level, and the less semantically coherent the arrangement. Sentence sorting and disordered sentence sequence construction are based on information entropy.

[0040] The present disclosure is described below with reference to the accompanying drawings.

[0041] The solution disclosed herein can be implemented by one or more computers in one or more locations, and the one or more computers in one or more locations may include terminals and / or servers and other devices with computing capabilities. The following description uses the implementation of the solution disclosed herein by a terminal as an example.

[0042] This disclosure provides a sentence sorting method. Using this method, information entropy is used to represent the disorder of a sorted sentence sequence, enabling sentence sorting to obtain semantically coherent paragraphs, and quantitatively evaluating various sorting methods using information entropy.

[0043] Figure 1 A flowchart of a sentence sorting method according to an exemplary embodiment of the present disclosure is shown, such as... Figure 1 As shown, the method includes steps S101 to S104.

[0044] Step S101: The terminal obtains multiple sentences to be sorted.

[0045] Multiple sentences to be sorted can form a semantically coherent paragraph. As an example, the sentences to be sorted are as follows: T0: Therefore, Socrates will die; T1: Socrates is a man; T2: All men will die. The sentences T0, T1, and T2 can form the following semantically coherent paragraph: All men will die (T2), Socrates is a man (T1), therefore Socrates will die (T0). This semantically coherent paragraph is a syllogism in logic.

[0046] In step S102, the terminal determines the probability of each sentence in each position within the paragraph.

[0047] A paragraph is a segment of text that includes multiple sentences that are to be ordered.

[0048] Multiple sentences to be sorted can form a semantically coherent paragraph, which contains multiple positions, the number of which is equal to the number of sentences to be sorted.

[0049] Following the example above, the paragraph has three positions, P0, P1, and P2. The probabilities of sentence T0 appearing in the three positions of the paragraph are as follows: P T0-P0 P T0-P1 and P T0-P2 The probabilities of sentence T1 appearing in the three positions of the paragraph are as follows: P T1-P0 P T1-P1 and P T1-P2 The probabilities of sentence T2 appearing in the three positions of the paragraph are as follows: P T2-P0 P T2-P1 and P T2-P2 .

[0050] In step S103, the terminal determines the information entropy of each sorting path based on the probability of each sentence in each position in the paragraph.

[0051] Taking m sentences as an example, a paragraph composed of m sentences can have m factorial (represented as m!) possible arrangements. The information entropy of each arrangement path is determined. Information entropy is used to represent the degree of disorder in a sorted sentence sequence. The smaller the information entropy of the sentence sequence, the lower its disorder, and the more semantically coherent the arrangement; conversely, the larger its information entropy, the higher its disorder, and the less semantically coherent the arrangement.

[0052] In one implementation, the multiple sentences are N sentences, where N is a natural number not less than 2. Step S102 involves determining the probability of each sentence in each position within the paragraph, including: determining the probability of each sentence in each of the N sentences in M ​​positions, where M is a preset value and is a natural number not less than 2, and M is not less than N. Step S103 involves determining the information entropy of each sorting path based on the probability of each sentence in each position within the paragraph, including: determining the information entropy of each sorting path based on the probability of each sentence in each of the N sentences in the first N positions out of the M positions.

[0053] As one implementation method, in step S103, the information entropy of each sorting path is determined based on the probability of each sentence in each position in the paragraph, including: determining the information of each sorting path based on the probability of each sentence in each position in the paragraph and the logarithm of the probability of each sentence in each position in the paragraph.

[0054] As an example, the information entropy of each sorting path is determined as follows:

[0055] H(P)=-∑ S (P(Tp k )*log P(Tp k )),

[0056] Where H(P) represents the information entropy of the sorting path P, and P(Tp) k Let ) represent the probability of sentence Tp at the k-th position, and S represent the set of the plurality of sentences, with Tp belonging to S.

[0057] In step S104, the terminal determines the sorting result of the multiple sentences based on the information entropy of each sorting path.

[0058] As one possible implementation, the sorting path with the minimum information entropy can be determined as the sorting result of the sentences to be sorted. As another possible implementation, several sorting paths with relatively low information entropy can be determined as candidate sorting results.

[0059] This disclosure provides a method for constructing disordered sentence sequences. By using this method to construct disordered sentence sequences and employing information entropy to represent the degree of disorder in a sorted sentence sequence, it is possible to construct disordered sentence sequences and quantitatively evaluate the degree of disorder of various sentence sequences using information entropy.

[0060] Figure 2 A flowchart of a method for constructing a disordered sentence sequence according to an exemplary embodiment of the present disclosure is shown, such as... Figure 2 As shown, the method includes steps S201 to S204.

[0061] Step S201: The terminal acquires multiple sentences to be processed.

[0062] As one possible implementation, the multiple sentences to be processed can be arranged into a semantically coherent paragraph. The multiple sentences to be processed can be in a semantically coherent order or in a random order.

[0063] As an example, the multiple sentences to be processed can be semantically coherent paragraphs extracted from news, articles, or books.

[0064] In step S202, the terminal determines the probability of each sentence in each position within the paragraph.

[0065] A paragraph refers to a segment of text that includes multiple sentences to be processed.

[0066] In step S203, the terminal determines the information entropy of each sorting path based on the probability of each sentence at each position in multiple sentences.

[0067] The method for determining information entropy is described in the foregoing description of this disclosure and will not be repeated here.

[0068] In step S204, the terminal selects at least one sorting path as the disordered sentence sequence based on the information entropy of each sorting path.

[0069] Information entropy is used to represent the degree of disorder in a sorted sequence of sentences. The smaller the information entropy of the sentence sequence, the lower its disorder and the more semantically coherent the order of the sentences. The larger the information entropy, the higher the disorder and the less semantically coherent the order of the sentences.

[0070] As one possible implementation, a specific number of sorting paths with high information entropy are selected as the disordered sentence sequence. Specifically, the sorting paths are sorted in descending order of information entropy; the top P sorting paths are selected as the disordered sentence sequence. The selected disordered sentence sequence can be used to train a sentence sorting model.

[0071] This disclosure also provides a sentence ranking model training method, which constructs multiple training samples based on the disordered sentence sequence construction method of this disclosure; and uses these multiple training samples to train the sentence ranking model. From the perspective of training the sentence ranking model, the more difficult it is to understand the disordered sentence sequence input to the model, the higher its semantic uncertainty and disorder level, the better the model's training effect and the higher its generalization ability. Based on the training samples constructed using the above disordered sentence sequence construction method, the disorder level of the disordered sentence sequence can be quantitatively evaluated. Using sentence sequences with a high degree of disorder helps to improve the training effect.

[0072] Sentence ranking models can be any model that takes a disordered sentence sequence as input and sorts sentences. For example, one model mentioned earlier uses a Bidirectional Long Short-Term Memory (Bi-LSTM) network to encode sentences, then uses a Transformer encoder with positional encoding removed to encode paragraphs, and finally uses a pointer network to decode the sentence order. Another model mentioned earlier replaces the sentence encoder with a Transformer-based Bidirectional Encoder Representation (BERT) and replaces the decoder with the output sentence ranking score. Yet another model mentioned earlier uses BERT-based sentence pair encoding to capture local and global information between sentences and incorporates relative order information into the Pointer Network to decode the sentence order. This embodiment does not limit this approach.

[0073] In some implementations, at least a portion of the training samples are constructed based on semantically coherent paragraphs (e.g., paragraphs extracted from news, articles, or books), where the sentence ordering path within the semantically coherent paragraphs is used as the label for the corresponding training sample. This eliminates the need for manual data labeling, enabling supervised training of the sentence ordering model.

[0074] In sentence ranking and disordered sentence construction, it is necessary to determine the probability of each sentence in a paragraph at each position. One possible implementation involves using a sentence ranking model to process multiple sentences and output the probability of each sentence in a paragraph at each position. Sentence ranking models can include various types of machine learning models. Considering the advantages of deep neural networks, one possible implementation of a deep neural network-based sentence ranking model is described below.

[0075] Figure 3 A schematic block diagram of a sentence ranking model according to an exemplary embodiment of the present disclosure is shown, such as... Figure 3 As shown, the sentence ranking model may include a first encoding module, a second encoding module, and a decoding module.

[0076] The first encoding module is configured to encode each sentence in a plurality of sentences separately, outputting a first vector for each sentence, where the first vector of each sentence is the sentence embedding representation of the sentence. Sentence embedding represents the entire sentence and its semantic information as a vector, allowing the machine to understand context, intent, and other subtle nuances hidden in the text. This enables the encoding of each sentence.

[0077] The first encoding module can process each sentence in a series of sentences sequentially, or process at least some sentences in a series of sentences in parallel.

[0078] The first encoding module takes as input a sequence of characters or words that make up the sentence when processing each sentence. For example, in the sentence "Socrates will die," the input to the encoding unit of the first encoding module includes at least the sequence of "Socrates," "Greek," "Socrates," "Die," "will," and "die." The first encoding module processes this sequence to output a vector of the sentence.

[0079] As one possible implementation, the encoding unit of the first encoding module may include: a Transformer encoder network, a Bidirectional Long Short-Term Memory (Bi-LSTM) network, a Transformer-based Bidirectional Encoding Unit Representation (BERT) model, etc. This embodiment does not limit this, and other sentence encoding models are also feasible, such as the Universal Sentence Encoder.

[0080] The BERT model can be trained using large-scale unlabeled corpora to obtain semantic representations of text containing rich semantic information. This will be illustrated using the first encoding module of the BERT model as the encoding unit.

[0081] Figure 4 A schematic block diagram of an exemplary BERT model is shown, such as... Figure 4 As shown, the input sequence consists of special classification embeddings ([CLS]), each character in the sentence, and special tokens ([SEP]). The final hidden state (output vector) corresponding to the special classification embeddings is used as the first vector of the sentence, and the special tokens are used to mark the end of the sentence. The input representation corresponding to the input sequence is the sum of character vectors (Token Embeddings), type vectors (Segment Embeddings), and position vectors (Position Embeddings).

[0082] The input representation is processed by a series of self-attention layers in the BERT model, each layer consisting of multiple self-attention units in parallel. In the series of self-attention layers, each self-attention unit in the first layer takes the input representation as input, and in each subsequent layer, each self-attention unit takes the output of all self-attention units in the previous layer as input. The output of the last layer is the output vector, which corresponds sequentially to the special classification embedding ([CLS]), each word in the sentence, and the special token ([SEP]). The output vector corresponding to the special classification embedding is used as the first vector of the sentence. Figure 4 (Shown as "sentence vector").

[0083] Figure 5 A schematic block diagram of an exemplary self-attention unit is shown, such as... Figure 5 As shown, the self-attention unit includes: a multi-head self-attention mechanism, residual connections and normalization, a feedforward layer, and residual connections and normalization.

[0084] The second encoding module is configured to perform paragraph encoding based on the input sequence composed of the first vectors of the aforementioned multiple sentences, outputting an encoding matrix. The encoding vector corresponding to each sentence in the encoding matrix is ​​used as the second vector for that sentence. The second vector of each sentence is the sentence embedding representation after information interaction between that sentence and other sentences in the input sequence. This enables information interaction between sentences, with each sentence's second vector containing global paragraph information.

[0085] The input sequence of the second encoding module includes first vectors of multiple sentences, with each sentence's first vector serving as an input element of the second encoding module. The first vectors can be arranged in any order within the input; as one possible implementation, the first vectors of multiple sentences are ordered according to the original order of the sentences.

[0086] As one possible implementation, the second encoding module may include a Transformer encoder network, a bidirectional long short-term memory network (Bi-LSTM), etc. This embodiment does not limit this.

[0087] This explanation will use the encoder network of the Transformer as an example. Figure 6 A schematic block diagram of an exemplary Transformer encoder network is shown, such as... Figure 6 As shown, the encoder network of the Transformer includes multiple encoders. Figure 6 The diagram shows 6 layers. The first vectors of multiple sentences form the input sequence of the Transformer's encoder network. For example, as shown... Figure 6As shown, the input sequence is exemplarily represented as {T0, T1, T2, T3, T4}, where T0, T1, T2, T3, and T4 represent the first vector of the corresponding sentence. The output of the Transformer encoder network includes the second vector corresponding to each of T0, T1, T2, T3, and T4.

[0088] The decoding module is configured to process the second vector of each of the multiple sentences to output the probability of each sentence at each position in the paragraph. For each sentence's second vector, the decoding module of the sentence ranking model processes the second vector to output the probability of that sentence at each position in the paragraph.

[0089] As one possible implementation, the decoding module is a multi-class neural network module, wherein each of the multiple categories output by the multi-class neural network module corresponds to a position in a paragraph, and each category represents the probability of a sentence at that position. As an example, the decoding module includes a fully connected layer network.

[0090] As one possible implementation method, such as Figure 7 As shown, the decoding module includes a fully connected layer network, where the number of neurons in the input layer of the fully connected neural network matches the dimension of the second vector, and the output of each neuron in the output layer of the fully connected neural network corresponds to the probability of a position in the paragraph. Figure 7 As shown, the first fully connected layer contains N neurons, and the last fully connected layer contains M neurons. Each of the M neurons corresponds to a position in the paragraph. As an example, from top to bottom, they are the first sentence, the second sentence, and so on up to the Mth sentence of the paragraph.

[0091] In some implementations, the sentence ranking model can handle any number of sentences less than or equal to a preset number (e.g., M as described above). The number of sentences input to the sentence ranking model is less than or equal to the preset number. When the number of sentences input to the pre-trained sentence ranking model is less than the preset number, the outputs of neurons exceeding the preset number of sentences are close to or equal to zero. Taking M neurons corresponding to M classification outputs, M positions in a paragraph, and from top to bottom, the first sentence, second sentence to Mth sentence of the paragraph, as an example, when the number of input sentences is M-5, the outputs of the last 5 neurons are close to or equal to zero.

[0092] In some implementations, the sentence ranking model of this disclosure is trained using semantically coherent sentence sequences as training samples, minimizing the information entropy of the correct ranking path as the objective function, and trained in a self-supervised manner. The parameter initialization and update methods of the model can employ well-known methods, which will not be elaborated upon in this embodiment.

[0093] Figure 8A flowchart illustrating a method for determining the probability of each sentence in a paragraph at various positions within a plurality of sentences using the sentence ranking model of this disclosure, according to exemplary embodiments of this disclosure, is shown. Figure 8 As shown, the method includes steps S801 to S803.

[0094] Step S801: For each sentence, the first encoding module performs sentence encoding on the sentence to output the first vector of the sentence, wherein the first vector of each sentence is the sentence embedding representation of the sentence.

[0095] In step S802, the first vectors of multiple sentences are combined into an input sequence. The second encoding module processes the input sequence to perform paragraph encoding to output an encoding matrix. The encoding vector corresponding to each sentence in the encoding matrix is ​​used as the second vector of that sentence.

[0096] Step S803: For the second vector of each sentence, the decoding module processes the second vector to output the probability of the sentence at each position in the paragraph, thereby obtaining the probability of each sentence at each position in the paragraph among multiple sentences.

[0097] In some implementations, the sentence ranking model can handle any number of sentences less than or equal to a preset number. The number of sentences input to the sentence ranking model is less than or equal to the preset number. When the number of sentences input to the sentence ranking model is less than the preset number, the outputs of neurons exceeding this number are close to or equal to zero. For example, if M neurons correspond to M classification outputs, and the M classification outputs are, from top to bottom, the first sentence, second sentence, to the Mth sentence of a paragraph, then when the number of input sentences is M-5, the outputs of the last 5 neurons are close to or equal to zero. When determining the information entropy of the ranking path, the probability of the outputs of the aforementioned last 5 neurons can be ignored.

[0098] In one implementation, the aforementioned multiple sentences refer to N sentences, where N is a natural number not less than 2. Determining the probability of each sentence in each position within the paragraph includes: determining the probability of each sentence in each of the N sentences in M ​​positions, where M is a preset value (the aforementioned preset number), and M is a natural number not less than 2, and M is not less than N. Based on the probabilities of each sentence in each position within the paragraph, determining the information entropy of each sorting path includes: determining the information entropy of each sorting path based on the probabilities of each sentence in the N sentences in the first N positions out of the M positions.

[0099] Figure 9 An exemplary diagram illustrating sentences, positions, and probabilities is shown, such as... Figure 9As shown, the vertical axis represents positions 0, 1 to M-1, and the horizontal axis represents sentences T0, T1 to T4. Each sentence includes M probability values. The probability value corresponding to each position represents the probability that the sentence appears at that position when multiple sentences form a coherent semantic paragraph. The connecting lines represent a sorting path, such as... Figure 9 As shown, one sorting path is: T0->T1->T2->T3->T4.

[0100] As one possible implementation, when the number of sentences in the input sentence ranking model is less than a preset number, the information entropy of each ranking path is determined based on the probability of the number and position corresponding to the sentences in multiple categories. Continuing with the example above, the information entropy of each ranking path is determined based on the probability of the first 5 positions, ignoring the probabilities of other positions in the information entropy calculation.

[0101] An example of an embodiment of this disclosure is described below.

[0102] In this example, information entropy is used to represent the degree of disorder in a sorted sequence of sentences. The lower the information entropy of the sentence sequence, the lower its disorder, and the more semantically coherent the order. Conversely, the higher the information entropy, the higher the disorder, and the less semantically coherent the order.

[0103] Based on the probability of each sentence in each position within a paragraph, the information entropy of each sorting path is determined. The specific calculation formula is as follows:

[0104] H(P)=-∑ S (P(Tp k )*log P(Tp k )).

[0105] Where H(P) represents the information entropy of the sorting path P, and P(Tp) k Let ) represent the probability of sentence Tp at the k-th position, and S represent the set of the plurality of sentences, with Tp belonging to S.

[0106] One aspect of this example provides a method for decoding, training, and inference of a sentence ranking model. Specifically, the input sentence sequence is encoded to obtain a first vector for each sentence. These first vectors are then used to form an input sequence for paragraph encoding. This input sequence is further encoded into a second sentence vector for each sentence. The second sentence vector is then decoded using a fully connected network with a preset output dimension, thus obtaining the probability distribution of each sentence at different positions. During training, self-supervised labels are used to calculate the information entropy of the correct ranking path, and the model is trained by minimizing this information entropy. During inference, a scrambled sentence sequence is input, and the information entropy of different ranking paths is calculated. The ranking path with the minimum output information entropy is taken as the correct ranking result.

[0107] The following is combined with Figures 3 to 7 The training of the sentence ranking model shown is explained.

[0108] S1: Obtain the dataset for the sentence sorting task.

[0109] Each sample in the dataset has complete semantic information, containing several sentences. The order of the sentences conforms to semantic coherence and logic, and the maximum number of sentences M in the data set is obtained (i.e., the preset number).

[0110] Exemplary samples include sentences T0 to T4.

[0111] T0: Sally is the manager of a restaurant.

[0112] T1: She did not meet her job expectations.

[0113] T2: She is very careless at work.

[0114] T3: Her boss fired her because she talked too much on her phone.

[0115] T4: Afterwards, Sally applied for unemployment at the local government office.

[0116] S2: Encode each sentence in the samples of the dataset.

[0117] Each sentence in the sample is input as an input sequence into a program such as... Figure 4 and 5 In the BERT model shown, the pooled output of the last hidden layer (the output of the [CLS] semantic unit) is taken as the first vector to obtain the encoding vector of each sentence in the sample.

[0118] S3: Perform paragraph encoding on the sentence sequences of the samples in the dataset. Specifically, this includes S3.1 to S3.3.

[0119] S3.1: Arrange the first vectors of each sentence in the sample obtained in S2 into the input sequence according to their original order.

[0120] An exemplary input sequence is represented as {T0,T1,T2,T3,T4}, where T0, T1, T2, T3, and T4 represent the first vector of the corresponding sentence.

[0121] S3.2: Input the input sequence of S3.1 as follows Figure 6 The Transformer's Encoder module shown removes the positional encoding of the input sequence.

[0122] S3.3: Obtain the semantic unit output vector (i.e., the output vectors of T0 to T4) of the last hidden layer to obtain the second sentence vector of each sentence in the sample.

[0123] S4: Decode the sentence order of the input sentence sequence.

[0124] Use such as Figure 7 The fully connected layer network shown processes the second sentence vector of each sentence in the sample to obtain the probability distribution of each sentence at different positions in the sample.

[0125] The input dimension of the fully connected layer network is the dimension N of each second sentence vector (semantic unit output vector) obtained in S3.3, the output dimension is the preset number M obtained in S1, and the activation function is SoftMax.

[0126] The network parameters are randomly initialized using a standard normal distribution. Self-supervised labels are obtained based on the original order of the sentence sequence in the input sample, and the correct order in the sentence order decoding module is determined. The information entropy of the correctly ordered path is calculated, and training is performed by minimizing the information entropy of the correctly ordered path. The model weights are updated through backpropagation.

[0127] The process of using the trained sentence ranking model for inference is as follows:

[0128] Input a randomized sequence of sentences of number m (m <= M) into a trained sentence ranking model to obtain the ranking of each sentence in different combination positions, for a total of m!.

[0129] Calculate the information entropy of each of the m! sorting paths obtained, thereby obtaining the information entropy of each sorting result in the input disordered sample, and take the sorting path with the smallest information entropy as the correct sorting result.

[0130] Another aspect of this example provides a method for constructing highly disordered sequences of sentences, specifically:

[0131] The input sentence sequence is processed using a trained sentence ranking model to obtain the probability distribution of each sentence in the sentence sequence at different positions.

[0132] Calculate the information entropy of different sorting paths based on the above probability distributions;

[0133] The top k sorted paths with the highest output information entropy are output as the results.

[0134] These k ordered sentence sequences can be used to train sentence ranking models that take randomized text as input. From the perspective of training sentence ranking models, the more difficult it is to understand the randomized sentence sequences input to the model, the higher the semantic uncertainty and disorder, the better the model's training performance and the higher its generalization ability. Therefore, it can significantly improve training results.

[0135] The application of the disclosed solution will be explained below using automated question processing as an example.

[0136] The automated question processing involves various question types, among which "sentence ordering" is quite common. Students use common sense and semantic logic to reorder the disordered set of sentences provided in the question, so that the ordered sentence sequence has semantic coherence and logic.

[0137] Figure 10 A flowchart of a sentence sorting question processing method according to an exemplary embodiment of the present disclosure is shown, such as... Figure 10 As shown, the method includes steps S1001 to S1007.

[0138] Step S1001: The terminal acquires the image of the question.

[0139] For example, a user can take a photo using a mobile phone or tablet and upload it to an application. The application then feeds the image into a question type determination model. This model includes a layout analysis model and a text line detection model. The layout analysis model is a text detection model that detects the coordinates of text regions. The text line detection model detects horizontal text lines, identifying the text boxes for each line. Based on the detection results from the layout analysis model and the text line detection model, layout analysis is performed, merging the boxes. Simultaneously, based on the characteristics of each question type, the model analyzes which boxes make up each question. Finally, based on the coordinates of the boxes, the model is cropped from the original image to obtain the question image.

[0140] For example, a selection box can be provided to receive a user's selection of the question area on the image, and the question image can be extracted based on the image area selected by the user.

[0141] It should be understood that other known extraction methods can be used to extract the title image from the image, and the exemplary embodiments disclosed herein are not limited thereto.

[0142] In step S1002, the terminal extracts the answer to the question and multiple sentences to be sorted from the question image.

[0143] In step S1003, the terminal uses a sentence sorting model to process multiple sentences to be sorted, so as to output the probability of each sentence in each position in the paragraph.

[0144] In step S1004, the terminal determines the information entropy of each sorting path based on the probability of each sentence in each position in the paragraph.

[0145] In step S1005, the terminal determines the sorting path with the minimum information entropy as the sorting result.

[0146] In step S1006, the terminal compares the determined sorting result with the extracted question answers.

[0147] Step S1007: The terminal outputs the comparison result.

[0148] For example, if the answer to the question matches the sorting result, output "answer is correct"; if the answer to the question does not match the sorting result, output "answer is incorrect", and the sorting result can be output as the correct answer.

[0149] For example, if the answer to the question matches the sorting result, output that the answer is correct; if the answer to the question does not match the sorting result, further determine the inconsistent parts, output the inconsistent parts and the consistent parts, and output the sorting result as the correct answer.

[0150] The sentence sorting question processing method of this disclosure obtains the sorting result without the need for manual marking of the question answers, thus reducing the processing cost of sentence sorting questions.

[0151] This disclosure also provides a sentence sorting device, such as... Figure 11 As shown, it includes:

[0152] Module 1110 is used to retrieve multiple sentences to be sorted;

[0153] The first determining module 1120 is used to determine the probability of each sentence in each position in the paragraph among multiple sentences;

[0154] The second determining module 1130 is used to determine the information entropy of each sorting path based on the probability of each sentence in each position in the paragraph.

[0155] The third determining module 1140 is used to determine the sorting result of multiple sentences based on the information entropy of each sorting path.

[0156] In some embodiments, the second determining module 1130 is configured to:

[0157] For each sentence, the first encoding module of the sentence ranking model is used to encode the sentence to output the first vector of the sentence, where the first vector of each sentence is the sentence embedding representation of the sentence;

[0158] The first vectors of multiple sentences are combined into an input sequence. The second encoding module of the sentence ranking model is used to encode paragraphs based on the input sequence to output an encoding matrix. The encoding vector corresponding to each sentence in the encoding matrix is ​​used as the second vector of that sentence.

[0159] For the second vector of each sentence, the decoding module of the sentence ranking model is used to process the second vector to output the probability of the sentence at each position in the paragraph.

[0160] In some embodiments, the decoding module includes a fully connected neural network, wherein the number of neurons in the input layer of the fully connected neural network matches the dimension of the second vector, and the output of each neuron in the output layer of the fully connected neural network corresponds to the probability of a position in the paragraph.

[0161] In some embodiments, the aforementioned plurality of sentences are N sentences, where N is a natural number not less than 2. The second determining module 1130 is configured to: determine the probability of each sentence in the N sentences at M positions, where M is a preset value, and M is a natural number not less than 2 and not less than N. The third determining module 1140 is configured to: determine the information entropy of each sorting path based on the probability of each sentence in the N sentences at the first N positions out of the M positions.

[0162] In some embodiments, the third determining module 1140 is configured to: determine information about each sorting path based on the probability of each sentence in each position in the paragraph and the logarithmic value of the probability of each sentence in each position in the paragraph.

[0163] As an example, the information entropy of each sorting path is determined as follows:

[0164] H(P)=-∑ S (P(Tp k )*log P(Tp k )),

[0165] Where H(P) represents the information entropy of the sorting path P, and P(Tp) k Let ) represent the probability of sentence Tp at the k-th position, and S represent the set of the plurality of sentences, with Tp belonging to S.

[0166] In some embodiments, the method further includes: sorting each sorting path in descending order of information entropy; selecting the sorting path ranked in the top P positions as a randomized sentence sequence for training a sentence sorting model.

[0167] This disclosure also provides an apparatus for constructing disordered sentence sequences, such as... Figure 12 As shown, it includes:

[0168] Module 1210 is used to acquire multiple sentences to be processed;

[0169] The first determining module 1220 is used to determine the probability of each sentence in each position in the paragraph among the above multiple sentences;

[0170] The second determining module 1230 is used to determine the information entropy of each sorting path based on the probability of each sentence at each position in the above multiple sentences;

[0171] Selection module 1240 is used to select at least one sorting path as a disordered sentence sequence based on the information entropy of each sorting path.

[0172] As one possible implementation, the selection module 1240 is used to sort the sorting paths in descending order of information entropy; and select the sorting path ranked in the first P positions as the disordered sentence sequence.

[0173] Optionally, the first determining module 1220 determines the probability of each sentence in each position within the paragraph, specifically including:

[0174] The sentence ranking model is used to process the multiple sentences to output the probability of each sentence in each position in the paragraph.

[0175] Optionally, the sentence sorting model is described in the foregoing description of this embodiment and will not be repeated here.

[0176] Exemplary embodiments of this disclosure also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of this disclosure.

[0177] Exemplary embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to embodiments of this disclosure.

[0178] Exemplary embodiments of this disclosure also provide a computer program product, including a computer program, wherein, when executed by a processor of a computer, the computer program is used to cause the computer to perform a method according to an embodiment of this disclosure.

[0179] refer to Figure 13 The present invention describes a structural block diagram of an electronic device 1300 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0180] like Figure 13 As shown, the electronic device 1300 includes a computing unit 1301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1302 or a computer program loaded from a storage unit 1308 into a random access memory (RAM) 1303. The RAM 1303 may also store various programs and data required for the operation of the device 1300. The computing unit 1301, ROM 1302, and RAM 1303 are interconnected via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0181] Multiple components in electronic device 1300 are connected to I / O interface 1305, including: input unit 1306, output unit 1307, storage unit 1308, and communication unit 1309. Input unit 1306 can be any type of device capable of inputting information to electronic device 1300. Input unit 1306 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 1307 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1308 may include, but is not limited to, disk and optical disk. Communication unit 1309 allows electronic device 1300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0182] The computing unit 1301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1301 performs the various methods and processes described above. For example, in some embodiments, the sentence sorting method, the scrambled sentence sequence construction method, and the sentence sorting model training method can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1300 via ROM 1302 and / or communication unit 1309. In some embodiments, the computing unit 1301 can be configured by any other suitable means (e.g., by means of firmware) to perform the sentence sorting method, the scrambled sentence sequence construction method, and the sentence sorting model training method.

[0183] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0184] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0185] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0186] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0187] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0188] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A sentence sorting method, characterized in that, include: Obtain multiple sentences to be sorted, where N sentences are natural numbers not less than 2; The probability of each sentence in each position within a paragraph is determined from the plurality of sentences. This probability is predicted by a sentence ranking model based on the input sentences. The sentence ranking model includes a first encoding module, a second encoding module, and a decoding module. The decoding module includes a fully connected neural network. The last fully connected layer of the fully connected neural network includes M neurons, each corresponding to a position within the paragraph. M is a natural number not less than 2 and not less than N. When the number of sentences input to the sentence ranking model is less than M, the neurons output zero for the portion exceeding the number of sentences. The sentence ranking model is trained using sentence sequences as training samples, with the objective function being minimizing the information entropy of the correct ranking path, and is trained using a self-supervised method. Wherein, determining the probability of each sentence in each position in the paragraph among the plurality of sentences includes: determining the probability of each sentence in M ​​positions among the N sentences; Based on the probability of each sentence in each position in the paragraph, the information entropy of each sorting path is determined, wherein the information entropy of the sorting path is negatively correlated with the semantic coherence of the sorting path. The step of determining the information entropy of each sorting path based on the probability of each sentence in each position within the paragraph includes: Based on the probability of each sentence in the N sentences being in the first N positions out of the M positions and the logarithm of the probability of each sentence in the N sentences being in the first N positions out of the M positions, the information entropy of each sorting path is determined. The formula for determining the information entropy of each sorting path is as follows: ,in, Indicates sorting path Information entropy Sentence In the The probability of each position. This represents the set of the multiple sentences. belong ; Based on the information entropy of each sorting path, the sorting result of the multiple sentences is determined, wherein the sorting path with the smallest information entropy is determined as the sorting result of the multiple sentences.

2. The sentence sorting method as described in claim 1, characterized in that, Determining the probability of each sentence in each position within the paragraph includes: For each sentence, the first encoding module of the sentence ranking model is used to encode the sentence to output the first vector of the sentence, where the first vector of each sentence is the sentence embedding representation of the sentence; The first vectors of the multiple sentences are used to form an input sequence. The second encoding module of the sentence ranking model is used to perform paragraph encoding based on the input sequence to output an encoding matrix. The encoding vector corresponding to each sentence in the encoding matrix is ​​used as the second vector of that sentence. For the second vector of each sentence, the decoding module of the sentence ranking model is used to process the second vector to output the probability of the sentence at each position in the paragraph.

3. The sentence sorting method as described in claim 2, characterized in that, The number of neurons in the input layer of the fully connected neural network matches the dimension of the second vector, and the output of each neuron in the output layer of the fully connected neural network corresponds to the probability of a position in the paragraph.

4. The sentence sorting method as described in any one of claims 1 to 3, characterized in that, The method further includes: Sort the sorting paths in descending order of information entropy; The sorting path ranked in the top P positions is selected as the disordered sentence sequence for training the sentence sorting model.

5. A sentence sorting device, characterized in that, include: The acquisition module is used to acquire multiple sentences to be sorted, wherein the multiple sentences are N sentences, and N is a natural number not less than 2; The first determining module is used to determine the probability of each sentence in the plurality of sentences at each position in the paragraph. The probability is predicted by the sentence ranking model based on the input plurality of sentences. The sentence ranking model includes a first encoding module, a second encoding module, and a decoding module. The decoding module includes a fully connected neural network. The last fully connected layer of the fully connected neural network includes M neurons. Each of the M neurons corresponds to a position in the paragraph, and M is a natural number not less than 2 and not less than N. When the number of sentences input to the sentence ranking model is less than M, the neurons for the portion exceeding the number of sentences output zero. The sentence ranking model is trained using sentence sequences as training samples, minimizing the information entropy of the correct ranking path as the objective function, and is trained in a self-supervised manner. The first determining module is further configured to: determine the probability of each sentence in the N sentences being in the M positions; The second determining module is used to determine the information entropy of each sorting path based on the probability of each sentence in each position in the paragraph, wherein the information entropy of the sorting path is negatively correlated with the semantic coherence of the sorting path. The second determining module is further configured to determine the information entropy of each sorting path based on the probability of each sentence in the N sentences being in the first N positions out of the M positions and the logarithm of the probability of each sentence in the N sentences being in the first N positions out of the M positions. The formula for determining the information entropy of each sorting path is as follows: ,in, Indicates sorting path Information entropy Sentence In the The probability of each position. This represents the set of the multiple sentences. belong ; The third determining module is used to determine the sorting result of the multiple sentences based on the information entropy of each sorting path, wherein the sorting path with the smallest information entropy is determined as the sorting result of the multiple sentences.

6. An electronic device, characterized in that, include: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-4.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

Images