Text scoring method, text scoring model, text scoring device, and storage medium

Abstract

Embodiments of the present specification provide a text scoring method, a text scoring model, a text scoring device and a storage medium, wherein the text scoring method comprises: dividing an input text according to a preset division granularity to obtain a plurality of text segments; performing data encoding based on each text segment to obtain corresponding segment encoding data; performing data encoding based on the input text to obtain sequence encoding data; wherein the sequence encoding data comprises text encoding sub-data and / or a plurality of language element encoding sub-data; and performing score prediction based on the segment encoding data corresponding to the plurality of text segments and the sequence encoding data to obtain a text predicted score. The above scheme can improve the accuracy of score prediction.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of natural language processing technology, and in particular to a text scoring method, a text scoring model, a text scoring device, and a storage medium. Background Technology

[0002] Traditional essay scoring is done manually, which is labor-intensive and highly subjective. With the development of computer science and technology, the ability and level of automated information processing have been significantly improved, leading to the emergence of Automated Essay Scoring (AES) technology.

[0003] AES technology can be mainly divided into traditional AES methods and deep learning-based AES methods. Traditional AES methods manually extract scoring-related data from the text and use simple neural network models such as regression, classification, or ranking models for score prediction. Because traditional AES methods only focus on global features of the text and ignore local features related to the scoring, the accuracy of score prediction is relatively low.

[0004] Deep learning-based AES methods offer an end-to-end solution, inputting text into a deep learning-based neural network model for score prediction and outputting the score result. Furthermore, current research in deep learning-based AES methods includes sentence-level scoring of text, primarily involving: splitting the text using punctuation marks, extracting feature information from each sentence, and then predicting the score.

[0005] However, if there are too many punctuation marks, the segmented text becomes fragmented. Furthermore, for long sentences, due to their length, some feature information may be lost during processing. Therefore, the accuracy of score prediction remains low when dealing with complex and varied text. Thus, existing AES methods suffer from poor score prediction performance. Summary of the Invention

[0006] In view of this, embodiments of this specification provide a text scoring method, a text scoring model, a text scoring device, and a storage medium, which can improve the accuracy of score prediction.

[0007] This specification provides an embodiment of a text scoring method, including:

[0008] The input text is divided into multiple text segments according to a preset granularity.

[0009] Based on each of the aforementioned text fragments, data encoding is performed to obtain the corresponding fragment encoding data;

[0010] Based on the input text, data encoding is performed to obtain sequence encoded data; wherein, the sequence encoded data includes: text encoded sub-data and / or multiple language element encoded sub-data;

[0011] Based on the segment encoding data and sequence encoding data corresponding to the multiple text segments, a score prediction is performed to obtain the text prediction score.

[0012] This specification also provides a text scoring model comprising: an encoder adapted to divide input text into multiple text segments according to a preset granularity, and to perform data encoding based on each text segment to obtain corresponding segment encoded data; and to perform data encoding based on the input text to obtain sequence encoded data; wherein the sequence encoded data includes: text encoded sub-data and / or multiple language element encoded sub-data; and a scorer adapted to perform score prediction based on the corresponding segment encoded data of the multiple text segments and the sequence encoded data to obtain a text prediction score.

[0013] This specification also provides a text scoring device, comprising: a segmentation module adapted to segment input text according to a preset segmentation granularity to obtain multiple text segments; a data encoding module adapted to perform data encoding based on each of the text segments to obtain corresponding segment encoding data; and to perform data encoding based on the input text to obtain sequence encoding data; wherein the sequence encoding data includes: text encoding sub-data and / or multiple language element encoding sub-data; and a score prediction module adapted to perform score prediction based on the corresponding segment encoding data of the multiple text segments and the sequence encoding data to obtain a text prediction score.

[0014] This specification also provides a text scoring device, including a memory and a processor; wherein the memory is adapted to store one or more computer instructions, and the processor executes the steps of the text scoring method described in any embodiment of this specification when running the computer instructions.

[0015] This specification also provides a computer-readable storage medium storing computer instructions that, when executed, perform the steps of the text scoring method described in any embodiment of this specification.

[0016] The text scoring scheme provided in this embodiment divides the input text into multiple text segments according to a preset granularity. Then, data encoding is performed on each text segment to obtain corresponding segment encoded data, and data encoding is also performed on the input text to obtain sequence encoded data. Finally, score prediction is performed based on the corresponding segment encoded data and the sequence encoded data of the multiple text segments to obtain a predicted text score. As can be seen, dividing the input text into segments with a fixed granularity avoids the problem of text fragmentation caused by excessive punctuation marks and the problem of feature information loss due to excessively long sentences during processing. It increases the amount of feature information contained in the segment encoded data, and multiple effective segment encoded data can reflect the correlation between multiple text segments, thereby improving the ability of segment encoded data to represent the corresponding text segments. Since the sequence encoded data contains feature information at the input text level and / or feature information at the language element level, combining multiple segment encoded data and the sequence encoded data can provide multi-level, rich, and useful feature information for score prediction, thus improving the accuracy of score prediction. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a text scoring method provided in an embodiment of this specification.

[0018] Figure 2 This is a flowchart for data encoding of text fragments, provided as an embodiment of this specification.

[0019] Figure 3 This is a flowchart for data encoding of input text provided as an embodiment of this specification.

[0020] Figure 4 This is a flowchart illustrating the text scoring method provided in the embodiments of this specification in a specific application scenario.

[0021] Figure 5 A flowchart illustrating the text scoring method provided in the embodiments of this specification in another specific application scenario.

[0022] Figure 6 This is a structural block diagram of a text scoring model provided in the embodiments of this specification.

[0023] Figure 7 This is a structural block diagram of a text scoring device provided in an embodiment of this specification.

[0024] Figure 8 This is a structural block diagram of another text scoring device provided in the embodiments of this specification. Detailed Implementation

[0025] As described in the background section, both traditional AES methods and deep learning-based AES methods suffer from poor score prediction performance.

[0026] To address the aforementioned issues, this specification provides a text scoring scheme. The input text is divided into multiple text segments according to a preset granularity. Then, data encoding is performed on each text segment to obtain corresponding segment encoded data. Additionally, data encoding is performed on the input text to obtain sequence encoded data. Finally, based on the sum of the segment encoded data and the sequence encoded data of the multiple text segments, a score prediction is performed to obtain a predicted text score. This improves the accuracy of score prediction.

[0027] To enable those skilled in the art to better understand and implement the concepts, implementation schemes, and advantages of this specification, the following illustrative description is provided with reference to the accompanying drawings and specific embodiments.

[0028] Reference Figure 1 The above is a flowchart of a text scoring method provided in an embodiment of this specification. In this embodiment, the text scoring method may include the following steps:

[0029] S11, the input text is divided according to the preset division granularity to obtain multiple text fragments.

[0030] The specific data format of the input text can be a sequence of computer-recognizable characters. These computer-recognizable characters can use any existing character set encoding format, such as ASCII, GB2312, BIG5, GB18030, etc.; or they can use a custom character set encoding format.

[0031] In practical applications, the source of the input text may include: manually edited text libraries, text resources available on public networks, text identified from images using image recognition technology, and text identified from audio using speech recognition technology. This specification does not impose specific limitations on these sources.

[0032] It is understood that, depending on the actual language scenario of the application, the input text may be characters from at least one language, or may include characters from multiple languages. For example, the input text may include at least one of Chinese characters (including Chinese characters and Chinese punctuation marks) and English characters (including English letters and English punctuation marks). This specification does not impose specific limitations on the types and number of languages ​​corresponding to the input text.

[0033] In specific implementation, the granularity of the division can be used to characterize the maximum number of the smallest divisible parts of the input text that the text fragment can include, i.e., the maximum length of the text fragment.

[0034] The smallest divisible part of the input text can be set according to the language of the input text. For example, for Chinese, the smallest divisible part can be characters and Chinese punctuation marks, or characters, words and Chinese punctuation marks, etc. For English, the smallest divisible part can be words and English punctuation marks, or word roots, words and English punctuation marks, or words, phrases and English punctuation marks, etc.

[0035] It should be noted that, for ease of description and understanding, this specification uses the term "language element" to refer to the smallest divisible part of the input text. In practical applications, other names may also be used to refer to the smallest divisible part of the input text. This specification does not impose any specific limitations on this. Correspondingly, the granularity of the division can also be understood as: the maximum number of language elements that the text segment can include.

[0036] Based on this, the input text can be decomposed to obtain multiple language elements arranged in sequence; the multiple language elements are divided according to a preset division granularity, and the multiple language elements divided into the same group are taken as a text segment to obtain multiple text segments.

[0037] S12, based on each of the text segments, perform data encoding to obtain the corresponding segment encoding data.

[0038] As an optional example, data encoding based on text fragments may include the following steps: converting the data form of each text fragment from characters to vectors, while retaining the data validity at the corresponding text fragment level after conversion.

[0039] S13, based on the input text, perform data encoding to obtain sequence encoded data; wherein, the sequence encoded data includes: text encoded sub-data and / or multiple language element encoded sub-data.

[0040] As an optional example, data encoding based on input text may include the following steps: converting the data form of the input text from characters to vectors, while preserving data validity at the input text level and / or data validity at the language element level after the conversion.

[0041] S14, based on the segment encoding data and sequence encoding data corresponding to the multiple text segments, score prediction is performed to obtain the text prediction score.

[0042] As an optional example, score prediction based on fragment encoding data and sequence encoding data may include the following steps: combining the fragment encoding data and sequence encoding data of multiple text fragments to directly or indirectly obtain the predicted score value.

[0043] As can be seen from the above, depending on the sentence length and fixed granularity of the input text, a sentence may be divided into the same text segment, or it may be divided into multiple different text segments. Therefore, unlike segmentation based on punctuation marks, the text segments in the embodiments of this specification are divided at a consistent scale, and may contain only part of the language elements of a sentence.

[0044] Text fragments that include at least one sentence and text fragments that include at least one sentence and a portion of the language elements of a sentence can retain the feature information of each sentence itself (such as the semantics, grammar, pragmatics, and contextual relationships of the sentence itself) and the feature information of a portion of the language elements (such as the semantics, grammar, pragmatics, and contextual relationships of that portion of the language elements). They can also retain the feature information between sentences and between portions of the language elements of sentences (such as the contextual relationships between sentences and between portions of the language elements of sentences). Text fragments that only include a portion of the language elements of a sentence can better highlight the feature information of that portion of the language elements.

[0045] Based on this, dividing text according to a fixed granularity can avoid the fragmentation problem caused by too many punctuation marks and the loss of feature data caused by excessive sentence length during processing. The segment encoded data obtained after data encoding can contain rich feature information at the text segment level, thereby improving the representation ability of the segment encoded data. Multiple valid segment encoded data can reflect the correlation between multiple text segments.

[0046] Furthermore, since the sequence-encoded data obtained through data encoding can contain feature information at the input text level and / or feature information at the language element level, combining multiple fragment-encoded data and the sequence-encoded data can provide multi-level, rich, and useful feature information for score prediction, thereby improving the accuracy of score prediction.

[0047] It should be noted that there is no necessary order of execution between steps S11-S12 and S13 in the above embodiments. Steps S11-S12 and S13 can be executed according to the specific circumstances. This specification does not impose any specific restrictions on this embodiment.

[0048] It is understood that the term "sentence length too long" as used in the embodiments of this specification is a relative concept, and the criteria for judging whether a sentence is too long may vary depending on the specific application scenario. For example, in some language application scenarios, sentences exceeding 10 characters may be considered too long, while in other language application scenarios, sentences exceeding 100 characters may be defined as too long. This specification does not impose specific limitations on this.

[0049] It is also understood that the term "pragmatics" as used in this specification refers to the correspondence between the meaning of linguistic symbols and their uses.

[0050] In practice, existing decomposition programs (such as existing word segmenters and word segmentation tools) can be used to decompose the input text, or self-compiled decomposition programs (such as self-compiled word segmenters and word segmentation tools) can be used to decompose the input text. This manual does not impose any specific restrictions on this.

[0051] In practice, when dividing the input text into segments with a fixed granularity, all segments except the last one contain the same number of language elements. However, the last segment may contain fewer language elements than the granularity. Therefore, for the last segment, a designated symbol can be used as a placeholder language element to ensure that the number of language elements in the subsequent segment reaches the granularity, thus guaranteeing a consistent number of language elements across all segments.

[0052] In practical implementation, the specific process for score prediction based on the corresponding segment encoding data and sequence encoding data of multiple text segments can be set according to the specific application scenario and requirements. For example, the step of performing score prediction based on the corresponding segment encoding data and sequence encoding data of multiple text segments to obtain a text prediction score may include any of the following steps:

[0053] A) Perform data fusion on multiple segment encoded data to obtain first fused data, and perform data fusion on the sequence encoded data to obtain second fused data, and perform score prediction based on the first fused data and the second fused data to obtain the text prediction score.

[0054] As a specific example, data fusion based on multiple fragment coded data may include the steps of updating and summarizing information on the multiple fragment coded data; information updating and summarizing can be achieved through steps of splicing, mapping, and arithmetic operations. The arithmetic operations may include one or more combinations of addition, subtraction, multiplication, etc. This specification does not limit the specific process of data fusion based on fragment coded data in the embodiments.

[0055] As a specific example, data fusion based on sequence-coded data may include the steps of updating and summarizing information on text-coded sub-data and / or multiple language element-coded sub-data in the sequence-coded data; information updating and summarizing can be achieved through steps of concatenation, mapping, and arithmetic operations. Arithmetic operations may include at least one of addition, subtraction, multiplication, etc. The embodiments in this specification do not limit the specific process of data fusion based on sequence-coded data.

[0056] As a specific example, score prediction based on the first fused data and the second fused data may include the step of directly or indirectly obtaining a numerical value by combining the first fused data and the second fused data.

[0057] B) Based on the multiple fragment encoded data and the sequence encoded data, data fusion is performed to obtain third fused data, and score prediction is performed based on the third fused data to obtain the text prediction score.

[0058] As a specific example, data fusion based on multiple fragment coded data and the sequence coded data may include the steps of updating and summarizing information by combining multiple fragment coded data and the sequence coded data; information updating and summarizing can be achieved through steps of splicing, mapping, and arithmetic operations. Arithmetic operations may include one or more combinations of addition, subtraction, multiplication, etc. This specification does not limit the specific process of data fusion based on multiple fragment coded data and the sequence coded data in the embodiments.

[0059] As a specific example, score prediction based on the third fusion data may include the step of converting the third fusion data from a vector to a numerical value.

[0060] Therefore, by updating the feature information of fragment encoded data and / or sequence encoded data through data fusion, important feature information can be retained while unimportant feature information is forgotten, and the correlation between feature information can be obtained, thereby enhancing the effectiveness of the data, improving the data representation ability, providing rich and useful feature information for score prediction, and thus improving the efficiency and accuracy of score prediction.

[0061] In practical implementation, the specific process for score prediction based on the first fused data and the second fused data can be set according to the specific application scenario and requirements. For example, the step of performing score prediction based on the first fused data and the second fused data to obtain the text prediction score may include any of the following steps:

[0062] A-1) Based on the first fused data, perform data transformation to obtain a first candidate score, and based on the second fused data, perform data transformation to obtain a second candidate score, and based on the first candidate score and the second candidate score, perform calculation operations to obtain the text prediction score.

[0063] A-2) Based on the first fused data and the second fused data, perform calculation operations, and based on the results of the calculation operations, perform data transformation to obtain the text prediction score.

[0064] The data transformation can be achieved through at least one of linear and nonlinear mappings, and the resulting value can be used as the first candidate score, the second candidate score, or the text prediction score. The computational operations can include one or more combinations of addition, subtraction, and multiplication.

[0065] As can be seen from the above, on the one hand, the first fused data corresponds to multiple text segments, and the obtained first candidate score can reflect the scoring information at the text segment level, while the second fused data corresponds to the input text, and the obtained second candidate score can reflect the scoring information at the input text level and / or the language element level. Combining the first candidate score and the second candidate score can make text prediction more accurately from multiple levels. On the other hand, the first fused data corresponds to multiple text segments, while the second fused data corresponds to the input text. Fusing the two can improve the data representation ability, thereby improving the accuracy of score prediction.

[0066] In a specific implementation, based on the arrangement order among the multiple segment encoded data, information iteration updates can be performed on each segment encoded data to obtain the corresponding information enhancement data for each segment encoded data, and the information enhancement data in a specified order can be extracted as the first fused data.

[0067] As an optional example, the information iterative update may specifically include: concatenating and mapping the information enhancement data corresponding to the segment encoding data to be updated with the corresponding information enhancement data of the previous segment encoding data (for the first segment encoding data to be updated, it may be combined with preset information enhancement data) to obtain the corresponding information enhancement data of the segment encoding data. The specified order can be set according to specific circumstances, such as being the last element.

[0068] In specific implementation, to improve the data fusion effect of multiple segment encoded data, during the process of fusing multiple segment encoded data to obtain the first fused data, the data fusion of multiple segment encoded data can be achieved through a first neural network used for processing sequence data. Specifically, multiple segment encoded data can be input into the first neural network used for processing sequence data for iterative information updates to obtain information enhancement data corresponding to each segment encoded data; then, the first neural network outputs information enhancement data in a specified order as the first fused data.

[0069] As an optional example, the first neural network may specifically be: LSTM (Long Short Term Memory networks), GRU (Gated Recurrent Unit) neural network, or other deep learning neural networks that have the function of fusing multiple input data into one output data.

[0070] In specific implementation, to further improve the data fusion effect, after obtaining the corresponding information enhancement data for each segment's encoded data, operations can be performed on multiple information enhancement data to obtain the first fused data. The operation methods between multiple information enhancement data can be set according to specific circumstances, such as one or more combinations of weighted operations, averaging operations, and pooling operations.

[0071] In a specific implementation, if multiple information enhancement data are obtained through the first neural network, the first neural network can be set to output multiple information enhancement data, and then the multiple information enhancement data can be processed to obtain the first fused data.

[0072] In specific implementations, when the computational operation of multiple information enhancement data includes a weighted operation, the weight of each information enhancement data can be determined based on the importance of its corresponding text segment within the input text. For example, an attention mechanism algorithm can be used to measure the importance of each information enhancement data segment within the input text, thereby determining the weight of each information enhancement data.

[0073] In an alternative example, multiple segment encoded data can be processed by a first neural network (such as LSTM or GRU) to obtain information augmentation data corresponding to each segment encoded data. Then, through an attention mechanism algorithm, the weight of each information augmentation data can be determined, and a weighted operation can be performed on multiple information augmentation data.

[0074] Specifically, the attention mechanism algorithm may include mapping and normalizing multiple pieces of the information-enhanced data. The mapping and normalization functions specifically included in the attention mechanism algorithm can be selected according to the application scenario. For example, the mapping function of the attention mechanism algorithm may be:

[0075] αi = tanh(Qa × hi + ba);

[0076] Where hi represents the i-th augmented data, Qa and ba are preset hyperparameters; tanh is the bitangent tangent curve; αi represents the mapping value of the i-th augmented data;

[0077] The normalization function for the attention mechanism algorithm can be:

[0078] βi=e qa×αi / Σe qa×αj ;

[0079] Where qa is a preset hyperparameter; aj represents the mapping value of the j-th information augmentation data; and βi represents the weight of the ith information augmentation data.

[0080] In specific implementation, when the operation mode of multiple information enhancement data includes pooling operation, the pooling operation may specifically include at least one of max pooling operation, min pooling operation and average pooling operation.

[0081] In an alternative example, multiple segment encoded data can be processed by a first neural network (such as LSTM or GRU) to obtain information augmentation data corresponding to each segment encoded data. Then, max pooling can be performed on the multiple information augmentation data.

[0082] In practical implementation, to improve fusion efficiency, a first neural network with an attention mechanism can be used to process the segment encoded data. For example, the first neural network with an attention mechanism can be a Transformer neural network or a neural network based on a Transformer variant.

[0083] In specific implementations, to improve the data fusion effect of the sequence-encoded data, during the process of fusing the sequence-encoded data to obtain the second fused data, pooling operations can be used to achieve data fusion of the sequence-encoded data, thereby obtaining the second fused data at the input text level and / or language element level. The pooling operation based on the sequence-encoded data can include at least one of max pooling, min pooling, and average pooling operations.

[0084] In specific implementation, when performing data fusion on the sequence encoded data to obtain the second fused data, in order to increase the feature information of the second fused data, pooling operations can be performed on the text encoded sub-data and multiple language element encoded sub-data to obtain pooled data at the language element level, and the pooled data and the text encoded sub-data can be concatenated to obtain the second fused data corresponding to the input text level and the language element level.

[0085] In practice, the granularity of the segmentation can be set according to the length distribution of sentences in the input text. To further improve the accuracy of score prediction, multiple different granularities can be set, thereby dividing the input text according to the preset granularities to obtain multiple text segments corresponding to each granularity. Based on this, the multiple text segments of the multiple granularities can be encoded to obtain corresponding segment encoded data. Based on the multiple segment encoded data of the multiple granularities and the sequence encoded data, score prediction is performed to obtain the text prediction score.

[0086] Therefore, by using different granularities of segmentation, text fragments of different scales can be obtained, thereby providing richer and more comprehensive feature information for score prediction, which is conducive to improving the accuracy of text prediction scores.

[0087] In practical implementation, the specific process for score prediction based on multiple segment encoded data with multiple partitioning granularities and the sequence encoded data can be defined according to the specific application scenario and requirements. For example, the process of predicting the text prediction score based on multiple segment encoded data with multiple partitioning granularities and the sequence encoded data can include any of the following steps:

[0088] C) Perform data fusion on the multiple segment encoded data corresponding to each of the division granularities to obtain the first fused data corresponding to each of the division granularities, and perform data fusion on the sequence encoded data to obtain the second fused data. Then, based on the multiple first fused data and the second fused data corresponding to the multiple division granularities, perform score prediction to obtain the text prediction score.

[0089] D) Based on the multiple segment coding data corresponding to the multiple division granularities and the sequence coding data, perform data fusion to obtain third fused data, and perform score prediction based on the third fused data to obtain the text prediction score.

[0090] Furthermore, as some optional examples, the step of performing score prediction based on multiple first fused data and second fused data corresponding to multiple said division granularities to obtain the text prediction score may include any of the following:

[0091] C-1) Based on the multiple first fusion data corresponding to each of the division granularities, perform data transformation to obtain the first candidate score corresponding to each of the division granularities, and based on the second fusion data, perform data transformation to obtain the second candidate score, and based on the first candidate score and the second candidate score corresponding to the multiple division granularities, perform calculation operations to obtain the text prediction score;

[0092] C-2) Based on the first fusion data and the second fusion data corresponding to the multiple division granularities, perform calculation operations, and perform data transformation based on the calculation operation results to obtain the text prediction score.

[0093] In specific implementation, such as Figure 2 The diagram shows a flowchart of data encoding for text fragments according to an embodiment of this specification. In some embodiments of this specification, the data encoding based on each text fragment to obtain corresponding fragment encoded data may specifically include the following steps:

[0094] S21, perform a first encoding process on each language element in each of the text segments to obtain intermediate encoded data for each language element in each of the text segments.

[0095] As an optional example, the first encoding process may include context-independent vectorization operations. For example, the first encoding process may include word embedding operations. Thus, each language element in each of the text segments can be represented using corresponding vectorized data (i.e., intermediate encoded data).

[0096] S22, based on the intermediate encoded data of multiple language elements in each of the text segments, a second encoding process is performed to obtain the corresponding segment encoded data.

[0097] In specific implementations, as an optional example, the second encoding process may include context-dependent processing operations. For example, the context-dependent processing operations may include one or more combinations of steps such as concatenating, transposing, and performing operations on intermediate encoded data of multiple language elements, wherein the operations may specifically include one or more combinations of addition, subtraction, and multiplication. This specification does not limit this aspect in the embodiments.

[0098] As can be seen from the above, by performing a second encoding process on the intermediate encoded data of multiple language elements in each of the aforementioned text fragments, feature information at multiple levels (such as the feature information of each language element itself, as well as the feature information between multiple language elements) can be integrated, making the corresponding fragment encoded data polysemous, improving the representation ability of the corresponding fragment encoded data, and thus improving the data processing effect.

[0099] In practical implementation, based on the intermediate encoded data of multiple language elements in each text segment, a second encoding process can be performed iteratively multiple times to improve the effectiveness of the data and better represent the text segments. Specifically, after performing a first encoding process on each language element in each text segment to obtain the intermediate encoded data of each language element in each text segment, a second encoding process can be performed iteratively multiple times based on the intermediate encoded data of multiple language elements in each text segment. The first input is the intermediate encoded data, and subsequent inputs are the results of the previous second encoding process.

[0100] In practical implementation, to improve the data encoding effect, data encoding can be achieved through a second neural network used for processing sequence data. Specifically, each text segment can be input into the second neural network used for processing sequence data for data encoding to obtain the segment encoded data.

[0101] As an optional example, the second neural network may include: a deep learning neural network, such as at least one of CNN (Convolutional Neural Network) and LSTM (Long Short Term Memory Network).

[0102] Optionally, the second neural network may include a deep learning neural network with an attention mechanism, such as a Transformer neural network or a neural network based on a Transformer variant. Further, the neural network based on a Transformer variant may be a Longformer (The Long-Document Transformer, a transformer capable of handling long documents) neural network, a BERT (Bidirectional Encoder Representations from Transformers) neural network, or a neural network based on a BERT variant. More specifically, the neural network based on a BERT variant may be an XLNet neural network (a neural network combining autoregressive and autoencoder functions) or a Roberta neural network (a neural network based on BERT for robust optimization), etc.

[0103] In specific implementation, such as Figure 3The diagram shown is a flowchart illustrating data encoding of input text according to an embodiment of this specification. In this embodiment, the step of encoding the input text to obtain sequence-encoded data may specifically include the following steps:

[0104] S31, perform a first encoding process on each language element in the input text to obtain intermediate encoded data for each language element in the input text.

[0105] The specific details of the first encoding process can be found in the descriptions of the relevant sections above, and will not be repeated here.

[0106] S32, based on the intermediate encoded data of multiple language elements in the input text, a second encoding process is performed to obtain the corresponding sequence encoded data.

[0107] In specific implementation, the details of the second encoding process can be found in the descriptions of the relevant sections above, and will not be repeated here.

[0108] As can be seen from the above, by performing a second encoding process on the intermediate encoded data of multiple language elements in the input text, feature information at multiple levels (such as the feature information of each language element itself, as well as the feature information between multiple language elements) can be integrated, making the corresponding sequence encoded data polysemous, improving the representation ability of the corresponding sequence encoded data, and helping to improve the data processing speed.

[0109] In practical implementation, based on the intermediate encoded data of multiple language elements in the input text, a second encoding process can be performed iteratively multiple times to improve the effectiveness of the data and better represent the input text. Specifically, after performing a first encoding process on each language element in the input text to obtain the intermediate encoded data of each language element in the input text, a second encoding process can be performed iteratively multiple times based on the intermediate encoded data of multiple language elements in the input text. The first input is the intermediate encoded data, and subsequent inputs are the results of the previous second encoding process.

[0110] In specific implementation, in order to improve the effect of data encoding, data encoding can be achieved through a third neural network used to process sequence data. Specifically, each language element of the input text can be input into the third neural network used to process sequence data for data encoding to obtain the sequence encoded data.

[0111] The specific type of the third neural network can be found in the description of the relevant part of the second neural network above, and will not be repeated here.

[0112] In practical implementation, to improve the accuracy of score prediction, a feedforward neural network can be used for data prediction. Specifically, based on the segment encoding data and sequence encoding data corresponding to multiple text segments, at least one of linear and nonlinear mappings can be performed using a feedforward neural network to obtain the predicted text score.

[0113] As an optional example, the feedforward neural network may include one or more combinations of fully connected feedforward neural networks and non-fully connected feedforward neural networks.

[0114] In practice, to improve the accuracy and precision of score prediction, at least one of the following can be adjusted: the granularity of the division, the parameters used to implement the data encoding, and the parameters used to implement the score prediction.

[0115] In practical implementation, the text scoring method provided in the embodiments of this specification can be applied to any professional field where text scoring is required. Correspondingly, the input text can be any text related to any field where text scoring is required. For example, the text scoring method provided in the embodiments of this specification can be applied to the fields of education, journalism, and finance. Accordingly, for the field of education, the input text can be essays, papers, etc.; for the field of journalism, the input text can be manuscripts, submitted articles, etc.; and for the field of finance, the input text can be research reports, analysis reports, etc.

[0116] To enable those skilled in the art to better understand and implement the embodiments of this specification, the following describes in detail how to perform text scoring through specific application scenarios.

[0117] In one embodiment of this specification, as Figure 4 The diagram shown is a flowchart of the text scoring method provided in this embodiment of the specification in a specific application scenario. The manuscript scoring platform can obtain the manuscript paper through input, loading, and other methods. By calling relevant parameters and program instructions, the following steps can be performed:

[0118] S1-1: The BERT segmenter decomposes the document Paper into m language elements. The preset granularity of the segmentation is k. Based on this granularity, p text fragments are obtained, namely text fragments Paper1 to Paper2. p Where m, k, and p are all positive integers, k is less than m, and p is the integer part of m divided by k.

[0119] For example, the first text fragment Paper1 = {t1……t} k}, and so on, the p-th text segment Paper p={t k*(p-1)+1 ...t m}. Where t1 and t k t k*(p-1)+1 t m These represent the 1st, kth, k*(p-1)+1th, and mth language elements, respectively.

[0120] S1-2, according to the input rules of the BERT neural network, process the text segments Paper1 to Paper2. p Perform preprocessing on the text fragments Paper1 to Paper2 respectively. p Add a leading identifier [CLS] before the first language element, and then add it to the text fragments Paper1 through Paper1 respectively. p Add a suffix [SEP] after the last language element. For example, see [link to relevant documentation]. Figure 4 The first text fragment, Paper1, becomes Paper1'={[CLS]t1……t after preprocessing. k [SEP]}, and so on, until the last text fragment, Paper. p After preprocessing, it becomes Paper. p '={[CLS]t k*(p-1)+1 ...t m [SEP]}.

[0121] S1-3, transfer the preprocessed text fragments Paper1' to Paper p 'Input the corresponding pre-trained language model 11-1 into the pre-trained language model 11-P respectively, perform the first encoding process and the second encoding process, and obtain the corresponding segment encoding data B. 11 To B 1p Among them, fragment encoded data B 11 This is the second encoding result of the first identifier [CLS] in the pre-trained language model 11-1, and so on, for the segment encoded data B. 1p This is the second encoding result of the first identifier [CLS] in the pre-trained language model 11-P.

[0122] Among them, pre-trained language models 11-1 to 11-P can be called during data encoding. Pre-trained language models 11-1 to 11-P each include a multi-layer Transformer neural network.

[0123] S1-4, encode the fragment data B 11 To B 1p The data is input into an LSTM 12, and then processed by the LSTM 12 to encode the fragment data B. 11 To B 1pThe information is iteratively updated to obtain fragment encoded data B. 11 To B 1p Corresponding information augmentation data C 11 To C 1p .

[0124] S1-5, Information Enhancement Data C 11 To C 1p In the input attention mechanism layer 13, the segment encoding data B is determined by the attention mechanism algorithm provided in the attention mechanism layer 13. 11 To B 1p The corresponding weights a1 to a p According to fragment encoding data B 11 To B 1p The corresponding weights a1 to a p Information augmentation data C 11 To C 1p Perform a weighted summation operation to obtain the first fused data D1.

[0125] S1-6, input the first fused data D1 into the fully connected neural network 14 to obtain the first candidate score E1.

[0126] S2-1, according to the input rules of the BERT neural network, preprocess the manuscript paper, obtain the first x language elements of the manuscript paper, add a first identifier [CLS] before the first language element, and add a last identifier [SEP] after the xth language element, to obtain the preprocessed manuscript paper' = {[CLS]t1……t x [SEP]}. Where x is a positive integer and not greater than the maximum number of language elements supported by the BERT neural network.

[0127] S2-2, Input the preprocessed manuscript Paper' into the corresponding pre-trained language model 21 for first and second encoding processing to obtain the corresponding sequence-encoded data B. 21 To B 2x Among them, sequence-coded data B 21 This is the second encoding result corresponding to the first language element in the pre-trained language model 21, and so on, for sequence-encoded data B. 2x This is the second encoding processing result corresponding to the xth language element in the pre-trained language model 21.

[0128] S2-3, the sequence encoded data B 21 To B 2x Input pooling layer 22, and obtain the second fused data D2 through the maximum pooling operation set in pooling layer 22.

[0129] S2-4, input the second fused data D2 into the fully connected neural network 23 to obtain the second candidate score E2.

[0130] S3-1, add the first candidate score E1 and the second candidate score E2 to obtain the text prediction score F1.

[0131] In another embodiment of this specification, such as Figure 5 The diagram shown is a flowchart of another specific application scenario of the text scoring method provided in the embodiments of this specification. The essay scoring platform can obtain the essay text through input, loading, and other methods. By calling relevant parameters and program instructions, the following steps can be performed:

[0132] S01, the essay text is decomposed using the BERT word segmenter to obtain n language elements, which are then divided according to s preset granularities, where the specific sizes of the s granularities are ka, kb, ..., ks. Based on multiple text fragments at each granularity, data encoding is performed to obtain corresponding fragment encoded data. Data fusion is then performed on the multiple fragment encoded data corresponding to each granularity to obtain the first fused data corresponding to each granularity. Based on the multiple first fused data corresponding to each granularity, data transformation is performed to obtain the first candidate score corresponding to each granularity. Thus, s first candidate scores, i.e., first candidate scores E, are obtained. 1a E 1b To E 1s .

[0133] For ease of understanding and description, the following illustration uses a partitioning granularity of ka. It can be understood that by referring to the illustration of a partitioning granularity of k, the scenarios of partitioning granularities of kb...ks can be deduced, which will not be elaborated here.

[0134] S01-1, based on a partitioning granularity of ka, divide the text into q text segments, i.e., text segments Text. a To Text q Where n, ka, and q are all positive integers, ka is less than n, and q is the integer part of n divided by ka. The first text segment Text1 = {t1……t} is obtained. ka}, and so on, the q-th text fragment Text q ={t ka*(q-1)+1 ...t n}. Where t1 and t ka t ka*(q-1)+1 t n These represent the 1st, ka, ka*(q-1)+1, and nth language elements, respectively.

[0135] S01-2, According to the input rules of the BERT neural network, process the text segments Text1 to Text2. q Perform preprocessing, respectively on text fragments Text1 to Text2. q Add a leading identifier [CLS] before the first language element, and then add it to the text fragments Text1 through Text2 respectively. q Add a suffix [SEP] after the last language element. For example, see [link to relevant documentation]. Figure 5 The first text fragment, Text1, becomes Text1'={[CLS]t1……t after preprocessing. ka [SEP]}, and so on, until the last text fragment is reached. q After preprocessing, it becomes Text. q '={[CLS]t ka*(q-1)+1 ...t n [SEP]}.

[0136] S01-3, transfer the preprocessed text fragments Text1' to Text... q Input the corresponding pre-trained language model A1-1 into the pre-trained language model A1-Q respectively, perform the first encoding process and the second encoding process, and obtain the corresponding segment encoding data B. 1a To B 1q Among them, fragment encoded data B 1a This is the second encoding result of the first identifier [CLS] in the pre-trained language model A1-1, and so on, for the segment encoded data B. 1q This is the second encoding result of the first identifier [CLS] in the pre-trained language model A1-Q.

[0137] Among them, the pre-trained language models A1-1 to A1-Q can be called during data encoding. The pre-trained language models A1-1 to A1-Q each consist of a multi-layer Transformer neural network.

[0138] S01-4, encode the fragment data B 1a To B 1q The data is input into LSTM A2, and then processed by LSTM A2 to encode the fragment data B. 1a To B 1q The information is iteratively updated to obtain fragment encoded data B. 1a To B 1q Corresponding information augmentation data C 1a To C 1q .

[0139] S01-5, Information Enhancement Data C 1a To C 1qIn the input attention mechanism layer A3, the segment encoding data B is determined through the attention mechanism algorithm provided in the attention mechanism layer A3. 1a To B 1q The corresponding weights b1 to b q According to fragment encoding data B 1a To B 1q The corresponding weights b1 to b q Information augmentation data C 1a To C 1q Perform a weighted summation operation to obtain the first fused data D. 1a .

[0140] S01-6, the first fused data D 1a The first candidate score E is obtained by inputting the fully connected neural network A4. 1a .

[0141] S02, according to the input rules of the BERT neural network, preprocess the essay Text, obtain the first y language elements of the essay Text, add a first identifier [CLS] before the first language element, and add a last identifier [SEP] after the y-th language element, to obtain the preprocessed essay Text' = {[CLS]t1……t y [SEP]}. Where x is a positive integer and not greater than the maximum number of language elements in the BERT neural network.

[0142] The preprocessed essay Text' is input into the corresponding pre-trained language model B1 for first and second encoding processing to obtain the corresponding sequence-encoded data B. b1 B b2 ...B by B by+1 Among them, sequence-coded data B b0 The second encoding result of the first identifier [CLS] in the pre-trained language model B1 is the sequence-encoded data B. b1 The second encoding result corresponding to the first language element, and so on, sequence-encoded data B by The sequence-encoded data B is the second encoding result corresponding to the y-th language element in the pre-trained language model B1. by+1 This is the second encoding result of the first identifier [CLS] in the pre-trained language model B1.

[0143] Then, the sequence-encoded data B b0 To B by+1 Input pooling layer B2, and obtain pooled data C through the max pooling operation set in pooling layer B2. b And pooled data C b and sequence-encoded data Bb0 By splicing the data together, we obtain the second fused data D. b Then the second fused data D b The second candidate score E is obtained by inputting the fully connected neural network B3. 2a .

[0144] S03, the first candidate score E 1a To E 1s and the second candidate score E 2a The scores are added together to obtain the text prediction score F. a .

[0145] It is understood that the above examples are merely illustrative. This specification provides multiple implementation schemes, which can be combined and cross-referenced without conflict, thereby extending into multiple possible implementation schemes. These can all be considered as the implementation schemes disclosed in the embodiments of this specification.

[0146] This specification also provides a text scoring model corresponding to the above-described text scoring method, which will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the text scoring model described below can be considered as a functional module required to implement the text scoring method provided in this specification; the content of the text scoring model described below can be referred to in correspondence with the content of the text scoring method described above.

[0147] In specific implementation, such as Figure 6 The diagram shown is a structural block diagram of a text scoring model provided in an embodiment of this specification. Figure 6 In this context, the text scoring model M1 may include:

[0148] Encoder M11 is adapted to divide input text into multiple text segments according to a preset granularity, and to perform data encoding on each text segment to obtain corresponding segment encoding data; and to perform data encoding on the input text to obtain sequence encoding data; wherein the sequence encoding data includes: text encoding sub-data and / or multiple language element encoding sub-data.

[0149] The scorer M12 is adapted to predict scores based on the segment encoding data and the sequence encoding data corresponding to the multiple text segments, and to obtain a text prediction score.

[0150] As can be seen from the above, the encoder, based on the sentence length and a fixed granularity of the input text, may divide a sentence into the same text segment or into multiple different text segments. Therefore, unlike segmentation based on punctuation marks, the text segments in the embodiments of this specification are divided at a consistent scale, and may contain only part of the language elements of a sentence.

[0151] Text fragments that include at least one sentence and text fragments that include at least one sentence and a portion of the language elements of a sentence can retain the feature information of each sentence itself (such as the semantics, grammar, pragmatics, and contextual relationships of the sentence itself) and the feature information of a portion of the language elements (such as the semantics, grammar, pragmatics, and contextual relationships of that portion of the language elements). They can also retain the feature information between sentences and between portions of the language elements of sentences (such as the contextual relationships between sentences and between portions of the language elements of sentences). Text fragments that only include a portion of the language elements of a sentence can better highlight the feature information of that portion of the language elements.

[0152] Based on this, dividing text according to a fixed granularity can avoid the fragmentation problem caused by too many punctuation marks and the loss of feature data caused by excessive sentence length during processing. The segment encoded data obtained after data encoding can contain rich feature information at the text segment level, thereby improving the representation ability of the segment encoded data. Multiple valid segment encoded data can reflect the correlation between multiple text segments.

[0153] Furthermore, since the sequence-encoded data obtained through data encoding can contain feature information at the input text level and / or feature information at the language element level, combining multiple fragment encoding data and the sequence-encoded data can provide the scorer with multi-level, rich, and useful feature information, thereby improving the accuracy of score prediction.

[0154] In practice, when dividing the input text into segments with a fixed granularity, all segments except the last one contain the same number of language elements. However, the last segment may contain fewer language elements than the granularity. Therefore, for the last segment, the encoder can use a specified symbol as a placeholder language element to ensure that the number of language elements in the subsequent segment reaches the granularity, thus guaranteeing a consistent number of language elements across all segments.

[0155] In practical implementation, the specific score prediction process of the scorer can be set according to the specific application scenario and requirements. For example, the scorer can perform any of the following steps:

[0156] A) Perform data fusion on multiple segment encoded data to obtain first fused data, and perform data fusion on the sequence encoded data to obtain second fused data, and perform score prediction based on the first fused data and the second fused data to obtain the text prediction score.

[0157] Specifically, as an optional example, data fusion based on multiple fragment coded data may include the steps of updating and summarizing information on the multiple fragment coded data; information updating and summarizing can be achieved through steps of splicing, mapping, and arithmetic operations. The arithmetic operations may include one or more combinations of addition, subtraction, multiplication, etc. This specification does not limit the specific process of data fusion based on fragment coded data in the embodiments.

[0158] As an optional example, data fusion based on sequence-coded data may include the steps of updating and summarizing information on text-coded sub-data and / or multiple language element-coded sub-data in the sequence-coded data; information updating and summarizing can be achieved through steps of concatenation, mapping, and arithmetic operations. Arithmetic operations may include at least one of addition, subtraction, multiplication, etc. The embodiments in this specification do not limit the specific process of data fusion based on sequence-coded data.

[0159] As an optional example, score prediction based on the first fused data and the second fused data may include the step of directly or indirectly obtaining a numerical value by combining the first fused data and the second fused data.

[0160] B) Based on the multiple fragment encoded data and the sequence encoded data, data fusion is performed to obtain third fused data, and score prediction is performed based on the third fused data to obtain the text prediction score.

[0161] Specifically, as an optional example, data fusion based on multiple fragment encoded data and the sequence encoded data may include the steps of updating and summarizing information by combining multiple fragment encoded data and the sequence encoded data; information updating and summarizing can be achieved through steps of splicing, mapping, and arithmetic operations. The arithmetic operations may include one or more combinations of addition, subtraction, multiplication, etc. This specification does not limit the specific process of data fusion based on multiple fragment encoded data and the sequence encoded data in the embodiments.

[0162] As an optional example, score prediction based on the third fusion data may include the step of converting the third fusion data from a vector to a numerical value.

[0163] Therefore, the scorer can update the feature information of the segment-coded data and / or the sequence-coded data through data fusion, retain important feature information, forget unimportant feature information, and obtain the correlation between feature information, thereby enhancing the effectiveness of the data, improving the data representation ability, providing rich and useful feature information for score prediction, and thus improving the efficiency and accuracy of score prediction.

[0164] In practical implementation, the specific steps of the scorer in predicting scores based on the first fused data and the second fused data can be set according to the specific application scenario and requirements. For example, the scorer can perform any of the following steps:

[0165] A-1) Based on the first fused data, perform data transformation to obtain a first candidate score, and based on the second fused data, perform data transformation to obtain a second candidate score, and based on the first candidate score and the second candidate score, perform calculation operations to obtain the text prediction score.

[0166] A-2) Based on the first fused data and the second fused data, perform calculation operations, and based on the results of the calculation operations, perform data transformation to obtain the text prediction score.

[0167] The data transformation can be achieved through at least one of linear and nonlinear mappings, and the resulting value can be used as the first candidate score, the second candidate score, or the text prediction score. The computational operations can include one or more combinations of addition, subtraction, and multiplication.

[0168] As can be seen from the above, on the one hand, the first fused data corresponds to multiple text fragments, and the first candidate score obtained by the scorer can reflect the scoring information at the text fragment level, while the second fused data corresponds to the input text, and the second candidate score obtained by the scorer can reflect the scoring information at the input text level and / or the language element level. By combining the first candidate score and the second candidate score, the scorer can make more accurate text predictions from multiple levels. On the other hand, the first fused data corresponds to multiple text fragments, while the second fused data corresponds to the input text. The scorer can fuse the two, thereby improving the data representation ability and thus improving the accuracy of score prediction.

[0169] In practice, the granularity of the segmentation can be set according to the length distribution of each sentence in the input text. To further improve the accuracy of score prediction, multiple different granularities can be set, so that the encoder can divide the input text according to the preset granularities to obtain multiple text segments corresponding to each granularity.

[0170] Based on this, the encoder can encode multiple text segments with the same granularity, and the scorer can predict scores based on the multiple text segments with each granularity to obtain multiple candidate scores. Then, the multiple candidate scores are weighted to obtain the predicted text score. The encoder can also encode multiple text segments with different granularities, and the scorer can predict scores based on the encoder's encoding results to obtain the predicted text score.

[0171] Therefore, by using different granularities of segmentation, multiple text fragments of different scales can be obtained, thereby providing the scorer with richer and more comprehensive feature information, which is conducive to further improving the accuracy of text prediction scores.

[0172] In practical implementation, the specific steps of the scorer in predicting scores based on multiple segment encoded data with multiple partitioning granularities and the sequence encoded data can be set according to the specific application scenario and requirements. For example, the scorer can perform any of the following steps:

[0173] C) Perform data fusion on the multiple segment encoded data corresponding to each of the division granularities to obtain the first fused data corresponding to each of the division granularities, and perform data fusion on the sequence encoded data to obtain the second fused data. Then, based on the multiple first fused data and the second fused data corresponding to the multiple division granularities, perform score prediction to obtain the text prediction score.

[0174] D) Based on the multiple segment coding data corresponding to the multiple division granularities and the sequence coding data, perform data fusion to obtain third fused data, and perform score prediction based on the third fused data to obtain the text prediction score.

[0175] In practical implementation, the specific process for score prediction based on multiple first fusion data and second fusion data corresponding to multiple division granularities can be set according to specific application scenarios and needs. For example, the scorer can perform any of the following steps:

[0176] C-1) Based on the multiple first fusion data corresponding to each of the division granularities, perform data transformation to obtain the first candidate score corresponding to each of the division granularities, and based on the second fusion data, perform data transformation to obtain the second candidate score, and based on the first candidate score and the second candidate score corresponding to the multiple division granularities, perform calculation operations to obtain the text prediction score;

[0177] C-2) Based on the first fusion data and the second fusion data corresponding to the multiple division granularities, perform calculation operations, and perform data transformation based on the calculation operation results to obtain the text prediction score.

[0178] It is understood that the function, principle, and specific implementation of the encoder in the text scoring model provided in the embodiments of this specification can be referred to the descriptions in the relevant sections above, and will not be repeated here. Similarly, the function, principle, and specific implementation of the scorer in the text scoring model provided in the embodiments of this specification can be referred to the descriptions in the relevant sections above, and will not be repeated here.

[0179] This specification also provides a text scoring device corresponding to the above-described text scoring method, which will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the text scoring device described below can be considered as a functional module required to implement the text scoring method provided in this specification; the content of the text scoring device described below can be referred to in correspondence with the content of the text scoring method described above.

[0180] In specific implementation, such as Figure 7 The diagram shown is a structural block diagram of a text scoring device provided in an embodiment of this specification. Figure 7 In this context, the text scoring device M2 may include:

[0181] The segmentation module M21 is suitable for dividing the input text according to a preset segmentation granularity to obtain multiple text segments;

[0182] The data encoding module M22 is adapted to perform data encoding based on each of the text fragments to obtain corresponding fragment encoded data; and to perform data encoding based on the input text to obtain sequence encoded data; wherein the sequence encoded data includes: text encoded sub-data and / or multiple language element encoded sub-data;

[0183] The score prediction module M23 is adapted to perform score prediction based on the segment encoding data and the sequence encoding data corresponding to the multiple text segments, and obtain the text prediction score.

[0184] As can be seen from the above, the data encoding module, based on the sentence length and fixed-size granularity of the input text, may divide a sentence into the same text segment or into multiple different text segments. Therefore, unlike segmentation based on punctuation marks, the text segmentation scale in the embodiments of this specification is consistent, and there may be cases where a portion of the language elements of a sentence are included.

[0185] Text fragments that include at least one sentence and text fragments that include at least one sentence and a portion of the language elements of a sentence can retain the feature information of each sentence itself (such as the semantics, grammar, pragmatics, and contextual relationships of the sentence itself) and the feature information of a portion of the language elements (such as the semantics, grammar, pragmatics, and contextual relationships of that portion of the language elements). They can also retain the feature information between sentences and between portions of the language elements of sentences (such as the contextual relationships between sentences and between portions of the language elements of sentences). Text fragments that only include a portion of the language elements of a sentence can better highlight the feature information of that portion of the language elements.

[0186] Based on this, dividing text according to a fixed granularity can avoid the fragmentation problem caused by too many punctuation marks and the loss of feature data caused by excessive sentence length during processing. The segment encoded data obtained after data encoding can contain rich feature information at the text segment level, thereby improving the representation ability of the segment encoded data. Multiple valid segment encoded data can reflect the correlation between multiple text segments.

[0187] Furthermore, since the sequence encoded data obtained through data encoding can contain feature information at the input text level and / or feature information at the language element level, combining multiple segment encoded data and the sequence encoded data can provide the score prediction module with multi-level, rich and useful feature information, thereby improving the accuracy of score prediction.

[0188] In practice, when dividing the input text into segments with a fixed granularity, all segments except the last one contain the same number of language elements. However, the last segment may contain fewer language elements than the granularity. Therefore, for the last segment, the segmentation module can use a designated symbol as a placeholder language element to ensure that the number of language elements in the subsequent segment reaches the granularity, thus guaranteeing a consistent number of language elements across all segments.

[0189] In practical implementation, the specific score prediction process of the score prediction module can be set according to the specific application scenario and requirements. For example, the score prediction module can perform any of the following steps:

[0190] A) Perform data fusion on multiple segment encoded data to obtain first fused data, and perform data fusion on the sequence encoded data to obtain second fused data, and perform score prediction based on the first fused data and the second fused data to obtain the text prediction score.

[0191] Specifically, as an optional example, data fusion based on multiple fragment coded data may include the steps of updating and summarizing information on the multiple fragment coded data; information updating and summarizing can be achieved through steps of splicing, mapping, and arithmetic operations. The arithmetic operations may include one or more combinations of addition, subtraction, multiplication, etc. This specification does not limit the specific process of data fusion based on fragment coded data in the embodiments.

[0192] As an optional example, data fusion based on sequence-coded data may include the steps of updating and summarizing information on text-coded sub-data and / or multiple language element-coded sub-data in the sequence-coded data; information updating and summarizing can be achieved through steps of concatenation, mapping, and arithmetic operations. Arithmetic operations may include at least one of addition, subtraction, multiplication, etc. The embodiments in this specification do not limit the specific process of data fusion based on sequence-coded data.

[0193] As an optional example, score prediction based on the first fused data and the second fused data may include the step of directly or indirectly obtaining a numerical value by combining the first fused data and the second fused data.

[0194] B) Based on the multiple fragment encoded data and the sequence encoded data, data fusion is performed to obtain third fused data, and score prediction is performed based on the third fused data to obtain the text prediction score.

[0195] Specifically, as an optional example, data fusion based on multiple fragment encoded data and the sequence encoded data may include the steps of updating and summarizing information by combining multiple fragment encoded data and the sequence encoded data; information updating and summarizing can be achieved through steps of splicing, mapping, and arithmetic operations. The arithmetic operations may include one or more combinations of addition, subtraction, multiplication, etc. This specification does not limit the specific process of data fusion based on multiple fragment encoded data and the sequence encoded data in the embodiments.

[0196] As an optional example, score prediction based on the third fusion data may include the step of converting the third fusion data from a vector to a numerical value.

[0197] Therefore, the score prediction module can update the feature information of the segment-coded data and / or the sequence-coded data through data fusion, retain important feature information, forget unimportant feature information, and obtain the correlation between feature information, thereby improving the data representation ability, providing rich and useful feature information for score prediction, and thus improving the efficiency and accuracy of score prediction.

[0198] In practical implementation, the specific steps of the score prediction module in predicting scores based on the first fused data and the second fused data can be set according to the specific application scenario and requirements. For example, the score prediction module can perform any of the following steps:

[0199] A-1) Based on the first fused data, perform data transformation to obtain a first candidate score, and based on the second fused data, perform data transformation to obtain a second candidate score, and based on the first candidate score and the second candidate score, perform calculation operations to obtain the text prediction score.

[0200] A-2) Based on the first fused data and the second fused data, perform calculation operations, and based on the results of the calculation operations, perform data transformation to obtain the text prediction score.

[0201] The data transformation can be achieved through at least one of linear and nonlinear mappings, and the resulting value can be used as the first candidate score, the second candidate score, or the text prediction score. The computational operations can include one or more combinations of addition, subtraction, and multiplication.

[0202] As can be seen from the above, on the one hand, the first fused data corresponds to multiple text fragments, and the first candidate score obtained by the score prediction module can reflect the scoring information at the text fragment level, while the second fused data corresponds to the input text, and the second candidate score obtained by the score prediction module can reflect the scoring information at the input text level and / or the language element level. By combining the first candidate score and the second candidate score, the score prediction module can make more accurate text predictions from multiple levels. On the other hand, the first fused data corresponds to multiple text fragments, while the second fused data corresponds to the input text. The score prediction module merges the two, thereby improving the data representation ability and thus improving the accuracy of score prediction.

[0203] In practical implementation, the granularity of the segmentation can be set according to the length distribution of each sentence in the input text. To further improve the accuracy of score prediction, multiple different granularities can be set. Thus, the segmentation module can divide the input text according to the preset granularities, obtaining multiple text segments corresponding to each granularity. Based on this, the data encoding module can encode multiple text segments with the same granularity, and the score prediction module can predict scores based on the multiple text segments with each granularity, thereby obtaining multiple candidate scores. These candidate scores are then weighted to obtain the predicted text score. Alternatively, the data encoding module can encode multiple text segments with different granularities, and the score prediction module can predict scores based on the encoder's encoding results, thereby obtaining the predicted text score.

[0204] Therefore, by using different granularities of division, multiple text fragments of different scales can be obtained, thereby providing richer and more comprehensive feature information for the score prediction module, which is conducive to further improving the accuracy of text prediction scores.

[0205] In practical implementation, the specific steps of the score prediction module in predicting scores based on multiple segment encoded data with multiple partitioning granularities and the sequence encoded data can be set according to specific application scenarios and requirements. For example, the score prediction module can perform any of the following steps:

[0206] C) Perform data fusion on the multiple segment encoded data corresponding to each of the division granularities to obtain the first fused data corresponding to each of the division granularities, and perform data fusion on the sequence encoded data to obtain the second fused data. Then, based on the multiple first fused data and the second fused data corresponding to the multiple division granularities, perform score prediction to obtain the text prediction score.

[0207] D) Based on the multiple segment coding data corresponding to the multiple division granularities and the sequence coding data, perform data fusion to obtain third fused data, and perform score prediction based on the third fused data to obtain the text prediction score.

[0208] In practical implementation, the specific process for score prediction based on multiple first fusion data and second fusion data corresponding to multiple division granularities can be set according to specific application scenarios and requirements. For example, the score prediction module can perform any of the following steps:

[0209] C-1) Based on the multiple first fusion data corresponding to each of the division granularities, perform data transformation to obtain the first candidate score corresponding to each of the division granularities, and based on the second fusion data, perform data transformation to obtain the second candidate score, and based on the first candidate score and the second candidate score corresponding to the multiple division granularities, perform calculation operations to obtain the text prediction score;

[0210] C-2) Based on the first fusion data and the second fusion data corresponding to the multiple division granularities, perform calculation operations, and perform data transformation based on the calculation operation results to obtain the text prediction score.

[0211] It is understood that the functions, principles, and specific implementation methods of the data encoding module in the text scoring model provided in the embodiments of this specification can be referred to the descriptions in the relevant sections above, and will not be repeated here. Similarly, the functions, principles, and specific implementation methods of the score prediction module in the text scoring model provided in the embodiments of this specification can be referred to the descriptions in the relevant sections above, and will not be repeated here.

[0212] It is also understood that the above examples are merely illustrative. This specification provides multiple implementation schemes, which can be combined and cross-referenced without conflict, thereby extending into multiple possible implementation schemes. These can all be considered as the implementation schemes disclosed in the embodiments of this specification.

[0213] In specific implementation, such as Figure 8 The diagram shown is a structural block diagram of another text scoring device provided in an embodiment of this specification. Figure 8 In the text scoring device M3, there may be a memory M31 and a processor M32. The memory M31 and the processor M32 can communicate with each other through a communication bus M33. The memory M31 stores computer instructions that can be executed on the processor M32. When the processor M32 executes the computer instructions, it can perform the steps of the text scoring method described in any of the above embodiments. For details, please refer to the above-mentioned related content, which will not be repeated here.

[0214] In specific implementations, the processor may include a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), etc. The memory may include Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Memory (NVM), etc.

[0215] In practice, computer instructions may include any suitable type of code implemented using any appropriate high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, encrypted code, etc.

[0216] In specific implementation, such as Figure 8 As shown, the text scoring device M3 may further include a display interface M34 and a display M35 connected through the display interface M34. The display interface M34 can communicate with the memory M31 and the processor M32 via the communication bus M33. The display M35 can display the correction results obtained by the processor M32 executing the text scoring method provided in the embodiments of this specification.

[0217] This specification also provides a computer-readable storage medium storing computer instructions that, when executed, can perform the steps of the text scoring method described in any of the above embodiments of this specification. For details, please refer to the relevant content above, which will not be repeated here.

[0218] The computer-readable storage medium may include any suitable type of memory cell, memory device, memory article, memory medium, storage device, storage article, storage medium, and / or storage cell. Examples include memory, removable or non-removable media, erasable or non-erasable media, writable or rewritable media, digital or analog media, hard disk, floppy disk, optical disk, etc. Furthermore, the computer instructions may include any suitable type of code implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, encrypted code, etc.

[0219] It is understood that, unless otherwise expressly specified and limited, the terminology used in this specification may be interpreted according to different application scenarios. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. Moreover, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein.

[0220] While the embodiments disclosed in this specification are as described above, they are not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the embodiments in this specification. Therefore, the scope of protection of the embodiments in this specification should be determined by the scope defined in the claims.

Claims

1. A text scoring method, characterized in that, include: The input text is divided into multiple text segments according to a preset granularity. Based on each of the aforementioned text fragments, data encoding is performed to obtain the corresponding fragment encoding data; Based on the input text, data encoding is performed to obtain sequence encoded data; wherein, the sequence encoded data includes: text encoded sub-data and / or multiple language element encoded sub-data; Based on the segment encoding data and sequence encoding data corresponding to multiple text segments, a score prediction is performed to obtain a text prediction score; the step of performing score prediction based on the segment encoding data and sequence encoding data corresponding to multiple text segments to obtain a text prediction score includes any one of the following: Data fusion is performed on multiple segment encoded data to obtain first fused data, and data fusion is performed on the sequence encoded data to obtain second fused data. Based on the first fused data and the second fused data, score prediction is performed to obtain the text prediction score. Based on multiple fragment encoded data and sequence encoded data, data fusion is performed to obtain third fused data, and score prediction is performed based on the third fused data to obtain the text prediction score; The step of fusing multiple segment encoded data to obtain first fused data includes: based on the arrangement order among the multiple segment encoded data, performing information iterative updates on each segment encoded data to obtain corresponding information enhancement data for each segment encoded data; The first fused data is obtained by performing calculations on multiple information enhancement data.

2. The text scoring method according to claim 1, characterized in that, The step of performing score prediction based on the first fused data and the second fused data to obtain the text prediction score includes any one of the following: Based on the first fused data, data transformation is performed to obtain a first candidate score, and based on the second fused data, data transformation is performed to obtain a second candidate score. Based on the first candidate score and the second candidate score, calculation operations are performed to obtain the text prediction score. Based on the first fused data and the second fused data, computational operations are performed, and data transformation is performed based on the results of the computational operations to obtain the text prediction score.

3. The text scoring method according to claim 1, characterized in that, The step of iteratively updating information on each of the multiple segment encoded data to obtain corresponding information enhancement data for each segment encoded data includes: By using a first neural network for processing sequence data, information is iteratively updated for each of the multiple segment encoded data to obtain information enhancement data corresponding to each segment encoded data.

4. The text scoring method according to claim 1, characterized in that, The operation on the multiple information enhancement data includes: Based on the importance of the corresponding text fragments of each information enhancement data in the input text, the weight of each information enhancement data is determined, and a weighted operation is performed on multiple information enhancement data.

5. The text scoring method according to claim 1, characterized in that, The operation on the multiple information enhancement data includes: The weights of each information-enhancing data point are determined using an attention mechanism algorithm, and a weighted operation is performed on multiple information-enhancing data points.

6. The text scoring method according to any one of claims 1-5, characterized in that, The input text is divided according to a preset granularity to obtain multiple text segments, including: The input text is broken down to obtain multiple language elements arranged in sequence; The multiple language elements are divided according to a preset granularity, and the multiple language elements divided into the same group are taken as a text segment to obtain multiple text segments.

7. The text scoring method according to any one of claims 1-5, characterized in that, The input text is divided according to a preset granularity to obtain multiple text segments, including: The input text is divided into multiple preset division granularities to obtain multiple text fragments corresponding to each division granularity.

8. The text scoring method according to claim 7, characterized in that, The process of processing data based on the sequence-encoded data and the corresponding segment-encoded data of multiple text segments to obtain a text prediction score includes: Data fusion is performed on the multiple segment encoded data corresponding to each of the aforementioned division granularities to obtain the first fused data corresponding to each of the aforementioned division granularities; The sequence-encoded data is fused to obtain second fused data; Based on the first fusion data and the second fusion data corresponding to multiple division granularities, score prediction is performed to obtain the text prediction score.

9. The text scoring method according to any one of claims 1-5, characterized in that, The process of encoding data based on each of the text fragments to obtain corresponding fragment encoded data includes: Each language element in each of the text segments is subjected to a first encoding process to obtain intermediate encoded data of each language element in each of the text segments; Based on the intermediate encoded data of multiple language elements in each of the text segments, a second encoding process is performed to obtain the corresponding segment encoded data.

10. The text scoring method according to claim 9, characterized in that, The second encoding process, based on intermediate encoded data of multiple language elements in each of the text segments, includes: Based on the intermediate encoded data of multiple language elements in each of the text segments, a second encoding process is performed in multiple iterations, wherein the first input is the intermediate encoded data, and subsequent inputs are the results of the previous second encoding process.

11. The text scoring method according to any one of claims 1-5, characterized in that, The process of encoding data based on each of the text fragments to obtain corresponding fragment encoded data includes: The text segments are encoded using a second neural network for processing sequence data to obtain the segment encoded data.

12. The text scoring method according to any one of claims 1-5, characterized in that, The score prediction includes at least one of linear and nonlinear mappings.

13. The text scoring method according to any one of claims 1-5, characterized in that, The score prediction is performed using a feedforward neural network.

14. A text scoring model, characterized in that, include: An encoder is suitable for dividing input text into multiple text segments according to a preset granularity, and for encoding data based on each text segment to obtain corresponding segment encoded data. Furthermore, based on the input text, data encoding is performed to obtain sequence encoded data; wherein, the sequence encoded data includes: text encoded sub-data and / or multiple language element encoded sub-data; A scorer is adapted to predict a text prediction score based on segment encoding data corresponding to multiple text segments and sequence encoding data; the step of predicting a text prediction score based on segment encoding data corresponding to multiple text segments and sequence encoding data includes any one of the following: Data fusion is performed on multiple segment encoded data to obtain first fused data, and data fusion is performed on the sequence encoded data to obtain second fused data. Based on the first fused data and the second fused data, score prediction is performed to obtain the text prediction score. Based on multiple fragment encoded data and sequence encoded data, data fusion is performed to obtain third fused data, and score prediction is performed based on the third fused data to obtain the text prediction score; The step of fusing multiple segment encoded data to obtain first fused data includes: based on the arrangement order among the multiple segment encoded data, performing information iterative updates on each segment encoded data to obtain corresponding information enhancement data for each segment encoded data; The first fused data is obtained by performing calculations on multiple information enhancement data.

15. A text scoring device, characterized in that, include: The segmentation module is suitable for dividing the input text according to a preset segmentation granularity to obtain multiple text fragments; The data encoding module is adapted to encode data based on each of the text fragments to obtain corresponding fragment encoded data; Furthermore, based on the input text, data encoding is performed to obtain sequence encoded data; wherein, the sequence encoded data includes: text encoded sub-data and / or multiple language element encoded sub-data; The score prediction module is adapted to perform score prediction based on the segment encoding data corresponding to multiple text segments and the sequence encoding data to obtain a text prediction score; the step of performing score prediction based on the segment encoding data corresponding to multiple text segments and the sequence encoding data to obtain a text prediction score includes any one of the following: Data fusion is performed on multiple segment encoded data to obtain first fused data, and data fusion is performed on the sequence encoded data to obtain second fused data. Based on the first fused data and the second fused data, score prediction is performed to obtain the text prediction score. Based on multiple fragment encoded data and sequence encoded data, data fusion is performed to obtain third fused data, and score prediction is performed based on the third fused data to obtain the text prediction score; The step of fusing multiple segment encoded data to obtain first fused data includes: based on the arrangement order among the multiple segment encoded data, performing information iterative updates on each segment encoded data to obtain corresponding information enhancement data for each segment encoded data; The first fused data is obtained by performing calculations on multiple information enhancement data.

16. A text scoring device, comprising a memory and a processor; wherein, The memory is adapted to store one or more computer instructions, characterized in that, when the processor executes the computer instructions, it performs the steps of the text scoring method according to any one of claims 1 to 13.

17. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed, they perform the steps of the text scoring method according to any one of claims 1 to 13.

Images