Text readability classification method, system and equipment based on comparative learning and medium

Abstract

The invention provides a text readability classification method, system and device based on comparative learning and a medium, and relates to the field of natural language process.The method comprises the steps that an obtained first target text is input into a preset BERT model, each word in the first target text is converted into vector representation through word embedding, and a plurality of word vectors are obtained; encoding the plurality of word vectors to determine an initial feature vector; extracting the most important word information of each token through maximum pooling to obtain a word level feature vector sequence; and training the model based on comparative learning to obtain a mature target classifier. According to the method, a comparative learning framework training model is introduced, the understanding of the whole sentence and the coding of semantic information are considered in the training process, and the detail information of the word level is also considered, so that the recognition capability of the model on the readability nuisance is enhanced; the general representation of the text is learned based on comparative learning, and the transportability of the feature set of the model in different data sets and application scenes is improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a text readability classification method, system, device and medium based on contrastive learning. Background Art

[0002] Automatic analysis of text readability aims to link text to the reader's reading level by quantifying the complexity of the text. This task has been applied to fields such as graded reading, information retrieval, and text simplification. If a book is too difficult, readers will lose interest in reading because they cannot understand the content. If a book is too simple, it will not significantly improve the reader's reading ability. It is crucial to choose books that are suitable for the reader's reading ability. Measuring the difficulty of a book requires text readability assessment.

[0003] In related technologies, text readability classification is mainly carried out through formula methods; the readability formula method calculates the readability score of the text through mathematical formulas. These formulas are usually based on specific features of the text, such as sentence length, vocabulary difficulty, and the frequency of use of complex grammatical structures. The formula method extracts features of the readability text to be evaluated, such as word length, sentence length, etc.; then performs feature selection and model fitting, and uses the extracted features to establish a readability formula. The output result of the formula is the readability score.

[0004] However, the readability formula method has the following disadvantages when evaluating text readability: First, the method relies on a limited number of features, mainly focusing on basic language features such as sentence length and vocabulary complexity, which makes it difficult to fully capture the deep semantics and contextual information of the text. Second, due to the small amount of information captured by the features, the accuracy of its evaluation is relatively low, and it cannot accurately reflect the reading experience and comprehension difficulty of different readers. In addition, most readability formulas are developed based on specific corpora, which means that they may not be able to adapt to diverse text types and styles, or maintain their prediction accuracy in new contexts. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present application provides a text readability classification method, system, device and medium based on contrastive learning, which solves the problems of large limitations and low accuracy in current text readability evaluation.

[0006] To achieve the above objectives, this application is implemented through the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a text readability classification method based on contrastive learning, which includes: inputting an acquired first target text into a preset BERT model, the first target text including labeled diverse text data, and the BERT model being a Transformer-based pre-trained language model for language understanding; the BERT model converts each word in the first target text into a vector representation through word embedding to obtain multiple word vectors; encoding the multiple word vectors through a Transformer encoder to obtain contextual representation information of each word, and determining an initial feature vector representing the output information of [CLS] token in the BERT model; extracting the most important word information of each token through maximum pooling to obtain a word-level feature vector sequence of the same length as the first target text; training the model based on contrastive learning according to the word-level feature vector sequence and the initial feature vector to obtain a mature target classifier for readability classification.

[0008] According to the first aspect of the embodiment of the present application, the aforementioned method of training a model based on contrastive learning according to a word-level feature vector sequence and an initial feature vector to obtain a mature target classifier for readability classification may include the following steps: integrating the word-level feature vector sequence with the initial feature vector to obtain a first target feature vector to represent the semantic information and vocabulary information corresponding to the first target text; inputting the first target feature vector into a classifier framework based on contrastive learning for model training; adjusting the hyperparameter distribution through a supervised contrastive learning loss function to obtain a mature target classifier through training.

[0009] According to the first aspect of the embodiment of the present application, the aforementioned integration of the word-level feature vector sequence and the initial feature vector to obtain a first target feature vector to represent the semantic information and lexical information corresponding to the first target text may include the following steps: inputting the word-level feature vector sequence and the initial feature vector into the attention layer of the BERT model; learning the interaction information between the word-level feature vector sequence and the initial feature vector based on the attention mechanism to capture the semantic information and lexical information of the text; determining the vector corresponding to the interaction information as the first target feature vector.

[0010] According to the first aspect of the embodiment of the present application, after the model is trained based on contrastive learning according to the word-level feature vector sequence and the initial feature vector to obtain a mature target classifier for readability classification, the text readability classification method based on contrastive learning may also include the following steps: obtaining a second target text to be readably classified; performing word conversion, encoding, maximum pooling and feature integration processing on the second target text through the BERT model to obtain a second target feature vector; processing the second target feature vector through the target classifier to obtain a classification result to characterize the readability level of the second target text.

[0011] According to the first aspect of the embodiment of the present application, the first target feature vector corresponds to multiple samples with label categories, and the label categories are used to characterize the readability difficulty of the text; during the training process of the target classifier, samples corresponding to the same label category are positive samples and are pulled close to each other, and samples corresponding to different label categories are negative samples and are pushed away from each other.

[0012] According to a first aspect of the embodiment of the present application, for an input sequence [x1, x2, ..., x n ], the maximum pooling operation satisfies the expression: WordPool([x1,x2,...,x n ])=max(x1,x2,...,x n ), where WordPool represents the maximum pooling operation, x i is the word-level feature vector of the i-th token.

[0013] According to the first aspect of the embodiment of the present application, the integration operation corresponding to the word-level feature vector sequence and the initial feature vector satisfies the expression:

[0014] Integration(x cls ,x word_pool )=Softmax(Attention(x cls ,x word_pool ))·x word_pool

[0015] In the formula, Integration{} represents the first target feature vector after integration, x cls is the initial eigenvector, x word_pool is a sequence of word-level feature vectors, Attention{} is the attention function and is used to calculate the weight between the initial feature vector and each word-level feature vector, and the Softmax function is used to normalize the weight.

[0016] In a second aspect, an embodiment of the present application provides a text readability classification system based on contrastive learning, which includes: an input module, a conversion module, an encoding module, a pooling module and a training module; wherein the input module is used to input the acquired first target text into a preset BERT model, the first target text includes labeled diverse text data, and the BERT model is a pre-trained language model based on Transformer for language understanding; the conversion module is used for the BERT model to convert each word in the first target text into a vector representation through word embedding to obtain multiple word vectors; the encoding module is used to encode the multiple word vectors through a Transformer encoder to obtain contextual representation information of each word, and determine an initial feature vector representing the output information of [CLS] token in the BERT model; the pooling module is used to extract the most important word information of each token through maximum pooling to obtain a word-level feature vector sequence of the same length as the first target text; the training module is used to train the model based on contrastive learning according to the word-level feature vector sequence and the initial feature vector to obtain a mature target classifier for readability classification.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the text readability classification method based on contrastive learning in the aforementioned first aspect is implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a program or instruction. When the program or instruction is executed by a processor, the text readability classification method based on contrastive learning in the aforementioned first aspect is implemented.

[0019] This application provides a text readability classification method, system, device and medium based on contrastive learning. Compared with the prior art, it has the following beneficial effects:

[0020] The present application is trained by diverse text data in the first target text, which can ensure the generalization ability of the target classifier and adapt to different text types and styles. The present application uses the BERT model to process the first target text, which can capture the deep semantic features and contextual information of the text, and determine the initial feature vector that represents the output information of the [CLS] token in the BERT model, which goes beyond the surface feature extraction of traditional methods; in addition, the BERT model also extracts the most important word information of each token through maximum pooling to obtain a word-level feature vector sequence; further, based on the semantic information and lexical information contained in the word-level feature vector sequence and the initial feature vector, a contrastive learning framework training model is introduced. The training process considers not only the understanding of the entire sentence and the encoding of semantic information, but also the detailed information at the word level, thereby enhancing the model's ability to recognize subtle differences in readability; the present application learns a universal representation of text based on contrastive learning, which improves the portability of the model's feature set in different data sets and application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 It is a flowchart of a text readability classification method based on contrastive learning provided in an embodiment of the present application;

[0023] Figure 2 yes Figure 1 An exemplary process diagram of S150;

[0024] Figure 3 It is a schematic diagram of the BERT model structure provided in an embodiment of the present application;

[0025] Figure 4 It is a structural schematic diagram of a text readability classification system based on contrastive learning provided in an embodiment of the present application;

[0026] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0029] The embodiments of the present application provide a text readability classification method, system, device and medium based on contrastive learning, thereby solving the problems of large limitations and low accuracy in current text readability evaluation and realizing automatic and accurate evaluation and classification of text readability.

[0030] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0031] Automatic analysis of text readability aims to link text with the reader's reading level by quantifying the complexity of the text. This task has been applied to fields such as graded reading, information retrieval, and text simplification. Today's society attaches more and more importance to education and reading. By reading various excellent books, children can not only develop good learning habits, but also increase their knowledge and improve their reading ability. Books that are too difficult will make readers lose interest in reading because they cannot understand the content. Books that are too simple will not significantly improve readers' reading ability. It is crucial to choose books that are suitable for readers' reading ability. Measuring the difficulty of books belongs to text readability assessment. Therefore, readability assessment is crucial for publishers, teachers, parents, and students. Specifically, for students, reading texts that are suitable for their level can improve their reading ability; for teachers, they can provide students with reading materials of appropriate difficulty; for publishers, they can scientifically write book content.

[0032] At present, text readability involves multiple languages. There are not only single-language text readability studies such as English, Chinese, French, German, Italian, Swedish, Portuguese and Japanese, but also multi-language text readability studies. Initially, the difficulty of text readability was mainly evaluated manually by experts. When the number of texts was small, manual evaluation was feasible to a certain extent. However, as the number of texts increased, automated text readability evaluation became more and more important. In the early stages, some scholars constructed readability formulas based on surface features such as word length and sentence length. This method was very simple, but it only focused on surface factors that affect text difficulty, lacked deeper features, and the final effect was not obvious. With the development of natural language processing tools, scholars considered more factors that affect text readability, used tools to extract corresponding features, and used machine learning methods to establish a readability evaluation model. However, the features used in this method were not comprehensive and detailed.

[0033] In order to better meet the reading needs of different language learners, improve teaching effectiveness, and promote cross-cultural communication and understanding, studying text readability has become an urgent task. Through in-depth research on text readability, educators can better assess students' reading level, develop personalized teaching plans, and improve the effectiveness of language teaching. In addition, studying text readability can also help promote cross-cultural communication and understanding, cultivate students' cross-cultural communication skills, and promote the civilized development of society. Therefore, studying text readability is of great significance to language education and social development.

[0034] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0035] The following first introduces a text readability classification method based on contrastive learning provided in an embodiment of the present application.

[0036] Please refer to the flowchart of a text readability classification method based on contrastive learning provided in the embodiment of the present application. Figure 1 and Figure 3 , the text readability classification method may include the following steps S110-S150.

[0037] S110, inputting the acquired first target text into a preset BERT model, where the first target text includes labeled diverse text data, and the BERT model is a pre-trained language model based on Transformer for language understanding.

[0038] S120. The BERT model converts each word in the first target text into a vector representation through word embedding to obtain multiple word vectors.

[0039] S130. Encode multiple word vectors through the Transformer encoder to obtain context representation information of each word, and determine the initial feature vector representing the [CLS] token output information in the BERT model.

[0040] S140, extracting the most important word information of each token through maximum pooling, and obtaining a word-level feature vector sequence with the same length as the first target text.

[0041] S150, training a model based on contrastive learning according to the word-level feature vector sequence and the initial feature vector to obtain a mature target classifier for readability classification.

[0042] The above is a specific implementation of a text readability classification method based on contrastive learning provided in the embodiment of the present application. It can be understood that BERT is a pre-trained language model based on Transformer, which has powerful language understanding ability. Please refer to Figure 3 , [token] is processed by the BERT model to obtain T[token], where T[token] represents the initial feature vector corresponding to [token].

[0043] This application is trained by diverse text data in the first target text, which can ensure the generalization ability of the target classifier and adapt to different text types and styles. This application uses the BERT model to process the first target text, which can capture the deep semantic features and contextual information of the text, and determine the initial feature vector that represents the output information of the [CLS] token in the BERT model, which surpasses the surface feature extraction of traditional methods; in addition, the BERT model also extracts the most important word information of each token through maximum pooling to obtain a word-level feature vector sequence.

[0044] It should be noted that in the BERT model, [CLS]token is the beginning marker of a sentence and is usually designed to represent the entire sentence and include the semantic information of the entire sentence. When performing text classification tasks, the output of [CLS]token is usually used as the representation of the entire sentence and this representation is passed to the classifier for classification. The representation of [CLS]token is calculated by passing the representation of the entire sentence through a multi-layer Transformer encoder, which includes the understanding of the entire sentence and the encoding of semantic information, but does not have detailed information at the word level. The current text readability evaluation has great limitations and low accuracy.

[0045] It should be emphasized that the present application introduces a contrastive learning framework training model based on the semantic information and lexical information contained in the word-level feature vector sequence and the initial feature vector. The training process not only considers the understanding of the entire sentence and the encoding of semantic information, but also considers the lexical detail information at the word level, thereby enhancing the model's ability to recognize subtle differences in readability. Furthermore, the present application learns a universal representation of text based on contrastive learning, thereby improving the portability of the model's feature set in different data sets and application scenarios.

[0046] In one example, for an input sequence [x1, x2, ..., x n ], the maximum pooling operation satisfies the expression: WordPool([x1,x2,...,x n ])=max(x1,x2,...,x n ), where WordPool represents the maximum pooling operation, x i is the word-level feature vector of the i-th token.

[0047] In some embodiments, Figure 2 As shown, the aforementioned training model is based on contrastive learning according to the word-level feature vector sequence and the initial feature vector to obtain a mature target classifier for readability classification, that is, the aforementioned S150 may specifically include the following steps:

[0048] S210: Integrate the word-level feature vector sequence with the initial feature vector to obtain a first target feature vector to represent semantic information and vocabulary information corresponding to the first target text.

[0049] S220: Input the first target feature vector into a classifier framework based on contrastive learning for model training.

[0050] S230. Adjust the hyperparameter distribution through supervised contrastive learning loss function and train a mature target classifier.

[0051] In the embodiments of the present application, it can be understood that the initial feature vector contains semantic information to characterize the understanding of the entire sentence and the encoding of the semantic information, and the word-level feature vector sequence contains lexical information to consider the detailed information at the word level. In order to enable the trained target classifier to consider the overall semantics and details of the text at the same time, the present application integrates the word-level feature vector sequence with the initial feature vector.

[0052] In an example, see Figure 3 , the above-mentioned word-level feature vector sequence is integrated with the initial feature vector to obtain a first target feature vector to represent the semantic information and vocabulary information corresponding to the first target text, that is, the above-mentioned S210 may specifically include the following steps:

[0053] S310: Input the word-level feature vector sequence and the initial feature vector into the attention layer of the BERT model.

[0054] S320, learning the interactive information between the word-level feature vector sequence and the initial feature vector based on the attention mechanism to capture the semantic information and vocabulary information of the text.

[0055] S330. Determine that the vector corresponding to the interactive information is represented as a first target feature vector.

[0056] In another example, the integration operation of the word-level feature vector sequence and the initial feature vector corresponds to the expression:

[0057] Integration(x cls ,x word_pool )=Softmax(Attention(x cls ,x word_pool ))·x word_pool

[0058] In the formula, Integration{} represents the first target feature vector after integration, x cls is the initial eigenvector, x word_pool is a sequence of word-level feature vectors, Attention{} is the attention function and is used to calculate the weight between the initial feature vector and each word-level feature vector, and the Softmax function is used to normalize the weight.

[0059] In some embodiments, after the model is trained based on contrastive learning according to the word-level feature vector sequence and the initial feature vector to obtain a mature target classifier for readability classification, that is, after the aforementioned S150, the text readability classification method based on contrastive learning may further include the following steps:

[0060] S410: Obtain a second target text to be classified according to readability.

[0061] S420, performing word conversion, encoding, maximum pooling and feature integration processing on the second target text through the BERT model to obtain a second target feature vector.

[0062] S430: Process the second target feature vector through a target classifier to obtain a classification result to characterize the readability level of the second target text.

[0063] In the embodiment of the present application, it can be understood that after the target classifier is obtained based on contrastive learning training, the text readability evaluation classification is performed through the target classifier. Exemplarily, the present application obtains a second target text to be classified for processing. It should be noted that before inputting information into the target classifier, consistent with the processing flow of the aforementioned first target text, the second target text needs to be processed to obtain a second target feature vector, and the second target feature vector has similar properties to the aforementioned first target feature vector.

[0064] In some embodiments, the first target feature vector corresponds to multiple samples with label categories, and the label categories are used to characterize the readability difficulty of the text; during the training process of the target classifier, samples corresponding to the same label category are positive samples and are pulled closer to each other, and samples corresponding to different label categories are negative samples and are pushed away from each other.

[0065] In the embodiments of the present application, it can be understood that, since the BERT model can learn more inter-text features and introduce a contrastive learning framework, the model's ability to recognize subtle differences in readability is enhanced by shortening the representation distance of texts with similar readability and extending the representation distance of texts with different readability categories. In other words, the present application uses samples with the same readability difficulty as positive samples and samples with different readability difficulties as negative samples; contrastive learning can shorten readability samples of the same type and extend readability samples of different types, thereby improving the readability classification effect.

[0066] In one example, although the supervised contrastive learning method has shown its superiority, the contrastive loss can only learn the general representation of the input example. In addition to the contrast term, the present application uses the cross entropy loss to learn the linear classifier. Specifically, the present application adopts the supervised contrastive learning loss function, which satisfies the expression:

[0067]

[0068] Among them, L sup is the loss value, N is the number of samples, |P i | for P i The cardinality, P i :={p∈A i :y p =y i} is the index set of positive samples, y i is the classification label corresponding to the text. For a K classification task with N samples, y i ∈{1,2,···,K}; z i is x i The normalized representation of A i:=I\{i} is the comparison sample index set, I={1,2,··,N} represents the index set of training samples, the i-th sample is the anchor point, the · symbol represents the dot product, and τ∈R+ is the temperature factor.

[0069] In some embodiments, it should be noted that the flexibility of the contrastive learning framework and deep learning model of the present application means that they can be adjusted and applied to text readability assessment in different languages, such as Chinese, English, and Spanish. The present application regards text readability as a classification task, and the core of contrastive learning is to distinguish samples of different categories; based on this, the technical solution of the present application can also be applied to other text classification tasks, such as sentiment analysis, topic classification, etc.

[0070] It should also be noted that the application scenarios of text readability assessment are rich, so it can be combined with a variety of systems, such as for personalized recommendation systems, by evaluating the readability of the text, to provide appropriate reading materials for users of different reading levels. It can also be used in the field of education, including textbook selection and language-assisted language learning. The technology of this application can help learners choose appropriate learning materials according to their level and improve learning efficiency. This technology can also be used in the medical field. When providing medical information to patients, this technology can help ensure the readability of the information so that patients can better understand their health conditions and treatment plans.

[0071] In some embodiments, the present application provides a text readability classification system 500 based on contrastive learning, such as Figure 4 As shown, the text readability classification system 500 may include the following modules:

[0072] An input module 510 is used to input the acquired first target text into a preset BERT model, where the first target text includes labeled diverse text data, and the BERT model is a pre-trained language model based on Transformer for language understanding;

[0073] A conversion module 520, configured to convert each word in the first target text into a vector representation by word embedding using a BERT model to obtain a plurality of word vectors;

[0074] An encoding module 530 is used to encode multiple word vectors through a Transformer encoder to obtain contextual representation information of each word and determine an initial feature vector representing the output information of [CLS] token in the BERT model;

[0075] A pooling module 540 is used to extract the most important word information of each token through maximum pooling to obtain a word-level feature vector sequence with the same length as the first target text;

[0076] The training module 550 is used to train the model based on contrastive learning according to the word-level feature vector sequence and the initial feature vector to obtain a mature target classifier for readability classification.

[0077] According to an embodiment of the present application, any multiple modules among the input module 510, the conversion module 520, the encoding module 530, the pooling module 540 and the training module 550 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.

[0078] In some embodiments, the text readability classification system based on contrastive learning may further include a readability classification module 560, which may be specifically used for:

[0079] Obtaining a second target text to be classified for readability;

[0080] The second target text is processed by word conversion, encoding, maximum pooling and feature integration through the BERT model to obtain the second target feature vector;

[0081] The second target feature vector is processed by the target classifier to obtain a classification result to characterize the readability level of the second target text.

[0082] In some embodiments, the aforementioned training module 550 may include the following units:

[0083] An integration unit 551 is used to integrate the word-level feature vector sequence with the initial feature vector to obtain a first target feature vector to represent semantic information and vocabulary information corresponding to the first target text;

[0084] An input training unit 552, used for inputting the first target feature vector into a classifier framework based on contrastive learning for model training;

[0085] The parameter adjustment unit 553 is used to adjust the hyperparameter distribution through the supervised contrast learning loss function to train a mature target classifier.

[0086] In some embodiments, the aforementioned integration unit 551 may be specifically used for:

[0087] Input the word-level feature vector sequence and the initial feature vector into the attention layer of the BERT model;

[0088] Based on the attention mechanism, the interaction information between the word-level feature vector sequence and the initial feature vector is learned to capture the semantic and lexical information of the text;

[0089] The vector corresponding to the mutual information is determined to be represented as the first target feature vector.

[0090] Figure 4 Each module in the system shown has the function of implementing each step in the aforementioned text readability classification method based on contrastive learning, and can achieve its corresponding technical effect. For the sake of concise description, it will not be repeated here.

[0091] In some embodiments, the present application provides an electronic device, the structure diagram of the electronic device is as follows Figure 5 shown.

[0092] The electronic device may include a processor 610 and a memory 620 storing computer program instructions.

[0093] Specifically, the processor 610 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0094] The memory 620 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 620 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 620 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 620 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 620 is a non-volatile solid-state memory.

[0095] The memory 620 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory 620 includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in any of the text readability classification methods based on contrastive learning in the above-mentioned embodiments.

[0096] The processor 610 implements any one of the text readability classification methods based on contrastive learning in the above embodiments by reading and executing computer program instructions stored in the memory 620 .

[0097] In one example, the electronic device may further include a communication interface 630 and a bus 600. Figure 5 As shown, the processor 610, the memory 620, and the communication interface 630 are connected via a bus 600 and communicate with each other.

[0098] The communication interface 630 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0099] Bus 600 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 600 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0100] In addition, in combination with the text readability classification method based on contrastive learning in the above embodiments, the present application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the text readability classification methods based on contrastive learning in the above embodiments is implemented.

[0101] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0102] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0103] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0104] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0105] In summary, compared with the prior art, this application has the following beneficial effects:

[0106] 1. This application is trained by diverse text data in the first target text, which can ensure the generalization ability of the target classifier and adapt to different text types and styles. This application uses the BERT model to process the first target text, which can capture the deep semantic features and contextual information of the text, and determine the initial feature vector that represents the [CLS] token output information in the BERT model, which surpasses the surface feature extraction of traditional methods.

[0107] 2. This application extracts the most important word information of each token through maximum pooling. Based on the semantic information and lexical information contained in the word-level feature vector sequence and the initial feature vector, the training process considers not only the understanding of the entire sentence and the encoding of semantic information, but also the lexical detail information at the word level to ensure the accuracy of text readability classification.

[0108] 3. This application introduces a contrastive learning framework to train the model, which shortens the representation distance of texts with similar readability and increases the representation distance of texts with different readability categories, thereby enhancing the model's ability to recognize subtle differences in readability. It also improves the portability of the model's feature set in different data sets and application scenarios by learning a universal representation of text.

[0109] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text readability classification method based on contrastive learning, characterized in that: include: Inputting the acquired first target text into a preset BERT model, wherein the first target text includes labeled diverse text data, and the BERT model is a pre-trained language model based on Transformer for language understanding; The BERT model converts each word in the first target text into a vector representation through word embedding to obtain multiple word vectors; Encode the multiple word vectors through a Transformer encoder to obtain context representation information of each word, and determine an initial feature vector representing the output information of the [CLS] token in the BERT model; Extract the most important word information of each token through maximum pooling to obtain a word-level feature vector sequence with the same length as the first target text; According to the word-level feature vector sequence and the initial feature vector, a model is trained based on contrastive learning to obtain a mature target classifier for readability classification.

2. The text readability classification method based on contrastive learning as claimed in claim 1, characterized in that: The method of training a model based on contrastive learning according to the word-level feature vector sequence and the initial feature vector to obtain a mature target classifier for readability classification includes: Integrating the word-level feature vector sequence with the initial feature vector to obtain a first target feature vector to represent semantic information and vocabulary information corresponding to the first target text; Inputting the first target feature vector into a classifier framework based on contrastive learning for model training; The hyperparameter distribution is adjusted through supervised contrastive learning loss function, and a mature target classifier is trained.

3. The text readability classification method based on contrastive learning as claimed in claim 2, characterized in that: The step of integrating the word-level feature vector sequence with the initial feature vector to obtain a first target feature vector to represent semantic information and vocabulary information corresponding to the first target text includes: Input the word-level feature vector sequence and the initial feature vector into the attention layer of the BERT model; Learning the interaction information between the word-level feature vector sequence and the initial feature vector based on an attention mechanism to capture the semantic information and vocabulary information of the text; Determine that the vector representation corresponding to the mutual information is a first target feature vector.

4. The text readability classification method based on contrastive learning according to any one of claims 1 to 3, characterized in that: After training the model based on contrastive learning according to the word-level feature vector sequence and the initial feature vector to obtain a mature target classifier for readability classification, the text readability classification method based on contrastive learning further includes: Obtaining a second target text to be classified for readability; Performing word conversion, encoding, maximum pooling, and feature integration processing on the second target text through the BERT model to obtain a second target feature vector; The second target feature vector is processed by the target classifier to obtain a classification result to characterize the readability level of the second target text.

5. The text readability classification method based on contrastive learning as claimed in claim 2 or 3, characterized in that: The first target feature vector corresponds to multiple samples with label categories, and the label categories are used to characterize the readability difficulty of the text; during the training process of the target classifier, samples corresponding to the same label category are positive samples and are pulled closer to each other, and samples corresponding to different label categories are negative samples and are pushed away from each other.

6. The text readability classification method based on contrastive learning as claimed in claim 1, characterized in that: For the input sequence [x1,x2,...,x n ], the maximum pooling operation satisfies the expression: WordPool([x1,x2,...,x n ])=max(x1,x2,...,x n ), where WordPool represents the maximum pooling operation, x i is the word-level feature vector of the i-th token.

7. The text readability classification method based on contrastive learning as claimed in claim 2, characterized in that: The integration operation of the word-level feature vector sequence and the initial feature vector satisfies the expression: Integration(x cls ,x word_pool )=Softmax(Attention(x cls ,x word_pool ))·x word_pool In the formula, Integration{} represents the first target feature vector after integration, x cls is the initial eigenvector, x word_pool is the word-level feature vector sequence, Attention{} is the attention function and is used to calculate the weight between the initial feature vector and each word-level feature vector, and the Softmax function is used to normalize the weight.

8. A text readability classification system based on contrastive learning, characterized in that: include: An input module, configured to input the acquired first target text into a preset BERT model, wherein the first target text includes labeled diverse text data, and the BERT model is a pre-trained language model based on Transformer for language understanding; A conversion module, used for the BERT model to convert each word in the first target text into a vector representation through word embedding to obtain multiple word vectors; An encoding module, configured to encode the plurality of word vectors through a Transformer encoder to obtain contextual representation information of each word, and determine an initial feature vector representing output information of a [CLS] token in the BERT model; A pooling module, used to extract the most important word information of each token through maximum pooling to obtain a word-level feature vector sequence with the same length as the first target text; A training module is used to train a model based on contrastive learning according to the word-level feature vector sequence and the initial feature vector to obtain a mature target classifier for readability classification.

9. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the text readability classification method based on contrastive learning as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the text readability classification method based on contrastive learning as described in any one of claims 1 to 7 is implemented.

Images