System text classification method and device, equipment and storage medium

Abstract

The invention discloses a system text classification method and device, equipment and a storage medium. The method comprises the following steps: acquiring a to-be-classified system text; inputting the to-be-classified system text into a pre-trained document vector model to obtain a first system text vector; inputting the to-be-classified system text into a pre-training language model based on a deep learning architecture to obtain a second system text vector; obtaining a third system text vector according to the first system text vector and the second system text vector; and inputting the third system text vector into a classification model, and classifying the system text to be classified. According to the method, the system text is classified, so that the efficiency of querying the system when the system is drawn up can be improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing technology, and in particular to a method, device, equipment and storage medium for classifying institutional texts. Background Art

[0002] With the rapid advancement of information technology, natural language processing technology has been greatly developed. Among them, text classification, as a key sub-field, has achieved remarkable results. Commercial banks need to formulate a large number of internal systems in a timely manner in the process of operation and management to standardize business processes, ensure risk control, correct employee behavior, etc. The traditional system drafting method based on manual learning and judgment is heavily dependent on the professional knowledge and learning ability of system drafters, and is inefficient. Especially in the face of massive text data and complex business scenarios, the shortcomings of relying solely on manual drafting of systems are becoming increasingly prominent.

[0003] The standardization and improvement of commercial bank internal systems are of great significance for ensuring bank security, improving work efficiency and service quality, and ensuring compliance operations. At present, the drafting of bank internal systems mainly adopts the method of manual document writing, which requires the drafters to have relatively comprehensive business knowledge and refer to a large amount of external bank system materials for reference. This is inefficient and difficult to ensure the integrity and consistency of the system content. In order to improve the efficiency of system drafting, a method that can assist system drafters is urgently needed. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for classifying system texts, so as to solve the problem of low system drafting efficiency caused by difficulty in classifying system texts in the prior art.

[0005] According to one aspect of the present invention, a method for classifying institutional texts is provided, the method comprising:

[0006] Get the system text to be classified;

[0007] Inputting the system text to be classified into a pre-trained document vector model to obtain a first system text vector;

[0008] Inputting the institutional text to be classified into a pre-trained language model based on a deep learning architecture to obtain a second institutional text vector;

[0009] Obtain a third system text vector according to the first system text vector and the second system text vector;

[0010] The third system text vector is input into a classification model to classify the system text to be classified.

[0011] According to another aspect of the present invention, a system text classification device is provided, the device comprising:

[0012] An acquisition module is used to obtain the system text to be classified;

[0013] A first input module, used for inputting the system text to be classified into a pre-trained document vector model to obtain a first system text vector;

[0014] A second input module, used for inputting the system text to be classified into a pre-trained language model based on a deep learning architecture to obtain a second system text vector;

[0015] A calculation module, used for obtaining a third system text vector according to the first system text vector and the second system text vector;

[0016] The classification module is used to input the third system text vector into the classification model to classify the system text to be classified.

[0017] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising: at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the institutional text classification method described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the institutional text classification method described in any embodiment of the present invention when executed.

[0021] The embodiment of the present invention provides a method, device, equipment and storage medium for classifying institutional texts, the method comprising: obtaining institutional texts to be classified; inputting the institutional texts to be classified into a pre-trained document vector model to obtain a first institutional text vector; inputting the institutional texts to be classified into a pre-trained language model based on a deep learning architecture to obtain a second institutional text vector; obtaining a third institutional text vector based on the first institutional text vector and the second institutional text vector; inputting the third institutional text vector into a classification model to classify the institutional texts to be classified. The method classifies institutional texts through a pre-trained model, enabling institutional drafters to quickly query the institutional texts based on the classified results when drafting the institutional texts, thereby improving the efficiency of querying the institutional texts and solving the problem of low institutional drafting efficiency caused by the difficulty in classifying institutional texts in the prior art.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic diagram of a process flow of a method for classifying institutional texts provided in the first embodiment of the present invention;

[0025] Figure 2 A schematic diagram of a flow chart of a method for classifying institutional texts provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of a system text classification result provided by an embodiment of the present invention;

[0027] Figure 4 A schematic diagram of the structure of a system text classification device provided in the second embodiment of the present invention;

[0028] Figure 5 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only an embodiment of a part of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should belong to the scope of protection of the present invention. It should be understood that the various steps recorded in the method implementation of the present invention can be performed in different orders and / or in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0030] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, any variations of the terms "including" and "having" are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0034] Embodiment 1

[0035] Figure 1 A flow chart of a method for classifying system texts provided in Example 1 of the present invention is provided. The method can be applied to classify system texts so that staff can draft systems based on the classification results. The method can be executed by a system text classification device, wherein the device can be implemented by software and / or hardware and is generally integrated on an electronic device. In this embodiment, the electronic device includes but is not limited to: computers and other devices.

[0036] like Figure 1 As shown, a method for classifying system texts provided in Embodiment 1 of the present invention includes the following steps:

[0037] S110. Obtain the system text to be classified.

[0038] Among them, the system text to be classified can be the system text that needs to be classified. The system text can be the content of the articles after splitting a complete system by articles. A system is a series of formal written documents formulated to regulate internal operations and external behaviors, including specific regulations, rules, guidelines, and provisions, etc., aiming to clarify the rights and obligations of members and ensure the normal operation and goal achievement of an organization or institution. The system can include external systems and internal systems. The external system can be a system applicable to the whole country or a region, and the internal system can be a system formulated by a commercial bank itself and applicable to the operation and management of the bank.

[0039] In this embodiment, the system text to be classified that needs to be classified can be obtained.

[0040] In one embodiment, the obtaining of the system text to be classified includes: obtaining an initial system text; removing special characters, spaces, and stop words from the initial system text to obtain a cleaned initial system text; and using the cleaned initial system text as the system text to be classified.

[0041] Among them, the initial system text can be an unprocessed system text. Special characters can include various special symbols. Stop words can be a custom word list, which can refer to certain words or terms that are automatically filtered out before or after processing natural language data (or text) in information retrieval to save storage space and improve search efficiency. For example, stop words can include "is", "of", "and", "system", etc.

[0042] In this embodiment, an initial system text can be obtained, and data cleaning is performed on the system text, such as removing special characters, spaces, and stop words from the initial system text to obtain a cleaned initial system text, and using the cleaned initial system text as the system text to be classified. Data cleaning is a process of correcting, filtering, and transforming data to remove noise and unnecessary information. Through data cleaning, irrelevant information in the text can be removed to improve the performance of the model. Exemplarily, data cleaning can be performed in the following ways:

[0043] Removing special characters:

[0044] removeSpecialCharacters(text) = re.sub(r'[^\w\s]', '', text);

[0045] Removing spaces:

[0046] removeWhitespace(text) = re.sub(r'\s+', '', text);

[0047] Removing stop words:

[0048] removeStopWords(text)=filter(word∈stopWords,text).

[0049] S120 , inputting the system text to be classified into a pre-trained document vector model to obtain a first system text vector.

[0050] Among them, the document vector model can be a doc2vec model. Doc2vec is also called Paragraph2vec, sentence embeddings, which is a text feature extraction method based on the word embeddings (word2vec) model and the distributed memory model. Doc2vec can convert a document into a vector of fixed length, and then perform subsequent text classification or document clustering tasks. It is developed based on the word2vec model. The difference is that word2vec mainly learns the vector representation of words, while doc2vec can learn the vector representation of the entire sentence, paragraph or document. The first system text vector can be a text vector after text vectorization through the document vector model. Text vectorization is the process of converting text data into a digital form that can be processed by a computer. Text vectorization is an important step in natural language processing (Natural Language Processing, NLP), which aims to represent text as a vector for further analysis and processing. NLP is a branch of artificial intelligence and linguistics. The core is to enable computers to understand and use human natural language, so that they can parse text meanings, identify text emotions, and answer text questions. It involves processing, analyzing and generating text or voice data to achieve effective communication between humans and machines. Natural language processing has a wide range of applications, including text classification, semantic understanding, information summarization, machine translation, etc.

[0051] In this embodiment, the system text to be classified may be input into a pre-trained document vector model to obtain a first system text vector.

[0052] In one embodiment, the training of the document vector model includes: obtaining institutional texts under multiple institutional categories; preprocessing the institutional texts through a preprocessing module to obtain an institutional text corpus; using the institutional text corpus as a training set, training the document vector model through a semantic extraction toolkit, and obtaining a trained document vector model.

[0053] Among them, the system category can be the category of system text, and the system category can include external system category and internal system category. The specific system category can be set according to the actual situation. For example, the internal system category of this embodiment is obtained by summarizing the work items in various fields of commercial banks, and can include export credit, pool financing, external assets, bad debt recognition, agency bonds, secured financing, mortgage setting, credit issuance, purchase of accounts receivable, overseas commercial refinancing and contract performance. Several major categories, each of which can also include multiple small categories, all system categories total about 800 categories, through which internal and external systems can be divided and classified into specific system categories. The preprocessing module can be a module for preprocessing system texts, and the preprocessing module can be taggedDocument, which is an illustrative class used to represent objects that Doc2Vec can use as text examples. The system text corpus can be a system text dataset for training models. The semantic extraction toolkit can be genism, which is a Python library for topic modeling, document indexing and similar retrieval of large corpora.

[0054] In this embodiment, when training the document vector model, the institutional texts under multiple institutional categories can be preprocessed to obtain an institutional text corpus, and the institutional text corpus can be used as a training set to train the document vector model through a semantic extraction toolkit to obtain a trained document vector model. Exemplarily, 4 / 5 of the institutional texts under each institutional category can be taken as text training corpus, and taggedDocument can be used to preprocess the corpus, and the institutional texts can be preprocessed to construct an institutional text corpus; genism can be used to train the doc2vec model on the institutional text corpus to obtain a trained model doc2vec_new. Later, when classifying the institutional text, the first institutional text vector can be obtained through doc2vec_new.

[0055] S130. Input the system text to be classified into a pre-trained language model based on a deep learning architecture to obtain a second system text vector.

[0056] Among them, the pre-trained language model can be an improved BERT model (Robustly Optimized BERTPretraining Approach, RoBERTa) model, specifically a Roberta-base-chinese model suitable for Chinese. The RoBERTa model is a pre-trained language model based on the Transformer architecture. It improves the training method and parameter setting based on the Bidirectional Encoder Representation from Transformers (BERT) model, and improves the performance and generalization ability of the model. The core idea of ​​RoBERTa is to use the self-attention mechanism of Transformer to capture the contextual dependencies in the text, thereby generating a more accurate text representation. By learning from a large-scale corpus, it is possible to learn the grammatical rules, lexical meanings, and semantic relationships of a language, and is widely used in various natural language processing tasks. The second system text vector can be a text vector after text vectorization through a pre-trained language model.

[0057] In this embodiment, the system text to be classified can be input into a pre-trained language model of a deep learning architecture to obtain a second system text vector.

[0058] In one embodiment, the training of the pre-trained language model based on the deep learning architecture includes: training the language model to be trained based on the deep learning architecture through a training data set to obtain a trained pre-trained language model; testing the trained pre-trained language model through a test data set to obtain a test result; calculating a loss function through the test result and the true label of each data in the test data set; when the loss function meets a threshold, obtaining a pre-trained language model; otherwise, continuing to train the pre-trained language model through a back propagation algorithm until the loss function meets the threshold; wherein the training data set and the test data set include institutional texts under multiple institutional categories.

[0059] Among them, the language model to be trained can be a language model that needs to be trained. Both the training data set and the test data set can be selected from institutional texts under multiple institutional categories, and the specific amount of data can be set according to actual conditions. The data in the training data set and the test data set can be data with category labels. The true label can be the original institutional category of the institutional text in the test data set. The loss function can be a function used to measure the degree of inconsistency between the predicted value and the true value of the model. The smaller the loss function, the better the robustness of the model. The back propagation algorithm is an optimization algorithm for training artificial neural networks. By calculating the gradient of the loss function with respect to each parameter in the network, it guides the parameter update to minimize the error between the predicted output and the true label.

[0060] In this embodiment, the language model to be trained based on the deep learning architecture can be trained and tested using a training data set and a test data set to obtain a pre-trained language model. When the loss function of the pre-trained language model meets a threshold, the training ends. Otherwise, the pre-trained language model can continue to be trained using a back-propagation algorithm until the loss function meets the threshold.

[0061] Exemplarily, the Roberta-base-chinese model can be fine-tuned through the pre-training data set, the general RoBERTa pre-trained Chinese model can be trained using labeled task-specific data, and the general language representation obtained by pre-training can be applied to specific banking system tasks. At the beginning of training, specific labeled training data and test data can be input into the Roberta model, and the loss function is calculated by comparing the predicted results with the true labels. Then, the back-propagation algorithm is used to update the model parameters. Through multiple iterative training, the model performance is continuously optimized until the best performance is achieved on the validation set, realizing the text semantic enhancement of the RoBERTa pre-trained language model in the field of banking systems. After that, the system text data to be classified can be input into the fine-tuned RoBERTa model to obtain the second system text vector.

[0062] S140. Obtain a third system text vector according to the first system text vector and the second system text vector.

[0063] In this embodiment, the third system text vector can be calculated by the first system text vector and the second system text vector, and the specific calculation method is not limited in this embodiment. For example, the average value of the first system text vector and the second system text vector can be directly calculated to obtain the third system text vector, or the weights of the first system text vector and the second system text vector can be pre-set to obtain the third system text vector by adding the weights. Exemplarily, the third system text vector s3 is obtained by calculating the average value of the first system text vector s1 and the second system text vector s2, where s1 and s2 are text vectors of the same dimension.

[0064] sentencevector(s3)=(s1+s2) / 2.

[0065] S150: Input the third system text vector into a classification model to classify the system text to be classified.

[0066] Among them, the classification model can be a text classification model based on convolutional neural networks (Convolutional Neural Networks for Sentence Classification, TextCNN). The TextCNN model is a convolutional neural network model specifically used to process text data. It can achieve efficient and accurate processing of text data by automatically learning text feature representation. Through structures such as convolutional layers and pooling layers, TextCNN can capture local features and contextual information of text, thereby generating vector representations of text. These vectors can be used for various text-related tasks, such as text classification, sentiment analysis, etc. Text classification can refer to the process of automatically dividing a piece of text (such as an article, sentence, paragraph or phrase) into predefined categories or labels. This classification can be based on the content, style, theme or other features of the text.

[0067] In this embodiment, the third system text vector can be input into the classification model to classify the system text to be classified. The classification model can be trained by using 4 / 5 system text vectors under each system category as a training data set, and the model parameters are updated according to the training results. This implementation does not limit the training process.

[0068] In one embodiment, the inputting the third system text vector into the classification model to classify the system text to be classified includes: performing convolution calculation on the third system text vector through the classification model to obtain one or more feature matrices of the system text to be classified; and classifying the system text to be classified through the one or more feature matrices.

[0069] Among them, convolution calculation can refer to the mathematical calculation process of combining two functions into a new function. The feature matrix can be used to represent the possibility that the institutional text to be classified belongs to different categories. Different paragraphs in an institutional text to be classified can correspond to a feature matrix, so an institutional text to be classified can have one or more feature matrices.

[0070] In this embodiment, the third system vector can be convolved by the classification model to obtain one or more feature matrices of the system text to be classified. The system text to be classified can be classified by these feature matrices.

[0071] A method for classifying institutional texts provided in a first embodiment of the present invention includes: obtaining institutional texts to be classified; inputting the institutional texts to be classified into a pre-trained document vector model to obtain a first institutional text vector; inputting the institutional texts to be classified into a pre-trained language model based on a deep learning architecture to obtain a second institutional text vector; obtaining a third institutional text vector based on the first institutional text vector and the second institutional text vector; inputting the third institutional text vector into a classification model to classify the institutional texts to be classified. This method classifies institutional texts through a pre-trained model, enabling institutional drafters to quickly query the system based on the classified results when drafting the system, thereby improving the efficiency of querying the system and solving the problem of low efficiency in drafting the system due to the difficulty in classifying institutional texts in the prior art.

[0072] Based on the above embodiment, a variant embodiment of the above embodiment is proposed. It should be noted that in order to make the description concise, only the differences from the above embodiment are described in the variant embodiment.

[0073] In one embodiment, the classifying of the institutional text to be classified by using the one or more feature matrices includes: performing a maximum pooling operation on the one or more feature matrices to obtain one or more feature matrices after pooling; performing multi-classification on the one or more feature matrices after pooling by splicing activation layers to obtain the probability of each paragraph of the institutional text to be classified under different institutional categories; determining the classification result of each paragraph of the text by using the probability, and classifying each paragraph of the institutional text to be classified by using the classification result.

[0074] Among them, the maximum pooling operation may refer to a downsampling operation, which is often used in convolutional neural networks. Its main function is to reduce the size of the feature map while retaining important feature information. The splicing activation layer may refer to a structure in a deep learning model, which is usually used for multi-class classification tasks. It combines the concepts of the Softmax activation function and feature splicing to more effectively process the output of the model. Multi-classification (N-class classification) may refer to the task of classifying input data into N different categories, which is used to handle multi-class problems, where each sample can only belong to one category. For example, if there are three categories (A, B, C), the N-classification task is to classify each sample as A, B, or C. The classification result may be a text classification result.

[0075] In this embodiment, each feature matrix can be subjected to a maximum pooling operation to obtain each pooled feature matrix. Each feature matrix can be multi-classified through the splicing activation layer in the classification model to obtain the probability of each paragraph of text in the institutional text to be classified under different institutional categories. The classification result of the text can be determined through the probability of each paragraph of text, so that each paragraph of text in the institutional text to be classified can be classified according to the classification result. Exemplarily, the third institutional text vector is input into the TextCNN model, and after a series of feature matrices are output through convolution calculation, a maximum pooling operation is performed on the output feature matrix, so that the generated features remain unchanged after a slight change in the input data, to prevent overfitting. Finally, N classifications are performed through the splicing softmax layer, where N is the number of classifications, and the probability of each classification is calculated, and the classification result is obtained according to the probability.

[0076] In one embodiment, determining the classification result of each paragraph of text by using the probability includes: taking the system category corresponding to the largest probability among the probabilities, or the probability ranking in the top N among all the probabilities, as the classification result of each paragraph of text.

[0077] In this embodiment, the system category corresponding to the probability with the highest probability in each text segment can be used as the classification result of the text, and the system category corresponding to the probability ranked in the top N positions of all the probabilities of the text can also be used as the classification result of the text.

[0078] The embodiments of the present invention provide several specific implementation methods based on the technical solutions of the above embodiments.

[0079] As a specific implementation method of this embodiment, Figure 2 A flowchart of a method for classifying institutional texts provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the obtained institutional text data set can be cleaned to obtain institutional texts to be classified, and the first institutional text vector and the second institutional text vector can be obtained through the Doc2vec model and the RoBERTa model respectively. The mean of the text vector is obtained by calculating the average of the first institutional text vector and the second institutional text vector. The mean of the text vector is input into TextCNN to obtain the classified institutional text. The classified institutional text can be used to recommend institutional clauses to the staff.

[0080] When drafting a system, the existing scheme requires relevant personnel to manually review a large number of external and internal systems, conduct sufficient learning and summarization, and extract the content involved in this drafting work from the massive data, which is time-consuming. The technical solution of this embodiment can better represent the system text by combining doc2vec and RoBERTa models to generate system text vectors; the textCNN model is used to classify external and internal system texts, provide data support for the drafting of internal systems, and improve the accuracy of system text classification. It can more efficiently, accurately and completely obtain the reference information required for the drafting of internal systems, which can reduce the heavy reliance of traditional system drafting methods on manual labor and improve the accuracy, completeness and timeliness of system drafting.

[0081] Figure 3 A schematic diagram of a system text classification result provided by an embodiment of the present invention, such as Figure 3 As shown in the figure, the "division of responsibilities" on the left is one of the system categories, and the right side is the relevant legal provisions. When commercial bank system drafters are compiling internal systems, they can quickly view the legal provisions under a certain system classification, which can play a good auxiliary role in the system drafting work.

[0082] Embodiment 2

[0083] Figure 4 This is a structural schematic diagram of a system text classification device provided in Example 2 of the present invention. The device can be used to classify system texts so that staff can draft systems based on the classification results. The device can be implemented by software and / or hardware and is generally integrated on an electronic device.

[0084] like Figure 4 As shown, the device comprises:

[0085] An acquisition module 210 is used to acquire the system text to be classified;

[0086] A first input module 220, used to input the system text to be classified into a pre-trained document vector model to obtain a first system text vector;

[0087] A second input module 230 is used to input the system text to be classified into a pre-trained language model based on a deep learning architecture to obtain a second system text vector;

[0088] A calculation module 240, configured to obtain a third system text vector according to the first system text vector and the second system text vector;

[0089] The classification module 250 is used to input the third system text vector into a classification model to classify the system text to be classified.

[0090] This embodiment provides a system text classification device, including: an acquisition module, used to acquire the system text to be classified; a first input module, used to input the system text to be classified into a pre-trained document vector model to obtain a first system text vector; a second input module, used to input the system text to be classified into a pre-trained language model based on a deep learning architecture to obtain a second system text vector; a calculation module, used to obtain a third system text vector according to the first system text vector and the second system text vector; a classification module, used to input the third system text vector into a classification model to classify the system text to be classified. By classifying the system text with a pre-trained model, system drafters can quickly query the system according to the classified results when drafting the system, thereby improving the efficiency of querying the system and solving the problem of low efficiency in system drafting caused by the difficulty in classifying system texts in the prior art.

[0091] Furthermore, the acquisition module 210 is specifically used for:

[0092] Obtaining the initial institutional text;

[0093] Removing special characters, spaces and stop words from the initial policy text to obtain a cleaned initial policy text;

[0094] The cleaned initial system text is used as the system text to be classified.

[0095] Furthermore, the training of the document vector model includes:

[0096] Get institutional texts under multiple institutional categories;

[0097] Preprocessing the system text by a preprocessing module to obtain a system text corpus;

[0098] The institutional text corpus is used as a training set, and a document vector model is trained through a semantic extraction toolkit to obtain a trained document vector model.

[0099] Furthermore, the training of the pre-trained language model based on the deep learning architecture includes:

[0100] The language model to be trained based on the deep learning architecture is trained using the training data set to obtain a pre-trained language model;

[0101] Test the pre-trained language model using the test data set to obtain the test results;

[0102] Calculate the loss function using the test results and the true label of each piece of data in the test data set;

[0103] When the loss function meets the threshold, a pre-trained language model is obtained; otherwise, the pre-trained language model is continuously trained by a back propagation algorithm until the loss function meets the threshold;

[0104] The training data set and the test data set include institutional texts under multiple institutional categories.

[0105] Furthermore, the classification module 250 is specifically used for:

[0106] Performing convolution calculation on the third system text vector through a classification model to obtain one or more feature matrices of the system text to be classified;

[0107] The institutional text to be classified is classified by using the one or more feature matrices.

[0108] Furthermore, the classifying the institutional text to be classified by using the one or more feature matrices includes:

[0109] Performing a maximum pooling operation on the one or more feature matrices to obtain one or more pooled feature matrices;

[0110] Performing multi-classification on one or more feature matrices after pooling by splicing activation layers, and obtaining the probability of each text in the institutional text to be classified under different institutional categories;

[0111] The classification result of each paragraph of text is determined by the probability, and each paragraph of text in the institutional text to be classified is classified by the classification result.

[0112] Furthermore, the determining the classification result of each text segment by the probability includes:

[0113] The institutional category corresponding to the maximum probability among the probabilities, or the probability ranking in the top N among all the probabilities, is taken as the classification result of each paragraph of text.

[0114] The above-mentioned system text classification device can execute the system text classification method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0115] Embodiment 3

[0116] Figure 5A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0117] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0118] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0119] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the institutional text classification method.

[0120] In some embodiments, the institutional text classification method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the institutional text classification method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the institutional text classification method in any other appropriate manner (for example, by means of firmware).

[0121] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0123] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0125] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0126] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0127] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0128] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for classifying institutional texts, characterized in that: The method comprises: Get the system text to be classified; Inputting the system text to be classified into a pre-trained document vector model to obtain a first system text vector; Inputting the institutional text to be classified into a pre-trained language model based on a deep learning architecture to obtain a second institutional text vector; Obtain a third system text vector according to the first system text vector and the second system text vector; The third system text vector is input into a classification model to classify the system text to be classified.

2. The method according to claim 1, characterized in that The step of obtaining the system text to be classified includes: Obtaining the initial institutional text; Removing special characters, spaces and stop words from the initial policy text to obtain a cleaned initial policy text; The cleaned initial system text is used as the system text to be classified.

3. The method according to claim 1, characterized in that The training of the document vector model includes: Get institutional texts under multiple institutional categories; Preprocessing the system text by a preprocessing module to obtain a system text corpus; The institutional text corpus is used as a training set, and a document vector model is trained through a semantic extraction toolkit to obtain a trained document vector model.

4. The method according to claim 1, characterized in that: The training of the pre-trained language model based on the deep learning architecture includes: The language model to be trained based on the deep learning architecture is trained using the training data set to obtain a pre-trained language model; Test the pre-trained language model using the test data set to obtain the test results; Calculate the loss function using the test results and the true label of each piece of data in the test data set; When the loss function meets the threshold, a pre-trained language model is obtained; otherwise, the pre-trained language model is continuously trained by a back propagation algorithm until the loss function meets the threshold; The training data set and the test data set include institutional texts under multiple institutional categories.

5. The method according to claim 1, characterized in that: The step of inputting the third system text vector into a classification model to classify the system text to be classified includes: Performing convolution calculation on the third system text vector through a classification model to obtain one or more feature matrices of the system text to be classified; The institutional text to be classified is classified by using the one or more feature matrices.

6. The method according to claim 5, characterized in that The classifying the institutional text to be classified by using the one or more feature matrices includes: Performing a maximum pooling operation on the one or more feature matrices to obtain one or more pooled feature matrices; Performing multi-classification on one or more feature matrices after pooling by splicing activation layers, and obtaining the probability of each text in the institutional text to be classified under different institutional categories; The classification result of each paragraph of text is determined by the probability, and each paragraph of text in the institutional text to be classified is classified by the classification result.

7. The method according to claim 6, characterized in that Determining the classification result of each text segment by the probability includes: The institutional category corresponding to the maximum probability among the probabilities, or the probability ranking in the top N among all the probabilities, is taken as the classification result of each paragraph of text.

8. A system text classification device, characterized in that: The device comprises: An acquisition module is used to obtain the system text to be classified; A first input module, used for inputting the system text to be classified into a pre-trained document vector model to obtain a first system text vector; A second input module, used for inputting the system text to be classified into a pre-trained language model based on a deep learning architecture to obtain a second system text vector; A calculation module, used for obtaining a third system text vector according to the first system text vector and the second system text vector; The classification module is used to input the third system text vector into the classification model to classify the system text to be classified.

9. An electronic device, characterized in that: The device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the institutional text classification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the institutional text classification method described in any one of claims 1 to 7 when executed.

Images