Model training method, text classification method and device, equipment, medium

Abstract

The present application provides a model training method, a text classification method and apparatus, a device, and a storage medium, which belong to the field of artificial intelligence technology. The training method includes: obtaining original training data, the original training data includes first original data and second original data; upsampling the second original data to obtain initial training data; enhancing the initial training data according to preset enhancement parameters to obtain enhanced training data; encoding the enhanced training data to obtain a target word embedding vector; perturbing the target word embedding vector to obtain target training data; training a preset neural network model according to the first original data and the target training data to obtain a target classification model, wherein the target classification model is a text classification model for classifying target text data. The present application can improve the recognition accuracy of the model for sample text data and the training effect of the model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, a text classification method and apparatus, equipment, and medium. Background Art

[0002] At present, when classifying text, the relevant text data set is often input into a trained supervised learning model, and the relevant text data set is classified through the supervised learning model; however, the commonly used supervised learning model often cannot accurately identify minority text data, which affects the training effect of the model. Therefore, how to improve the recognition accuracy of the model for sample text data and improve the training effect of the model has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a model training method, text classification method and device, equipment, and medium, aiming to improve the model's recognition accuracy of sample text data to improve the training effect of the model.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a model training method, the method is used to train a target classification model, the method comprising:

[0005] Acquire original training data, wherein the original training data includes first original data and second original data;

[0006] Performing upsampling processing on the second original data to obtain initial training data;

[0007] Performing enhancement processing on the initial training data according to preset enhancement parameters to obtain enhanced training data;

[0008] Encoding the enhanced training data to obtain a target word embedding vector;

[0009] Performing perturbation processing on the target word embedding vector to obtain target training data;

[0010] A preset neural network model is trained according to the first original data and the target training data to obtain a target classification model, wherein the target classification model is a text classification model for classifying target text data.

[0011] In some embodiments, the enhancement parameter includes a first disturbance ratio, and the step of performing enhancement processing on the initial training data according to the preset enhancement parameter to obtain enhanced training data includes:

[0012] Obtaining the first sentence length of the initial training data;

[0013] Calculating a first disturbance amount according to the first sentence length and the first disturbance ratio;

[0014] The initial training data is pruned according to the first disturbance amount to obtain the enhanced training data.

[0015] In some embodiments, the enhancement parameter includes a second disturbance ratio, and the step of performing enhancement processing on the initial training data according to the preset enhancement parameter to obtain enhanced training data includes:

[0016] Obtaining a second sentence length of the initial training data;

[0017] Calculating a second disturbance amount according to the second sentence length and the second disturbance ratio;

[0018] The initial training data is expanded according to the second disturbance amount and preset punctuation marks to obtain the enhanced training data.

[0019] In some embodiments, the step of training a preset neural network model according to the first original data and the target training data to obtain a target classification model includes:

[0020] Performing disturbance calculation on the first original data and the target training data through a preset function to obtain a text disturbance value;

[0021] Calculating the loss function of the neural network model according to the text disturbance value to obtain a loss value;

[0022] The loss value is used as the back-propagation amount to adjust the model parameters of the neural network model to train the neural network model and obtain the text classification model.

[0023] To achieve the above purpose, a first aspect of an embodiment of the present application proposes a text classification method, the method comprising:

[0024] Obtain target text data to be classified;

[0025] The target text data is input into a target classification model for label classification processing to obtain label text data, wherein the target classification model is trained according to the training method described in the first aspect.

[0026] In some embodiments, the step of inputting the target text data into a target classification model for label classification processing to obtain label text data includes:

[0027] Mapping the target text data to a preset vector space through the fully connected layer of the target classification model to obtain a target text vector;

[0028] The target text vector is subjected to label classification processing by using the classification function of the fully connected layer and the preset text category label to obtain the label text data.

[0029] To achieve the above-mentioned purpose, a third aspect of an embodiment of the present application proposes a model training device, the device comprising:

[0030] A training data acquisition module, used to acquire original training data, wherein the original training data includes first original data and second original data;

[0031] An upsampling module, used for performing upsampling processing on the second original data to obtain initial training data;

[0032] A data enhancement module, used to enhance the initial training data according to preset enhancement parameters to obtain enhanced training data;

[0033] An encoding module, used to encode the enhanced training data to obtain a target word embedding vector;

[0034] A perturbation module, used to perform perturbation processing on the target word embedding vector to obtain target training data;

[0035] A model training module is used to train a preset neural network model according to the first original data and the target training data to obtain a target classification model, wherein the target classification model is a text classification model used to classify target text data.

[0036] To achieve the above-mentioned purpose, a fourth aspect of the embodiments of the present application provides a text classification device, the device comprising:

[0037] A text data acquisition module, used to acquire target text data to be classified;

[0038] The label classification module is used to input the target text data into the target classification model for label classification processing to obtain label text data, wherein the target classification model is trained according to the training method described in the first aspect.

[0039] To achieve the above-mentioned purpose, the fifth aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the method described in the first aspect or the second aspect is realized.

[0040] To achieve the above-mentioned purpose, the sixth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium used for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method described in the first aspect or the second aspect above.

[0041] The training method, text classification method and device, electronic device and storage medium of the model proposed in the present application obtain original training data, wherein the original training data includes first original data and second original data; the second original data is upsampled to obtain initial training data, which can effectively correct abnormal data in the second original data and improve data rationality. Then, the initial training data is enhanced according to the preset enhancement parameters to obtain enhanced training data, and then the enhanced training data is encoded to obtain the target word embedding vector, and the target word embedding vector is perturbed to obtain the target training data. In this way, the target training data that meets the requirements can be easily obtained, so that the obtained target training data can better highlight the characteristics of the minority class training data and improve the attention of the neural network model to the minority class training data. Finally, the preset neural network model is trained according to the first original data and the target training data, which can improve the recognition accuracy of the model for the sample text data, improve the training effect of the model, and obtain a target classification model that meets the requirements, wherein the target classification model is a text classification model, which can be used to classify the target text data, and the target text data is classified by the target classification model, which can improve the accuracy of text classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the training method of the model provided in the embodiment of the present application;

[0043] Figure 2 yes Figure 1 Flow chart of step S103 in FIG.

[0044] Figure 3 yes Figure 1 Another flow chart of step S103 in FIG. 1 ;

[0045] Figure 4 yes Figure 1 Flow chart of step S106 in FIG.

[0046] Figure 5 is a flowchart of a text classification method provided in an embodiment of the present application;

[0047] Figure 6 yes Figure 5 Flow chart of step S502 in FIG.

[0048] Figure 7 It is a structural schematic diagram of a training device for a model provided in an embodiment of the present application;

[0049] Figure 8 is a structural diagram of a text classification device provided in an embodiment of the present application;

[0050] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0054] First, some nouns involved in this application are analyzed:

[0055] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0056] Natural language processing (NLP): NLP uses computers to process, understand and apply human languages ​​(such as Chinese, English, etc.). NLP is a branch of artificial intelligence and an interdisciplinary subject between computer science and linguistics. It is often referred to as computational linguistics. Natural language processing includes grammatical analysis, semantic analysis, and text understanding. Natural language processing is often used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information intent recognition, information extraction and filtering, text classification and clustering, public opinion analysis and opinion mining. It involves data mining related to language processing, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computing.

[0057] Information Extraction (NER): A text processing technology that extracts specified types of entity, relationship, event and other factual information from natural language text and forms structured data output. Information extraction is a technology that extracts specific information from text data. Text data is composed of some specific units, such as sentences, paragraphs, and chapters. Text information is composed of some small specific units, such as characters, words, phrases, sentences, paragraphs, or a combination of these specific units. Extracting noun phrases, names, place names, etc. from text data are all text information extraction. Of course, the information extracted by text information extraction technology can be various types of information.

[0058] Data SMOTE: Data SMOTE refers to expanding a minority of samples to the same number of samples as the majority of samples. For example, take a data from a minority of samples, find the distance between the sample and other samples, sort them according to the Euclidean distance, and take the top 5 data.

[0059] Data Augmentation: Data augmentation is also called data amplification, which means making limited data produce the equivalent value of more data without substantially increasing the data. Data augmentation can be divided into supervised data augmentation and unsupervised data augmentation methods. Supervised data augmentation can be divided into single-sample data augmentation and multi-sample data augmentation methods, while unsupervised data augmentation can be divided into two directions: generating new data and learning augmentation strategies.

[0060] Encoder: Encoding is to convert the input sequence into a vector of fixed length; decoding is to convert the previously generated fixed vector into an output sequence; the input sequence can be text, voice, image, video; the output sequence can be text or image.

[0061] BERT (Bidirectional Encoder Representations from Transformers): is a language representation model. BERT uses Transformer Encoder block for connection and is a typical bidirectional encoding model.

[0062] Embedding: Embedding is a vector representation, which means using a low-dimensional vector to represent an object. The object can be a word, a product, a movie, etc. The property of this embedding vector is that it can make objects corresponding to vectors with similar distances have similar meanings. For example, the distance between embedding (Avengers) and embedding (Iron Man) will be very close, but the distance between embedding (Avengers) and embedding (Gone with the Wind) will be farther. Embedding is essentially a mapping from semantic space to vector space, while trying to maintain the relationship of the original samples in the semantic space in the vector space as much as possible. For example, the positions of two semantically similar words in the vector space are also relatively close. Embedding can encode objects with low-dimensional vectors and retain their meaning. It is often used in machine learning. In the process of building machine learning models, objects are encoded into a low-dimensional dense vector and then passed to DNN to improve efficiency.

[0063] Softmax classifier: It is a generalization of the logistic regression classifier facing multiple classifications, and the output is the probability value belonging to different categories.

[0064] At present, when classifying text, the relevant text data set is often input into a trained supervised learning model, and the relevant text data set is classified through the supervised learning model; since the training effect of the supervised learning model often depends on the quantity and quality of the training set, in the text classification scenario, there is a widespread problem of unbalanced training data. The sample categories that need to be paid attention to are often minority sample categories, and the minority sample categories account for a small proportion of the entire data set. If the data is directly input into the model training, the model tends to predict all samples as the majority class, and the recognition accuracy of minority class sample data is poor. Therefore, how to improve the recognition accuracy of the model for sample text data to improve the training effect of the model has become a technical problem that needs to be solved urgently.

[0065] Based on this, the embodiments of the present application provide a model training method, a text classification method and apparatus, a device, and a medium, which aim to improve the model's recognition accuracy of sample text data, thereby improving the model's training effect.

[0066] The model training method, text classification method and apparatus, device, and medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the model training method in the embodiments of the present application is described.

[0067] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0068] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0069] The training method of the model, the text classification method and apparatus, equipment, and medium provided in the embodiments of the present application relate to the field of artificial intelligence technology. The training method of the model, the text classification method and apparatus, equipment, and medium provided in the embodiments of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements a text classification method, etc., but is not limited to the above forms.

[0070] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0071] Figure 1 is an optional flowchart of the training method of the model provided in the embodiment of the present application, Figure 1 The method may include but is not limited to steps S101 to S106.

[0072] Step S101, obtaining original training data, wherein the original training data includes first original data and second original data;

[0073] Step S102, performing upsampling processing on the second original data to obtain initial training data;

[0074] Step S103, performing enhancement processing on the initial training data according to preset enhancement parameters to obtain enhanced training data;

[0075] Step S104, encoding the enhanced training data to obtain a target word embedding vector;

[0076] Step S105, performing perturbation processing on the target word embedding vector to obtain target training data;

[0077] Step S106, training a preset neural network model according to the first original data and the target training data to obtain a target classification model, wherein the target classification model is a text classification model for classifying the target text data.

[0078] Steps S101 to S106 shown in the embodiment of the present application, by upsampling the second original data to obtain initial training data, can effectively correct abnormal data in the second original data and improve data rationality. The initial training data is enhanced according to preset enhancement parameters to obtain enhanced training data, and then the enhanced training data is encoded to obtain a target word embedding vector, and the target word embedding vector is perturbed to obtain target training data, which can easily obtain target training data that meets the requirements, so that the obtained target training data can better highlight the characteristics of minority class training data and improve the attention of the neural network model to minority class training data. Training the preset neural network model according to the first original data and the target training data can improve the model's recognition accuracy of sample text data, improve the model's training effect, and obtain a target classification model that meets the requirements.

[0079] In step S101 of some embodiments, a web crawler can be written, and after setting a data source, data can be crawled in a targeted manner to obtain sample data. Sample data can also be obtained in other ways, not limited to this. It should be noted that the sample data is text data with text category labels. According to a preset ratio parameter, the sample data is divided into original training data, original verification data, and original test data. In order to improve the training effect of the model, it is necessary to perform data enhancement processing on the original training data. Specifically, firstly, data statistics are performed on the original training data to obtain the number of samples of each text category in the original training data. According to the number of samples corresponding to each text category label, the original training data is divided into the first original data and the second original data, that is, the first original data and the second original data can be distinguished according to the text category label on the original training data. The first original data is the original training data with a sample number greater than a preset number threshold, marked as the majority class sample data label 0, and the second original data is the original training data with a sample number less than or equal to the preset number threshold, marked as the minority class sample data label 1, wherein the majority class sample data label0 (i.e., the first original data) is not subjected to data enhancement processing, and the minority class sample data label 1 (i.e., the second original data) needs to be subjected to data enhancement processing. For example, if the number of samples of the majority class sample data label 0 (i.e., the first original data) is m, and the number of samples of the minority class sample data label 1 (i.e., the second original data) is n, then the minority class sample data label 1 needs to be enhanced. 1 (i.e., the second original data) is data enhanced to obtain mn sample data, so that the sample quantity ratio of the enhanced second original data to the first original data is 1:1.

[0080] In step S102 of some embodiments, the minority class sample data label 1 (i.e., the second original data) that needs to be enhanced is randomly upsampled, and the sampling number is the majority class sample data label 0 (i.e., the first original data). Therefore, the second original data after sampling will generate mn repeated sample data, thereby obtaining new training data, which is recorded as initial training data.

[0081] See also Figure 2 In some embodiments, the enhancement parameter includes a first disturbance ratio, and step S103 may include but is not limited to steps S201 to S203:

[0082] Step S201, obtaining the length of the first sentence of the initial training data;

[0083] Step S202, calculating a first disturbance amount according to the first sentence length and the first disturbance ratio;

[0084] Step S203: perform deletion processing on the initial training data according to the first disturbance amount to obtain enhanced training data.

[0085] In step S201 of some embodiments, the first sentence length s1 of each text sentence in the initial training data set is counted in characters. For example, if a text sentence consists of five characters and three punctuation marks, then the first sentence length s1 of the text sentence is 8.

[0086] In step S202 of some embodiments, the first disturbance ratio can be set according to actual needs. For example, the first disturbance ratio r1 is set to 0.1, and the first disturbance amount d1 is calculated according to the first sentence length s1 and the first disturbance ratio r1. The first disturbance amount d1 can be the value corresponding to s1*r1 when rounded, that is, the first disturbance amount d1 is int(s1*r1).

[0087] In step S203 of some embodiments, int(s1*r1) positions are randomly selected from the current text sentence as replacement positions, and the characters at these replacement positions are replaced with blanks, thereby achieving deletion processing of the text sentences of the initial training data and obtaining enhanced training data.

[0088] See also Figure 3 In some other embodiments, the enhancement parameter includes a second disturbance ratio, and step S103 may include but is not limited to steps S301 to S303:

[0089] Step S301, obtaining the second sentence length of the initial training data;

[0090] Step S302, calculating a second disturbance amount according to the second sentence length and the second disturbance ratio;

[0091] Step S303: Expand the initial training data according to the second disturbance amount and the preset punctuation marks to obtain enhanced training data.

[0092] In step S301 of some embodiments, the second sentence length s2 of each text sentence in the initial training data set is counted in units of characters. For example, if a text sentence consists of six characters and two punctuation marks, then the second sentence length s2 of the text sentence is 8.

[0093] In step S302 of some embodiments, the second disturbance ratio can be set according to actual needs. For example, the second disturbance ratio r2 is set to 0.1, and the second disturbance amount d2 is calculated according to the second sentence length s2 and the second disturbance ratio r2. The second disturbance amount d2 can be the value corresponding to s2*r2 when rounded, that is, the second disturbance amount d2 is int(s2*r2).

[0094] In step S303 of some embodiments, the preset punctuation mark p is a neutral mark, such as a comma, a colon, a semicolon, a period, an ellipsis, etc. Randomly select int(s2*r2) positions from the current text sentence as replacement positions, randomly extract int(s2*r2) symbols from p (repeated extraction is allowed), and replace the characters at the replacement positions with punctuation marks, thereby expanding the text sentences of the initial training data and obtaining enhanced training data.

[0095] It should be noted that the first disturbance ratio and the second disturbance ratio can be understood as enhancement ratios, which are used to determine the proportion of characters that need to be modified in a text sentence, and the first disturbance amount and the second disturbance amount can be understood as the number of enhanced characters, which are used to determine the number of characters that need to be modified in a text sentence.

[0096] Taking steps S201 to S203 as an example, setting the first disturbance ratio r1 to 0.1 indicates that 10% of the characters in a text sentence need to be modified. If the length of a text sentence is 10, the first disturbance amount d1 is int(10*0.1)=1. The number of characters that need to be modified in the text sentence is 1. Then, a position is randomly selected in the text sentence as a replacement position, and the character at the replacement position is replaced with empty, thereby achieving deletion processing of the text sentence and obtaining enhanced training data.

[0097] It should be noted that when performing data enhancement processing on the initial training data, the above two data enhancement methods can be selected for data enhancement at the same time, or one of the data enhancement methods can be used alone for data enhancement. For example, in order to improve the efficiency of data enhancement, the above two data enhancement methods are selected at the same time to perform data enhancement on the initial training data, and the proportion of one of the data enhancement methods is set to k. Then, there are (mn)*k sample data in the initial training data that use this method for data enhancement processing, and other sample data in the initial training data use another data enhancement method for data enhancement processing. For example, the above steps S201 to S203 are used to delete (mn)*k sample data, and the above steps S301 to S303 are used to expand the other sample data except for (mn)*k sample data, so as to obtain enhanced training data.

[0098] In step S104 of some embodiments, the enhanced training data can be encoded using a BERT encoder to obtain a target word embedding vector. Since BERT uses a Transformer Encoder block for connection, it is a typical bidirectional encoding model. Therefore, the enhanced training data can be bidirectionally encoded using a BERT encoder, that is, the enhanced training data is encoded from left to right and from right to left, respectively, to obtain a target word embedding vector (token embedding).

[0099] In step S105 of some embodiments, when the target word embedding vector is perturbated, the target word embedding vector (token embedding) can be perturbated along the gradient direction according to a preset perturbation factor. The preset perturbation factor can be expressed as a word embedding weight matrix, that is, the target word embedding vector and the preset word embedding weight matrix are matrix multiplied along the gradient direction to obtain the target training data.

[0100] See also Figure 4 In some embodiments, step S106 may include but is not limited to steps S401 to S403:

[0101] Step S401, performing disturbance calculation on the first original data and the target training data by using a preset function to obtain a text disturbance value;

[0102] Step S402, calculating the loss function of the neural network model according to the text disturbance value to obtain a loss value;

[0103] Step S403, using the loss value as the back propagation amount, adjusting the model parameters of the neural network model to train the neural network model and obtain a text classification model.

[0104] In step S401 of some embodiments, the first original data and the target training data are first input into a preset neural network model, and the number of iterations (epoches_num) and the data batch size (batchsize) of the neural network model are set, and the first original data and the target training data are divided into multiple batches according to the data batch size to obtain batch data. The preset function is a cross entropy function.

[0105] Specifically, in each iteration process, cross entropy is used to obtain the loss value loss1 of each batch of data, and the parameter gradient of the batch of data is calculated. i The original gradient value grad_β i Divide by the norm L2 of the original parameter and multiply by a hyperparameter α to get the text perturbation value, and add the text perturbation value to the original parameter to get the intermediate parameter β′ for each batch of data i , the calculation process is shown in formula (1):

[0106]

[0107] Among them, the norm The text perturbation value is The value range of the hyperparameter α is (0, 1]. If you want to add a larger disturbance to the target word embedding vector, set the hyperparameter α to a larger value. After multiple verifications, when the hyperparameter α is 0.1 to 0.3, the training effect of the model is better.

[0108] Further, calculate the intermediate parameter β′ i With the original parameter β i The absolute value of the difference between i , and set a threshold ε, where the value range of ε is (0, 1], so that the parameter ε is used to control whether the disturbance is added to the original parameter.

[0109] For example, if r i >ε, then ε*r i / Norm(r i ) is added as a perturbation to the original parameter β i The final target parameter β″ is obtained i , the calculation formula is shown in formula (2) and formula (3):

[0110] r i = abs(β′ i -β i ) Formula (2)

[0111]

[0112] It should be noted that the larger the value of ε is, the more difficult it is to add the disturbance to the parameter matrix of the original parameters. After multiple verifications, when ε is between 0.8 and 1, the training effect of the model is better.

[0113] Furthermore, in order to improve the training effect of the model, the parameter k is set to control the number of disturbances, and the above intermediate parameter calculation process and target parameter calculation process are cycled k times. Since too many disturbances will bring too much noise and affect the prediction accuracy of the neural network model in various text categories, the number of disturbances is generally set to 2 or 3 times to obtain the final text disturbance value.

[0114] In step S402 of some embodiments, the loss function of the neural network model is calculated according to the final text perturbation value to obtain a loss value. Specifically, the loss function corresponding to the fully connected layer of the neural network model can be calculated to obtain the loss value.

[0115] In step S403 of some embodiments, the loss value is used as the back-propagation amount to adjust the model parameters of the neural network model to train the neural network model and obtain a text classification model, so that the labeled text data generated by the neural network model is more accurate, thereby improving the recognition accuracy of the neural network model for minority class text data.

[0116] The training method of the model of the embodiment of the present application obtains the original training data, wherein the original training data includes the first original data and the second original data; the second original data is upsampled to obtain the initial training data, which can effectively correct the abnormal data in the second original data and improve the rationality of the data. Then, the initial training data is enhanced according to the preset enhancement parameters to obtain the enhanced training data, and then the enhanced training data is encoded to obtain the target word embedding vector, and the target word embedding vector is perturbed to obtain the target training data. In this way, the target training data that meets the requirements can be easily obtained, so that the obtained target training data can better highlight the characteristics of the minority class training data and improve the attention of the neural network model to the minority class training data. Finally, the preset neural network model is trained according to the first original data and the target training data, which can improve the recognition accuracy of the model for the sample text data, improve the training effect of the model, and obtain a target classification model that meets the requirements.

[0117] Figure 5 is an optional flowchart of the text classification method provided in the embodiment of the present application. Figure 5 The method may include but is not limited to steps S501 to S502.

[0118] Step S501, obtaining target text data to be classified;

[0119] Step S502: input the target text data into a target classification model for label classification processing to obtain label text data, wherein the target classification model is trained according to the training method of the first aspect embodiment.

[0120] In step S501 of some embodiments, a web crawler may be written, and after setting a data source, data may be crawled in a targeted manner to obtain target text data to be classified. Sample data may also be obtained in other ways, not limited thereto. It should be noted that the target text data may be an article, a text field, a text segment, and the like.

[0121] In step S502 of some embodiments, the target text data is input into a target classification model, the target text data is mapped to a preset vector space through the target classification model to obtain a target text vector, and the target text vector is subjected to label classification processing through a preset classification function to obtain label text data.

[0122] See also Figure 6 In some embodiments, step S502 may also include but is not limited to steps S601 to S602:

[0123] Step S601, mapping the target text data to a preset vector space through the fully connected layer of the target classification model to obtain a target text vector;

[0124] Step S602, performing label classification processing on the target text vector through the classification function of the fully connected layer and the preset text category label to obtain label text data.

[0125] In step S601 of some embodiments, the characteristic dimension of a preset text category label is obtained, and the target text data is mapped from the semantic space to the vector space through the MLP network of the fully connected layer, and the target text data is mapped to the vector space with the same characteristic dimension as the preset text category label to obtain a target text vector.

[0126] In step S602 of some embodiments, the classification function may be a softmax function. For example, a probability distribution is created on each text category label through the softmax function to obtain a predicted probability value that the target text vector belongs to each text category. Finally, according to the size of the classification probability value, the target text vector is subjected to text category judgment and labeling processing to obtain label text data.

[0127] It should be noted that the preset text category labels can be set according to actual needs, and the text category labels in different business scenarios may be different. For example, in the application scenario of book classification, the preset text category labels include classical literature, foreign literature, prose, novels, poetry, etc. In daily life scenarios, the preset text category labels may include traffic, weather conditions, time information, etc.

[0128] The text classification method of the embodiment of the present application obtains target text data to be classified, and inputs the target text data into a target classification model for label classification processing. The target classification model has good recognition accuracy for minority text data. The target classification model can recognize target text data of different categories, and classify the target text data according to different category labels to obtain labeled text data, thereby improving the accuracy of text classification.

[0129] See also Figure 7 The present application also provides a model training device, which can implement the above-mentioned model training method. The model training device includes:

[0130] A training data acquisition module 701 is used to acquire original training data, wherein the original training data includes first original data and second original data;

[0131] An up-sampling module 702 is used to perform up-sampling processing on the second original data to obtain initial training data;

[0132] The data enhancement module 703 is used to enhance the initial training data according to preset enhancement parameters to obtain enhanced training data;

[0133] An encoding module 704 is used to encode the enhanced training data to obtain a target word embedding vector;

[0134] A perturbation module 705 is used to perform perturbation processing on the target word embedding vector to obtain target training data;

[0135] The model training module 706 is used to train a preset neural network model according to the first original data and the target training data to obtain a target classification model, wherein the target classification model is a text classification model used to classify the target text data.

[0136] In some embodiments, the data enhancement module 703 includes:

[0137] A first sentence length obtaining unit, used to obtain a first sentence length of initial training data;

[0138] A first disturbance amount calculation unit, used for calculating a first disturbance amount according to a first sentence length and a first disturbance ratio;

[0139] The data deletion unit is used to delete the initial training data according to the first disturbance amount to obtain enhanced training data.

[0140] In some other embodiments, the data enhancement module 703 includes:

[0141] A second sentence length obtaining unit, used to obtain a second sentence length of the initial training data;

[0142] A second disturbance amount calculation unit, used for calculating a second disturbance amount according to a second sentence length and a second disturbance ratio;

[0143] The data expansion unit is used to expand the initial training data according to the second disturbance amount and the preset punctuation marks to obtain enhanced training data.

[0144] In some embodiments, the model training module 706 includes:

[0145] A disturbance calculation unit, used to perform disturbance calculation on the first original data and the target training data through a preset function to obtain a text disturbance value;

[0146] A loss value calculation unit, used to calculate the loss function of the neural network model according to the text disturbance value to obtain a loss value;

[0147] The training unit is used to use the loss value as a back-propagation amount to adjust the model parameters of the neural network model so as to train the neural network model and obtain a text classification model.

[0148] The model training device of the embodiment of the present application is used to execute the model training method in the above embodiment. Its specific processing process is the same as the model training method in the above embodiment, and will not be repeated here.

[0149] See also Figure 8 The present application also provides a text classification device, which can implement the above-mentioned text classification method. The text classification device includes:

[0150] The text data acquisition module 801 is used to acquire target text data to be classified;

[0151] The label classification module 802 is used to input the target text data into the target classification model for label classification processing to obtain label text data, wherein the target classification model is trained according to the training method of any one of the embodiments of the first aspect.

[0152] In some embodiments, the tag classification module 802 includes:

[0153] A mapping unit, used to map the target text data to a preset vector space through a fully connected layer of the target classification model to obtain a target text vector;

[0154] The label classification unit is used to perform label classification processing on the target text vector through the classification function of the fully connected layer and the preset text category label to obtain label text data.

[0155] The text classification device of the embodiment of the present application is used to execute the text classification method in the above embodiment. Its specific processing process is the same as the text classification method in the above embodiment, and will not be repeated here.

[0156] The embodiment of the present application also provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the program is executed by the processor, the training method of the above-mentioned model or the text classification method is realized. The electronic device may be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0157] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0158] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0159] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the model training method or text classification method of the embodiment of this application;

[0160] Input / output interface 903, used to implement information input and output;

[0161] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0162] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0163] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0164] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned model training method or text classification method.

[0165] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0166] The model training method, model training device, text classification method, text classification device, electronic device and storage medium provided in the embodiment of the present application are obtained by acquiring original training data, wherein the original training data includes first original data and second original data; upsampling the second original data to obtain initial training data, which can effectively correct abnormal data in the second original data and improve data rationality. Then, the initial training data is enhanced according to the preset enhancement parameters to obtain enhanced training data, and then the enhanced training data is encoded to obtain the target word embedding vector, and the target word embedding vector is perturbed to obtain the target training data. In this way, the target training data that meets the requirements can be easily obtained, so that the obtained target training data can better highlight the characteristics of the minority class training data and improve the attention of the neural network model to the minority class training data. Finally, the preset neural network model is trained according to the first original data and the target training data, which can improve the recognition accuracy of the model for the sample text data, improve the training effect of the model, and obtain a target classification model that meets the requirements, wherein the target classification model is a text classification model, which can be used to classify the target text data, and the target text data is classified by the target classification model, which can improve the accuracy of text classification.

[0167] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0168] It can be understood by those skilled in the art that Figure 1-4 , Figure 5-6 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0169] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0170] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0171] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0172] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0173] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0174] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0176] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0177] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A model training method, characterized in that: The method is used to train a target classification model, and the method comprises: Acquire original training data, wherein the original training data includes first original data and second original data; Performing upsampling processing on the second original data to obtain initial training data; Performing enhancement processing on the initial training data according to preset enhancement parameters to obtain enhanced training data; Encoding the enhanced training data to obtain a target word embedding vector; Performing perturbation processing on the target word embedding vector to obtain target training data; Training a preset neural network model according to the first original data and the target training data to obtain a target classification model, wherein the target classification model is a text classification model for classifying target text data; The enhancement parameters include a first disturbance ratio and a second disturbance ratio; the initial training data is enhanced according to the preset enhancement parameters to obtain enhanced training data, including: Dividing the initial training data into first sample data and second sample data according to a preset ratio; Obtaining a first sentence length of first sample data, taking the integer part of the product of the first sentence length and the first disturbance ratio as a first disturbance amount, and performing a deletion process on the first sample data according to the first disturbance amount; obtaining a second sentence length of second sample data, taking the integer part of the product of the second sentence length and the second disturbance ratio as a second disturbance amount; performing an expansion process on the second sample data according to the second disturbance amount and a preset punctuation mark to obtain the enhanced training data; The perturbation processing is performed on the target word embedding vector to obtain target training data, including: matrix multiplication of the target word embedding vector and a preset perturbation factor along a gradient direction to obtain target training data, wherein the preset perturbation factor is a word embedding weight matrix.

2. The training method according to claim 1, characterized in that: The step of training a preset neural network model according to the first original data and the target training data to obtain a target classification model includes: Performing disturbance calculation on the first original data and the target training data through a preset function to obtain a text disturbance value; Calculating the loss function of the neural network model according to the text disturbance value to obtain a loss value; The loss value is used as the back-propagation amount to adjust the model parameters of the neural network model to train the neural network model and obtain the text classification model.

3. A text classification method, characterized in that: The method comprises: Obtain target text data to be classified; The target text data is input into a target classification model for label classification processing to obtain label text data, wherein the target classification model is trained according to the training method according to any one of claims 1 to 2.

4. The text classification method according to claim 3, characterized in that: The step of inputting the target text data into a target classification model for label classification processing to obtain label text data comprises: Mapping the target text data to a preset vector space through the fully connected layer of the target classification model to obtain a target text vector; The target text vector is subjected to label classification processing by using the classification function of the fully connected layer and the preset text category label to obtain the label text data.

5. A model training device, characterized in that: The device comprises: A training data acquisition module, used to acquire original training data, wherein the original training data includes first original data and second original data; An upsampling module, used for performing upsampling processing on the second original data to obtain initial training data; A data enhancement module, used to enhance the initial training data according to preset enhancement parameters to obtain enhanced training data; An encoding module, used to encode the enhanced training data to obtain a target word embedding vector; A perturbation module, used to perform perturbation processing on the target word embedding vector to obtain target training data; A model training module, used for training a preset neural network model according to the first original data and the target training data to obtain a target classification model, wherein the target classification model is a text classification model, used for classifying target text data; The enhancement parameters include a first disturbance ratio and a second disturbance ratio; the initial training data is enhanced according to the preset enhancement parameters to obtain enhanced training data, including: Dividing the initial training data into first sample data and second sample data according to a preset ratio; Obtaining a first sentence length of first sample data, taking the integer part of the product of the first sentence length and the first disturbance ratio as a first disturbance amount, and performing a deletion process on the first sample data according to the first disturbance amount; obtaining a second sentence length of second sample data, taking the integer part of the product of the second sentence length and the second disturbance ratio as a second disturbance amount; performing an expansion process on the second sample data according to the second disturbance amount and a preset punctuation mark to obtain the enhanced training data; The perturbation processing is performed on the target word embedding vector to obtain target training data, including: matrix multiplication of the target word embedding vector and a preset perturbation factor along a gradient direction to obtain target training data, wherein the preset perturbation factor is a word embedding weight matrix.

6. A text classification device, characterized in that: The device comprises: A text data acquisition module, used to acquire target text data to be classified; The label classification module is used to input the target text data into a target classification model for label classification processing to obtain label text data, wherein the target classification model is trained according to the training method according to any one of claims 1 to 2.

7. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the training method of the model as described in any one of claims 1 to 2 or the steps of the text classification method as described in any one of claims 3 to 4 are realized.

8. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the model training method as described in any one of claims 1 to 2 or the steps of the text classification method as described in any one of claims 3 to 4.

Images