Industry early warning method and device based on multi-task joint learning
Through the multi-task joint learning method, combined with the BERT model and early warning characteristic words, the accuracy of the public opinion analysis platform to identify sensitive data in specific industries is solved, and the accurate identification and push of data in specific industries is achieved, which avoids confusion in the entire industry and improves the improvement of data in the entire industry. The accuracy of the warning results.
Patent Information
- Application Number
- CN202211522717.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-30
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-11-30
AI Technical Summary
The existing public opinion analysis platform cannot accurately identify sensitive data in specific industries and push them to users in the corresponding industry, resulting in confusion and interference in data across the industry and being unable to adapt to the early warning needs of different industries.
Using a multi-task joint learning method, we can classify emotions by obtaining training corpus, identify negative characteristic words in the industry, build a BERT model based on label corpus and early warning characteristic words, conduct overall training, and form a multi-task joint early warning model for identification and extraction Warning characteristic word, determine whether it is warning data.
It improves the accuracy of identifying sensitive data for specific industries, avoids confusing push of sensitive data across the industry, and enhances the accuracy and pertinence of early warning results.
Smart Images

Figure CN115905536B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of information security technology, and specifically relates to an industry early warning method and device based on multi-task joint learning. Background Art
[0002] Existing public opinion analysis platforms only issue warnings for negative data. However, some negative data are particularly sensitive to specific industries, while others are not, which makes it impossible to accurately issue warnings for sensitive data in specific industries. In addition, the warning data only provides a warning label, but cannot accurately give warning feature vocabulary to prove the necessity of warning.
[0003] In the related technologies, most of the existing solutions use text classification to identify warning data, including multi-model stacking, large model pre-training, small model distillation, etc. These solutions use a large amount of training data to train semantic feature vectors, and then accurately identify warning data through contextual semantic features. They can achieve semantic disambiguation, but they cannot provide warning features, and people cannot identify it as warning data at a glance, nor can they be applied to warnings in different industries.
[0004] In summary, for the full amount of data on the existing public opinion analysis platform, some industry-sensitive data needs to be warned for different industries, so that users in different industries can conduct research and management on public opinion events in their own industries at the first time. How to accurately identify sensitive data of a specific industry from the data of the entire industry and push it to users in a specific industry to avoid the existing confusion of sensitive data from the entire industry and interfere with users is an urgent problem that needs to be solved. Summary of the invention
[0005] In view of this, the purpose of the present invention is to overcome the shortcomings of the prior art and provide an industry early warning method and device based on multi-task joint learning to solve the problem of how to accurately identify sensitive data of a specific industry from data of the entire industry in the prior art, and push it to users in the specific industry, so as to avoid the problem of the existing industry-wide sensitive confusion push that interferes with users.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: an industry early warning method based on multi-task joint learning, comprising:
[0007] Acquire training corpus, perform sentiment classification on the training corpus, and obtain label corpus;
[0008] Determine an industry text corpus, and select negative feature words corresponding to the industry from the industry text corpus as warning feature words;
[0009] Combining the label corpus and the warning feature words to obtain a bigram label;
[0010] Based on the labeled corpus, the pre-trained model BERT is used to build the first task model for sentiment polarity classification; based on the warning feature words, the pre-trained model BERT is used to build the second task model for extracting warning feature words;
[0011] The binary labels are used as a training data set to train the combined first task model and the second task model as a whole to obtain a multi-task joint warning model;
[0012] The text to be predicted is input into the multi-task joint warning model to obtain a warning result.
[0013] Furthermore, the selecting of negative characteristic words corresponding to the industry from the industry text corpus as warning characteristic words includes:
[0014] Collect negative words about the preset industry;
[0015] Collect negative Internet data of preset industries;
[0016] The negative words are added to the custom dictionary through the N-Gram model, and the negative Internet data is segmented, and the negative words containing three consecutive negative words are saved, and the word frequency is counted;
[0017] The negative words whose word frequency is lower than a preset threshold are filtered out, and the remaining negative words are used as warning feature words.
[0018] Furthermore, the combination of the label corpus and the warning feature words to obtain a bigram label includes:
[0019] The warning feature words and the label corpus are matched to obtain a bigram label.
[0020] Furthermore, the method of constructing a first task model based on the labeled corpus using the pre-trained model BERT includes:
[0021] Input the labeled corpus into the pre-trained model BERT, and output the classification task cls and sentiment polarity tokens sequence;
[0022] Input the classification task cls and the sentiment polarity tokens sequence into the pre-built textcnn model, and output the first feature matrix;
[0023] Combining the classification task cls with the first feature matrix to obtain a second feature matrix;
[0024] The second feature matrix is modeled to obtain a first task model.
[0025] Furthermore, the second task model is constructed based on the warning feature words using the pre-trained model BERT, including:
[0026] Input the warning feature words into the pre-trained model BERT and output the feature word tokens sequence;
[0027] Inputting the feature word tokens sequence into a pre-built single pointer network model to obtain a third feature matrix;
[0028] The third feature matrix is modeled to obtain a second task model.
[0029] Furthermore, the binary labels are used as training data sets to train the combined first task model and second task model as a whole to obtain a multi-task joint warning model, including:
[0030] Through training, a first loss function of the first task model and a second loss function of the second task model are obtained respectively;
[0031] Perform weighted calculation on the first loss function and the second loss function with specific weights to obtain a total loss function;
[0032] A multi-task joint early warning model is determined according to the total loss function.
[0033] Furthermore, the first loss function and the second loss function are weighted by specific weights to obtain a total loss function, including:
[0034] Loss = αloss2 + (1-α)loss1
[0035] Among them, Loss is the total loss function, α is the specific weight, loss1 is the first loss function, and loss2 is the second loss function.
[0036] Furthermore, the label corpus includes positive, neutral and negative.
[0037] The embodiment of the present application provides an industry early warning device based on multi-task joint learning, including:
[0038] An acquisition module is used to acquire training corpus, perform sentiment classification on the training corpus, and obtain label corpus;
[0039] A determination module, used for determining industry text corpus, and selecting negative characteristic words corresponding to the industry from the industry text corpus as warning characteristic words;
[0040] A combining module, used for combining the label corpus and the warning feature words to obtain a bigram label;
[0041] A construction module is used to construct a first task model based on the labeled corpus using the pre-trained model BERT for sentiment polarity classification; a second task model is constructed based on the warning feature words using the pre-trained model BERT for extracting warning feature words;
[0042] A training module, used to use the binary labels as a training data set to perform overall training on the combined first task model and second task model to obtain a multi-task joint warning model;
[0043] The prediction module is used to input the text to be predicted into the multi-task joint warning model to obtain a warning result.
[0044] The present invention adopts the above technical solution, and the beneficial effects that can be achieved include:
[0045] The present invention provides an industry early warning method and device based on multi-task joint learning. The application first performs sentiment classification on the training corpus to obtain a label corpus, and selects the corresponding negative feature words of the industry from the industry text corpus as the warning feature words, and uses the bigram label composed of the label corpus and the warning feature words as the training set to perform overall training on the constructed first task model and the second task model to obtain a multi-task joint early warning model, so that the input text to be predicted can be predicted through the multi-task joint early warning model to determine whether an early warning is needed, that is, the early warning result. The present application is trained through two task models. The first task model is used to judge the positive and negative sentiment polarity; the second task model can extract the warning feature vocabulary; and through the joint learning of the two tasks, it is determined whether it is early warning data. The accuracy of the judgment is improved, and the existing industry-wide sensitive confusion push interference to users is effectively avoided. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0047] Figure 1 It is a schematic diagram of the steps of the industry early warning method based on multi-task joint learning of the present invention;
[0048] Figure 2 A flow chart of the binary tag formation provided by the present invention;
[0049] Figure 3 A schematic diagram of a flow chart of a first task model provided by the present invention;
[0050] Figure 4A schematic diagram of the structure of the second task model provided by the present invention;
[0051] Figure 5 A schematic diagram of the structure of the multi-task joint warning model provided by the present invention;
[0052] Figure 6 It is a structural schematic diagram of the industry early warning device based on multi-task joint learning of the present invention. DETAILED DESCRIPTION
[0053] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.
[0054] A specific industry early warning method and device based on multi-task joint learning provided in an embodiment of the present application are introduced below in conjunction with the accompanying drawings.
[0055] like Figure 1 As shown, the industry early warning method based on multi-task joint learning provided in the embodiment of the present application includes:
[0056] S101, obtaining training corpus, performing sentiment classification on the training corpus, and obtaining label corpus;
[0057] The training corpus in this application is some data collected from all industries, and then the sentiment labels are manually assigned. The sentiment labels include positive, neutral and negative.
[0058] S102, determining industry text corpus, and selecting negative feature words corresponding to the industry from the industry text corpus as warning feature words;
[0059] The technical solution provided by this application can be used in multiple industries. There are special arrangements for specific fields, and the warning feature words in different industry fields are different. For example, in the public security field, the feature words can be A, B, C, D, etc.
[0060] S103, combining the label corpus and the warning feature words to obtain a bigram label;
[0061] Combining the label corpus and the warning feature words can obtain a bigram label, for example, (negative, A), (negative, B).
[0062] S104, based on the label corpus, a pre-trained model BERT is used to build a first task model for sentiment polarity classification; based on the warning feature words, a pre-trained model BERT is used to build a second task model for extracting warning feature words;
[0063] In this application, the first task model is used to identify positive and negative sentiment polarities; the second task model is used to extract warning feature vocabulary.
[0064] S105, using the binary label as a training data set, training the combined first task model and the second task model as a whole to obtain a multi-task joint warning model;
[0065] In this application, the first task model and the second task model are trained as a whole, and the total loss function is obtained by combining the loss function of the first task model and the loss function of the second task model, and the multi-task joint early warning model is obtained according to the total loss function.
[0066] S106, inputting the text to be predicted into the multi-task joint warning model to obtain a warning result.
[0067] After obtaining the multi-task warning model, the text to be predicted can be input into the multi-task warning model to obtain the result of whether a warning is needed.
[0068] The working principle of the industry early warning method based on multi-task joint learning is as follows: the present invention first performs sentiment classification on the training corpus to obtain the label corpus, and selects the corresponding negative feature words of the industry from the industry text corpus as the warning feature words, and uses the bigram label composed of the label corpus and the warning feature words as the training set to perform overall training on the constructed first task model and the second task model to obtain the multi-task joint early warning model, so that the input text to be predicted can be predicted through the multi-task joint early warning model to determine whether an early warning is needed, that is, the early warning result.
[0069] The industry warning method based on multi-task joint learning provided by the present application can extract the sentiment polarity of Internet data at the semantic level, and extract the warning feature vocabulary of its specific industry through the pointer network, and accurately determine whether the data is sensitive data for a specific industry through the joint calculation of semantic sentiment features and warning feature vocabulary. Specifically, it is through joint training of two task models, the first task model is used to judge positive and negative sentiment polarity; the second task model can extract warning feature vocabulary; through the joint learning of the two tasks, it is determined whether it is warning data. The technical solution of the present application improves the accuracy of judgment, and effectively avoids the existing industry-wide sensitive confusion push interference to users.
[0070] In some embodiments, the selecting of negative characteristic words corresponding to the industry from the industry text corpus as warning characteristic words includes:
[0071] Collect negative words about the preset industry;
[0072] Collect negative Internet data of preset industries;
[0073] The negative words are added to the custom dictionary through the N-Gram model, and the negative Internet data is segmented, and the negative words containing three consecutive negative words are saved, and the word frequency is counted;
[0074] The negative words whose word frequency is lower than a preset threshold are filtered out, and the remaining negative words are used as warning feature words.
[0075] Specifically, assuming that in the public security field, this application first manually collects negative words, such as "A", "B", "C", "D", etc., short negative words. Then, collect negative Internet data in the public security field, add the above negative words to the custom dictionary through the N-Gram model, and segment the negative data, save the three consecutive words containing the above negative words, and count the word frequency; filter the saved triplets according to the specific (manually set) word frequency, filter the low-frequency words, and use the remaining words as warning feature words. Repeat the above steps to automatically learn and accumulate warning feature words.
[0076] In some embodiments, combining the label corpus and the warning feature word to obtain a bigram label includes:
[0077] The warning feature words and the label corpus are matched to obtain a bigram label.
[0078] like Figure 2 As shown, in this application, feature vocabulary is constructed through automatic learning, and the above training corpus is matched with words, and each piece of information is labeled with a bigram, such as a bigram: (neutral, "kindergarten E event"), (positive, "kindergarten F event"), (negative, "A event"), (negative, "none"); where "none" in the bigram means that the three conjunctions in 2) are not matched. The training corpus and the labels of the corpus are constructed.
[0079] In some embodiments, the step of constructing a first task model based on the labeled corpus using a pre-trained model BERT includes:
[0080] Input the labeled corpus into the pre-trained model BERT, and output the classification task cls and sentiment polarity tokens sequence;
[0081] Input the classification task cls and the sentiment polarity tokens sequence into the pre-built textcnn model, and output the first feature matrix;
[0082] Combining the classification task cls with the first feature matrix to obtain a second feature matrix;
[0083] The second feature matrix is modeled to obtain a first task model.
[0084] like Figure 3 As shown, in this application, the labeled corpus is first used as input and input into the pre-trained model BERT. The output cls of the pre-trained model BERT and the tokens sequence output by the pre-trained model BERT are input into the feature matrix obtained by the textcnn model. The cls and the feature matrix are concatenated to obtain a new feature matrix, and this feature matrix is modeled to identify the sentiment polarity of the text. That is, the first task model is used to identify sentiment polarity. It should be understood that the textcnn model in this application includes one-dimensional convolution, one-dimensional pooling, multi-core concatenation, classification layer and class label.
[0085] In some embodiments, the method of constructing a second task model based on the warning feature words using a pre-trained model BERT includes:
[0086] Input the warning feature words into the pre-trained model BERT and output the feature word tokens sequence;
[0087] Inputting the feature word tokens sequence into a pre-built single pointer network model to obtain a third feature matrix;
[0088] The third feature matrix is modeled to obtain a second task model.
[0089] Specifically, the tokens sequence output by BERT is input into the single pointer network model to obtain the feature matrix, and the feature matrix is modeled to obtain the second task model, which is used to extract warning feature words. The specific process is as follows: Figure 4 As shown,
[0090] Assume that the length of the input single text is n , the text is encoded by the pre-trained BERT model to obtain a token vector sequence [ h 1, h 2, h 3... h n ], through the matrix transformation of the dense layer, that is, the fully connected layer, it is transformed into two matrices of the same shape q , k ;as follows:
[0091]
[0092]
[0093] After matrix transformation, we can get q , k The vector sequence is as follows:
[0094] [ q 1,s ,q 2,s ,..., q n,s ] ,[ k 1,s , k 2,s ,... , k n,s ]
[0095] This vector sequence is used to identify the s The specific identification formula used for this type of event is defined as follows:
[0096]
[0097] The principle is: score the continuous subsequence span from i to j as the argument of event type s. q i,s and k j,s The inner product of the matrix is used as the event argument to score the continuous subsequence span[i:j].
[0098] q , k Vector calculations are similar to the following:
[0099] Mark the starting and ending positions of the argument to find the span of the argument cloud, and then use qk The matrix scores the arguments. For example, the sentence can be, someone said that comprehensive xx made money. The third feature matrix obtained is shown in Table 1.
[0100] Table 1 The third characteristic matrix
[0101]
[0102] In some embodiments, the binary labels are used as a training data set to train the combined first task model and the second task model as a whole to obtain a multi-task joint warning model, including:
[0103] Through training, a first loss function of the first task model and a second loss function of the second task model are obtained respectively;
[0104] Perform weighted calculation on the first loss function and the second loss function with specific weights to obtain a total loss function;
[0105] A multi-task joint early warning model is determined according to the total loss function.
[0106] Preferably, the first loss function and the second loss function are weighted by specific weights to obtain a total loss function, including:
[0107] Loss = αloss2 + (1-α)loss1
[0108] Among them, Loss is the total loss function, α is the specific weight, loss1 is the first loss function, and loss2 is the second loss function.
[0109] Specifically, the multi-task joint warning model obtained in this application is as follows Figure 5 As shown, the hidden of the last layer in the first task model is output through a fully connected layer:
[0110]
[0111] in, w t1 is the training parameter of the fully connected layer, b t1 is the bias of the neural network in this layer;
[0112] The classification results probabilities are output by the softmax function:
[0113]
[0114] Loss1 is calculated by cross entropy, and the error is back-propagated to update the parameters:
[0115]
[0116] in: is the true label of the category, It is the predicted category probability distribution calculated by the above softmax function.
[0117] The score in the second task model is output through a fully connected layer:
[0118]
[0119] in, score i is the score of the second task model, w t2 is the training parameter of the fully connected layer. b t2 is the bias of the neural network in this layer.
[0120] The sigmiod function outputs the classification result probabilities respectively:
[0121]
[0122] Loss2 is calculated by cross entropy, and the error is back-propagated to update the parameters:
[0123]
[0124] in: is the true label of the category, is the predicted category probability distribution calculated by the above sigmiod function. The total loss of the multi-task joint warning model is calculated as follows:
[0125] Loss = αloss2 + (1-α)loss1
[0126] In this application, the first task model and the second task model are linked for learning and training, and the final output result is a binary (label1, result2), where label1 is the result of the task, which is one of [neutral, positive, negative]; result2 is the extraction result of task 2, which is a continuous sequence of short characters in the original text. If the result binary is (negative, result2) and result2 is not empty, it is considered that the text is negative sensitive data, and a sensitive feature word sequence is extracted, and an early warning is required.
[0127] Finally, the unlabeled data is input into the multi-task joint warning model. The prediction output binary tuples have the following possibilities: (positive, "none"), (positive, "warning feature words"), (negative, "none"), (negative, "warning feature words"), (neutral, "none"), (neutral, "warning feature words"); the binary tuple (negative, "warning feature words") is sensitive data for a specific industry and requires warning.
[0128] Through the technical solution provided by this application, this application has the following beneficial effects:
[0129] (1) This application uses Google's open source BERT pre-training model to perform word-level embedding vectorization on the input text data, solve the problem of contextual semantic ambiguity, and better and more accurately understand the contextual semantics.
[0130] (2) Through a large number of corpora, warning feature vocabulary in specific fields is constructed and automatically collected. Through a single pointer network, supervised model training is performed on the constructed learning corpus to extract warning feature vocabulary.
[0131] (3) The single pointer network switches the input text into all possible continuous subsequences, with a subsequence length of n. The continuous subsequence is called a span, and the span may be the required warning feature word sequence. The method provided by this application can solve the problem of feature word nesting, as well as the context and global uniqueness problems. Each span is unique and is calculated through its context semantics;
[0132] (4) This application obtains the loss loss1 of sentiment polarity training through the first task model, and the loss loss2 of single pointer warning feature vocabulary extraction through the second task model. The two loss values are weighted with a specific weight α to obtain the total model loss, and the error back propagation is performed to update the learning parameters. Multi-task joint learning enables the model to have the ability of semantic reasoning and feature extraction. The two are perfectly integrated and continuously adjusted through α to achieve the convergence progress of the two tasks and the problem of category imbalance.
[0133] like Figure 6 As shown, the embodiment of the present application provides an industry early warning device based on multi-task joint learning, including:
[0134] An acquisition module 601 is used to acquire training corpus, perform sentiment classification on the training corpus, and obtain label corpus;
[0135] A determination module 602 is used to determine an industry text corpus, and select negative feature words corresponding to the industry from the industry text corpus as warning feature words;
[0136] A combining module 603 is used to combine the label corpus and the warning feature word to obtain a bigram label;
[0137] A construction module 604 is used to construct a first task model based on the label corpus using the pre-trained model BERT for sentiment polarity classification; and to construct a second task model based on the warning feature words using the pre-trained model BERT for extracting the warning feature words;
[0138] A training module 605 is used to use the binary label as a training data set to perform overall training on the combined first task model and the second task model to obtain a multi-task joint warning model;
[0139] The prediction module 606 is used to input the text to be predicted into the multi-task joint warning model to obtain a warning result.
[0140] The working principle of the industry early warning device based on multi-task joint learning provided in the embodiment of the present application is as follows: the acquisition module 601 acquires the training corpus, performs sentiment classification on the training corpus, and obtains the label corpus; the determination module 602 determines the industry text corpus, and selects the corresponding negative feature words of the industry from the industry text corpus as the warning feature words; the combination module 603 is used to combine the label corpus and the warning feature words to obtain a bigram label; the construction module 604 is used to construct a first task model based on the label corpus using the pre-trained model BERT for sentiment polarity classification; the second task model is constructed based on the warning feature words using the pre-trained model BERT for extracting warning feature words; the training module 605 is used to use the bigram label as the training data set to perform overall training on the combined first task model and second task model to obtain a multi-task joint warning model; the prediction module 606 is used to input the text to be predicted into the multi-task joint warning model to obtain a warning result.
[0141] The application embodiment provides a computer device, including a processor, and a memory connected to the processor;
[0142] The memory is used to store a computer program, and the computer program is used to execute the industry early warning method based on multi-task joint learning provided in any of the above embodiments;
[0143] The processor is used to call and execute the computer program in the memory.
[0144] It can be understood that the method embodiment provided above corresponds to the device embodiment described above, and the corresponding specific contents can be referenced to each other and will not be repeated here.
[0145] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.
[0146] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0147] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction method, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0149] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. An industry early warning method based on multi-task joint learning, characterized in that: include: Acquire training corpus, perform sentiment classification on the training corpus, and obtain label corpus; Determine an industry text corpus, and select negative feature words corresponding to the industry from the industry text corpus as warning feature words; Combining the label corpus and the warning feature words to obtain a bigram label; The first task model is constructed based on the labeled corpus using the pre-trained model BERT for sentiment polarity classification. Based on the warning feature words, the pre-trained model BERT is used to construct a second task model for extracting the warning feature words; wherein, based on the label corpus, the pre-trained model BERT is used to construct the first task model, including: inputting the label corpus into the pre-trained model BERT, outputting the classification task cls and the sentiment polarity tokens sequence; inputting the classification task cls and the sentiment polarity tokens sequence into the pre-constructed textcnn model, and outputting the first feature matrix; combining the classification task cls with the first feature matrix to obtain the second feature matrix; modeling the second feature matrix to obtain the first task model; The binary labels are used as a training data set to train the combined first task model and the second task model as a whole to obtain a multi-task joint warning model; The text to be predicted is input into the multi-task joint warning model to obtain a warning result.
2. The method according to claim 1, characterized in that The step of selecting negative characteristic words corresponding to the industry from the industry text corpus as warning characteristic words includes: Collect negative words about the preset industry; Collect negative Internet data of preset industries; The negative words are added to the custom dictionary through the N-Gram model, and the negative Internet data is segmented, and the negative words containing three consecutive negative words are saved, and the word frequency is counted; The negative words whose word frequency is lower than a preset threshold are filtered out, and the remaining negative words are used as warning feature words.
3. The method according to claim 2, characterized in that The step of combining the label corpus and the warning feature words to obtain a bigram label includes: The warning feature words and the label corpus are matched to obtain a bigram label.
4. The method according to claim 1, characterized in that: The second task model is constructed based on the warning feature words using the pre-trained model BERT, including: Input the warning feature words into the pre-trained model BERT and output the feature word tokens sequence; Inputting the feature word tokens sequence into a pre-built single pointer network model to obtain a third feature matrix; The third feature matrix is modeled to obtain a second task model.
5. The method according to claim 1, characterized in that The binary labels are used as training data sets to train the combined first task model and second task model as a whole to obtain a multi-task joint warning model, including: Through training, a first loss function of the first task model and a second loss function of the second task model are obtained respectively; Perform weighted calculation on the first loss function and the second loss function with specific weights to obtain a total loss function; A multi-task joint early warning model is determined according to the total loss function.
6. The method according to claim 5, characterized in that The first loss function and the second loss function are weighted by specific weights to obtain a total loss function, including: Loss = αloss2 + (1-α)loss1 Among them, Loss is the total loss function, α is the specific weight, loss1 is the first loss function, and loss2 is the second loss function.
7. The method according to claim 1, characterized in that The label corpus includes positive, neutral and negative.
8. An industry early warning device based on multi-task joint learning, characterized in that: include: An acquisition module is used to acquire training corpus, perform sentiment classification on the training corpus, and obtain label corpus; A determination module, used for determining industry text corpus, and selecting negative characteristic words corresponding to the industry from the industry text corpus as warning characteristic words; A combining module, used for combining the label corpus and the warning feature words to obtain a bigram label; A construction module is used to construct a first task model based on the label corpus using the pre-trained model BERT for sentiment polarity classification; a second task model is constructed based on the warning feature words using the pre-trained model BERT for extracting warning feature words; specifically, the label corpus is input into the pre-trained model BERT, and a classification task cls and a sentiment polarity tokens sequence are output; the classification task cls and the sentiment polarity tokens sequence are input into the pre-built textcnn model, and a first feature matrix is output; Combining the classification task cls with the first feature matrix to obtain a second feature matrix; Modeling the second feature matrix to obtain a first task model; A training module, used to use the binary labels as a training data set to perform overall training on the combined first task model and second task model to obtain a multi-task joint warning model; The prediction module is used to input the text to be predicted into the multi-task joint warning model to obtain a warning result.
