A big data information processing method, system, medium and server based on deep learning

Abstract

The present application provides a method, system, medium and server for processing big data information based on deep learning, which relates to the technical field of big data information processing, wherein the method includes steps such as data collection, model building, model training and classification; the system includes a data collection module, a model building module, a model training module and a classification module. The present application reduces the influence of domain shift on sentiment classification by setting the structure of the classification model, and improves the accuracy of cross-domain classification.

Description

Technical Field

[0001] The present application relates to the technical field of big data information processing, and in particular, to a big data information processing method, system, medium and server based on deep learning. Background Art

[0002] With the continuous growth of public information on the Internet, there are a large number of texts with subjective emotions on social networks, news resources, business and government system platforms. By using natural language processing technology, we can extract, analyze and process the opinions of unstructured text information, and finally obtain structured data and present it to users intuitively.

[0003] Cross-domain sentiment classification is an important task in natural language processing, which aims to use one or more data sources with labeled sentiment to train a model to accurately classify data from different domains without labeled sentiment. Due to the differences in language style, topic, and sentiment expression between different domains, cross-domain sentiment classification faces huge challenges.

[0004] The Chinese invention patent with the application publication date of July 19, 2019 and the application publication number of CN110032646A provides a cross-domain text sentiment classification method based on multi-source domain adaptation joint learning. The loss function of the joint training designed by the invention includes four parts: sentiment classification loss, parameter transfer loss, domain fusion loss and regularization term to prevent overfitting. The sentiment classification loss includes the sentiment classification loss on the source domain task and the target domain task. The soft parameter transfer method can effectively transfer the sentiment knowledge of the source domain to the target domain, and the deep domain fusion can ensure that the marginal distribution of different domains is as similar as possible during the learning process. Although the patent realizes sentiment transfer by reducing the domain difference between the source domain and the target domain, it does not consider the impact of the domain shift caused by a single classifier on the classification result in the cross-domain sentiment classification problem, that is, if a single classifier is trained too fully in the source domain, it may learn specific features or patterns of the source domain, which may not exist or behave differently in the target domain, which will cause the performance of the classifier in the target domain to decline, that is, the domain shift problem occurs. Summary of the invention

[0005] In order to reduce the impact of domain shift on classification results, the present application provides a big data information processing method, system, medium and server based on deep learning.

[0006] In the first aspect, the present application provides a method for processing big data information based on deep learning, which adopts the following technical solutions:

[0007] A big data information processing method based on deep learning comprises the following steps:

[0008] Data collection: Collect source domain data and target domain data separately. The source domain data has category labels, while the target domain data has no category labels.

[0009] Establishing a model: establishing a classification model based on a deep learning algorithm, wherein the classification model includes: a BERT sub-model, an MLP sub-model, a first classifier, and a second classifier;

[0010] Model training: including first feature extraction, first training, second training and solving the mean;

[0011] First feature extraction: input the source domain data into the BERT sub-model to obtain the first data, use the first data as the input of the MLP sub-model, and output the source domain features;

[0012] First training: taking the source domain features and the corresponding category labels as the input of the first classifier to obtain the first output data;

[0013] Second training: Use the source domain features and the corresponding category labels as the input of the second classifier to obtain the second output data;

[0014] Solving the mean: Calculate the mean of the first output data and the second output data, record it as the second data, and use the second data as the output data of the classification model;

[0015] Classification: Input the target domain data into the trained classification model and output the category label of the target domain data.

[0016] By adopting the above technical solution, the present application sets the structure of the classification model, realizes feature extraction through the BERT sub-model and the MLP sub-model, and adopts two classifiers, the first classifier and the second classifier. The performance of the two classifiers is complementary, and they can complement each other during classification, and calculate the mean of the two at the final output, which helps to reduce the deviation of a single classifier due to overtraining, reduce the impact of domain shift on the classification results, and improve the generalization ability and robustness of the classification model. In the case where the target domain data is unlabeled, the classification model classifies the target domain data through the knowledge and features learned from the source domain, which can overcome the offset between domains to a certain extent and realize the classification of the target domain data. The present application learns the features of the source domain data through the BERT sub-model and learns complex nonlinear changes through the MLP sub-model. The BERT sub-model provides rich contextual representations, while the MLP sub-model is good at extracting and combining features from these representations. The combination of the two enables the classification model to utilize contextual information while also having excellent feature extraction capabilities, thereby better coping with the problem of domain shift.

[0017] Optionally, after executing the step of solving the mean and before executing the step of classification, the method further includes:

[0018] Add labels: select part of the target domain data, add category labels to each target domain data, and record the category labels as the third data;

[0019] Retraining: including second feature extraction, third training and fourth training;

[0020] Second feature extraction: input the target domain data into the BERT sub-model of the trained classification model to obtain fourth data, and input the fourth data into the MLP sub-model of the trained classification model to obtain target domain features;

[0021] Third training: using the target domain features and the third data as inputs of the first classifier of the trained classification model to obtain third output data;

[0022] Fourth training: using the target domain features and the third data as inputs of a second classifier of the trained classification model to obtain fourth output data.

[0023] By adopting the above technical solution, the present application selects part of the target domain data and adds category labels to it, obtains the third data (i.e., the category labels of the target domain data), and uses the target domain data and the third data to retrain the trained classification model, so that the classification model can further adapt to the data distribution of the target domain and improve the classification performance in the target domain. In the step of adding labels, the present application tries to keep the data of each category balanced and improve the generalization ability of the classification model. In the retraining step, since the categories of the first classifier and the second classifier are different, the diversity of the classification model can be increased. By comparing the distribution differences of the third output data and the fourth output data, the performance of different classifiers can be evaluated. Similar to the model training step, the present application can further reduce the probability of the occurrence of domain deviation problems that may be caused by a single classifier by solving the mean of the third output data and the fourth output data, and further improve the generalization ability and robustness of the classification model.

[0024] Optionally, after performing the fourth training step and before performing the classification step, the method further includes:

[0025] Calculate classification differences: Calculate the classification differences between the third output data and the fourth output data. The calculation model is as follows:

[0026] ;

[0027] ;

[0028] ;

[0029] in, is a classification difference between the third output data and the fourth output data; is the probability distribution of the third output data; is the probability value of the third output data of the i-th category; is the probability distribution of the fourth output data; is the probability value of the fourth output data of the i-th category; D is the probability distribution of the third output data And the probability distribution of the fourth output data The average distribution of is the probability value of the average distribution of the i-th category, and the calculation model is as follows:

[0030] .

[0031] By adopting the above technical solution, the present application calculates the classification difference, and evaluates the performance difference between the two classifiers (the first classifier and the second classifier) ​​through the classification difference. If the difference is very small, it means that the two classifiers are similar in performance (that is, the classification results output by the two classifiers are similar), indicating that the classification model training effect is excellent and the application effect on the classification problem of the target domain data is obvious. If the difference is large, it means that they capture different features or patterns in the target domain data and need further analysis.

[0032] Optionally, after executing the step of calculating the classification difference and before executing the step of classification, the method further includes:

[0033] Minimize the difference between categories: As the adversarial loss, the parameters of the BERT sub-model and the MLP sub-model are updated alternately until the minimum adversarial loss is obtained.

[0034] By adopting the above technical solution, the present application minimizes the classification differences and alternately updates the parameters of the BERT sub-model and the MLP sub-model until the minimum adversarial loss is obtained. In this process, the classification model can learn a more consistent and stable feature representation, thereby improving the robustness to noise, adversarial samples, etc., and helps the classification model learn more generalized features, rather than just remembering specific patterns in the training data, reducing the risk of overfitting and further reducing the impact of domain shift. Minimizing classification differences can also improve the classification performance of the classification model because it encourages the classification model to reach a consensus between multiple classifiers, thereby obtaining more accurate classification results.

[0035] Optionally, in the third training and fourth training steps, a Dropout algorithm is used to train the first classifier and the second classifier.

[0036] By adopting the above technical solution, the present application adopts the Dropout algorithm to realize the dependence of the first classifier and the second classifier on the target domain data, reducing the risk of overfitting, making it more likely that the classification model will learn the generalized features in the target domain data. Since the Dropout algorithm randomly discards neurons during the training process, it forces the two classifiers to learn multiple different feature combinations to make predictions, increasing the diversity of the classification model and enabling it to better handle unseen data. The classifier trained by the Dropout algorithm is more robust to slight changes in the input data because the classifier has learned to make predictions in the absence of some information during training. Although the Dropout algorithm itself does not directly reduce training time, it indirectly optimizes training time by reducing overfitting, allowing the classification model to achieve better performance in fewer training cycles.

[0037] Optionally, after performing the step of adding labels and before performing the step of retraining, the method further includes:

[0038] Label evaluation: including obtaining category center, calculating distance, threshold filtering and deletion;

[0039] Get the category center: Get the category center of all samples contained in the kth category, recorded as the fifth data, and the calculation model is as follows:

[0040] ;

[0041] in, is the category center of all samples contained in the kth category; is the number of all samples in the kth class; is the i-th sample of the k-th class;

[0042] Calculate distance: Calculate the distance between the third and fifth data of the i-th , the calculation model is as follows:

[0043] ;

[0044] in, is the dimension of the feature space of the kth class; is the j-th eigenvalue of the i-th third data;

[0045] Threshold filtering: judging distance Is it greater than a preset threshold? If so, execute the step of deleting; if not, execute the step of calculating the distance based on the i+1th third data;

[0046] Delete: Delete the current category label. In the selected target domain data, delete the target domain data corresponding to the current category label.

[0047] By adopting the above technical solution, the present application calculates the center of each category, which helps the classification model understand the typical characteristics of each category. By calculating the category center, the representative characteristics of each category can be obtained. The category center provides an overview of the data distribution within the category, which helps to identify outliers or data points with incorrect labels. The present application evaluates the distance between the newly added target domain data and the category center to which it is assigned to judge the rationality of the label. If the distance between the target domain data and its category center is too large, it means that the data point does not really belong to the category, or its label is incorrectly assigned. By calculating the distance, the rationality of the label can be quantified and provide a basis for subsequent threshold filtering. By setting a reasonable threshold, data points that are too far from the category center and belong to the wrong category label can be effectively identified and deleted, which helps to reduce noise and errors in the data set and improve the training effect and generalization ability of the classification model. By deleting the target domain data with the wrong category label, the negative impact of these erroneous target domain data on the classification model training can be reduced, thereby improving the performance and stability of the classification model.

[0048] Optionally, after executing the deleting step and before executing the retraining step, the method further includes:

[0049] Data collection: Collect the deleted target domain data and organize them into a data set, which is recorded as the first data set;

[0050] Re-labeling: inputting the target domain data in the first data set into the retrained classification model, outputting the category label of the target domain data in the first data set, and recording the category label of the target domain data in the first data set as the sixth data;

[0051] Recalculate the distance: Calculate the distance H between the i-th sixth data and the fifth data. The calculation model is as follows:

[0052] ;

[0053] in, is the dimension of the feature space of the kth class; is the j-th eigenvalue of the i-th sixth data;

[0054] Determine the distance: determine whether the distance H is greater than a preset threshold. If so, execute the retraining step; if not, execute the distance recalculation step based on the (i+1)th sixth data.

[0055] By adopting the above technical solution, the present application collects the deleted target domain data together and organizes it into a data set, and re-labels the classification model after retraining, so that the wrong category label can be replaced with the correct category label; after re-adding the label, the distance between the category label of the target domain data and the corresponding category center is recalculated to evaluate the rationality of the new category label. If the distance between the category center corresponding to the new category label of the target domain data is close, then the new category label can be considered reasonable, which further verifies that the retrained classification model can be well applied to the classification of the target domain data. Otherwise, the classification model needs to re-execute the retraining step so that it can adapt to the data distribution of the target domain data.

[0056] In the second aspect, the present application provides a big data information processing system based on deep learning, which adopts the following technical solutions:

[0057] A big data information processing system based on deep learning, comprising:

[0058] The data collection module is used to collect source domain data and target domain data respectively. The source domain data has category labels, while the target domain data has no category labels.

[0059] A model building module is used to build a classification model, wherein the classification model includes: a BERT sub-model, an MLP sub-model, a first classifier, and a second classifier;

[0060] The model training module is connected to the data acquisition module and the model building module in communication, and includes a first feature extraction unit, a first training unit, a second training unit and a mean value solving unit;

[0061] A first feature extraction unit is used to input the source domain data into the BERT sub-model, obtain the output of the BERT sub-model, use the output of the BERT sub-model as the input of the MLP sub-model, and output the source domain features;

[0062] A first training unit is used to use the source domain features and the corresponding category labels as inputs of a first classifier to obtain output data of the first classifier;

[0063] A second training unit is used to use the source domain features and the corresponding category labels as inputs of a second classifier to obtain output data of the second classifier;

[0064] A mean solving unit, used for calculating the mean of the output data of the first classifier and the output data of the second classifier, and taking the mean as the output data of the classification model;

[0065] The classification module is connected to the model training module and the data acquisition module for inputting the target domain data into the trained classification model and outputting the category label of the target domain data.

[0066] By adopting the above technical solutions, this system integrates the various links of data collection, model building, model training and classification to form an efficient transfer learning framework. The data collection module collects source domain data with category labels and unlabeled target domain data respectively, providing a solid foundation for subsequent model training. In the model building stage, the powerful feature extraction capability of the BERT sub-model and the nonlinear transformation characteristics of the MLP sub-model are combined with the design of dual classifiers to enhance the robustness and diversity of the classification model. The model training module effectively integrates the advantages of different classification strategies by training two classifiers in parallel and calculating their output mean, thereby improving the accuracy and stability of classification. Finally, the classification module applies the trained model to the target domain data, achieves high-precision category label prediction, and provides strong support for practical applications.

[0067] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used to implement the described method when processed by a processor.

[0068] In a fourth aspect, the present application provides a server, on which the system is loaded, and the server includes a processor and a memory in communication with the processor;

[0069] A computer-readable storage medium is provided in the memory, and a computer program is stored in the computer-readable storage medium;

[0070] When the processor processes the computer program stored on the computer-readable storage medium, the method described above is implemented.

[0071] In summary, the present application includes at least one of the following beneficial technical effects:

[0072] 1. This application sets the structure of the classification model, realizes feature extraction through the BERT sub-model and the MLP sub-model, and adopts two classifiers, the first classifier and the second classifier. The performance of the two classifiers is complementary, and they can complement each other during classification, and calculate the mean of the two at the final output, which helps to reduce the deviation of a single classifier due to over-training, reduce the impact of domain shift on the classification results, and improve the generalization ability and robustness of the classification model. In the case that the target domain data is not labeled, the knowledge and features learned from the source domain are transferred to overcome the offset between domains to a certain extent, and the classification of the target domain data can be realized. This application learns the features of the source domain data through the BERT sub-model and learns complex nonlinear changes through the MLP sub-model. The BERT sub-model provides rich contextual representations, and the MLP sub-model is good at extracting and combining features from these representations. The combination of the two enables the classification model to utilize contextual information while also having excellent feature extraction capabilities, thereby better coping with the problem of domain shift.

[0073] 2. This application selects part of the target domain data and adds category labels to it, obtains the third data (i.e., the category labels of the target domain data), and uses the target domain data and the third data to retrain the trained classification model, so that the classification model can further adapt to the data distribution of the target domain and improve the classification performance in the target domain. In the step of adding labels, this application tries to keep the data of each category balanced and improve the generalization ability of the classification model. In the retraining step, since the categories of the first classifier and the second classifier are different, the diversity of the classification model can be increased. By comparing the distribution differences of the third output data and the fourth output data, the performance of different classifiers can be evaluated. Similar to the model training step, this application can further reduce the probability of the occurrence of domain deviation problems caused by a single classifier by solving the mean of the third output data and the fourth output data, and further improve the generalization ability and robustness of the classification model.

[0074] 3. This application collects the deleted target domain data together and organizes them into a data set, and re-labels them through the retrained classification model, so that the wrong category labels can be replaced with the correct category labels; after re-labeling, the distance between the category label of the target domain data and the corresponding category center is recalculated to evaluate the rationality of the new category label. If the distance between the category centers corresponding to the new category labels of the target domain data is close, then the new category label can be considered reasonable, which further verifies that the retrained classification model can be well applied to the classification of the target domain data. Otherwise, the classification model needs to re-execute the retraining step so that it can adapt to the data distribution of the target domain data. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 is a flow chart of Example 1 of the present application;

[0076] Figure 2 is a flow chart of Example 2 of the present application;

[0077] Figure 3 This is a flow chart of Example 3 of the present application. DETAILED DESCRIPTION

[0078] The following combination Figures 1 to 3 This application is described in further detail.

[0079] Embodiment 1: This embodiment discloses a method for processing big data information based on deep learning. Figure 1 The method includes: S1 data collection, S2 model building, S3 model training and S4 classification steps. First, source domain data for model training and target domain data to be classified are collected, then a classification model is built, then the built classification model is trained, and then the trained classification model is used to classify the target domain data to be classified to obtain the category label of the target domain data. The specific steps of this embodiment are as follows:

[0080] S1 Data collection, collect source domain data and target domain data respectively. Source domain data refers to data in a certain field that already has correct category labels. These source domain data will be used in the training and verification stages of the classification model to ensure that the classification model can learn the mapping relationship between the features of the source domain data and the category labels. Target domain data refers to data in another field without category labels. These target domain data will be used for actual classification tasks after the classification model training is completed, and will evaluate the generalization ability of the classification model on unseen data.

[0081] S2 builds a model, and builds a classification model based on a deep learning algorithm. The classification model includes: a BERT sub-model, an MLP sub-model, a first classifier, and a second classifier. The BERT sub-model is responsible for deeply understanding the text content of the data and extracting high-level semantic features of the data; the MLP sub-model uses its nonlinear transformation characteristics to further process these high-level semantic features to extract low-dimensional features that are more suitable for classification tasks. The first classifier and the second classifier use different types of classifiers or have complementary performance. The two can classify data through different strategies, reduce the impact of domain shift on classification results, and improve the robustness and accuracy of the classification model.

[0082] S3 model training includes S31 first feature extraction, S32 first training, S33 second training and S34 solving the mean.

[0083] S31 first feature extraction, input the source domain data into the BERT sub-model, use the text understanding ability of the BERT sub-model to extract high-level semantic features in the source domain data, that is, the first data, and use the first data as the input of the MLP sub-model, use the nonlinear transformation characteristics of the MLP sub-model to further process the first data and output the source domain features.

[0084] S32 is the first training, using the extracted source domain features and the corresponding category labels (i.e., the category labels of the source domain data corresponding to the source domain features) to train a first classifier, wherein the first classifier focuses on capturing the feature pattern of a certain dimension in the source domain data and outputs first output data.

[0085] S33 second training, the same source domain features and corresponding category labels (i.e., the category labels of the source domain data corresponding to the source domain features) are also used to train the second classifier. The second classifier is originally designed to capture feature dimensions that are different or complementary to the first classifier and output second output data.

[0086] S34 solves the mean, calculates the mean of the first output data and the second output data, records it as the second data, and uses the second data as the output data of the classification model.

[0087] S4 classification, inputs the target domain data into the trained classification model, and the classification model outputs the category label of the target domain data based on the learned feature extraction and classification strategies.

[0088] This embodiment also evaluates the classification effect of the classification model on the target domain data by comparing the category labels of the target domain data output in the S4 classification step with the actual labels (if any) or the external validation data set. The evaluation indicators include accuracy, recall, F1 score, etc., to comprehensively measure the generalization ability and practical application value of the classification model. In addition, the classification model can be further adjusted and optimized based on the evaluation results. For example, when the evaluation results do not meet expectations, the S3 model training is re-executed to optimize the classification model parameters, etc.

[0089] In this embodiment, the classification model is trained with source domain data with correct classification labels, so that it can correctly classify target domain data without classification labels. The classification model of this embodiment can simultaneously utilize context information and extract features by using the BERT sub-model and the MLP sub-model, thereby better coping with the problem of domain shift, and in this embodiment, the first classifier and the second classifier use classifiers of different types or with complementary performance, further reducing the impact of domain shift on the classification results.

[0090] Embodiment 2: The difference from Embodiment 1 is as follows:

[0091] like Figure 2 As shown, after executing S34 to solve the mean and before executing S4 classification, it also includes:

[0092] S5 adds labels: select part of the target domain data, add category labels to each target domain data, and record the category labels as the third data.

[0093] The third data is the category label of the target domain data, and the category label has the same name as the category label of the source domain data collected by S1 data.

[0094] Adding category labels to the target domain data in the first data set refers to manually assigning a clear category label to the target domain data based on its attributes, characteristics or the meaning it represents. The purpose of manually assigning labels is to provide correct category labels as much as possible for S6 retraining.

[0095] S6 retraining: includes S61 second feature extraction, S62 third training and S63 fourth training.

[0096] S61 second feature extraction: input the third data in the label added in S5 into the BERT sub-model to obtain fourth data, and input the fourth data into the MLP sub-model to obtain the target domain feature. In this step, the BERT sub-model and the MLP sub-model are both the BERT sub-model and the MLP sub-model in the classification model trained in the step of S3 model training.

[0097] S62 Third training: The target domain features and the third data are used as the input of the first classifier to obtain the third output data. During the training process, the Dropout algorithm is used. The Dropout algorithm is an effective regularization technique that reduces the classification model's dependence on the output of specific neurons by randomly discarding some neurons (i.e., temporarily ignoring the output of some neurons in each iteration), thereby avoiding the occurrence of overfitting.

[0098] In this step, the first classifier is the first classifier in the classification model trained in the S3 model training step.

[0099] S63 Fourth training: using the target domain feature as the input of the second classifier to obtain fourth output data. The second classifier is of a different type from the first classifier or has complementary performance, and can capture classification information or information dimensions different from the first classifier, thereby providing a more comprehensive and accurate classification result. The Dropout algorithm is also used in training the second classifier to further improve its robustness and generalization ability.

[0100] In this step, the second classifier is the second classifier in the classification model trained in the S3 model training step.

[0101] S7 calculates classification differences: calculates the distribution differences between the third output data and the fourth output data, and the calculation model is as follows:

[0102] ;

[0103] ;

[0104] ;

[0105] in, is the classification difference between the third output data and the fourth output data; the smaller the classification difference is, the more similar the third output data and the fourth output data are; is the probability distribution of the third output data, that is, the probability that the target domain data output by the first classifier belongs to each category; is the probability value of the third output data of the i-th category; is the probability distribution of the fourth output data, that is, the probability that the target domain data output by the second classifier belongs to each category; is the probability value of the fourth output data of the i-th category; D is the probability distribution of the third output data And the probability distribution of the fourth output data The average distribution of is the probability value of the average distribution of the i-th category, and the calculation model is as follows:

[0106] .

[0107] The probability distribution of the third output data The output layer of the first classifier is: There are K neurons corresponding to K categories. For the input target domain data, the first classifier will output a vector of length K. Each element of the vector is the probability that the target domain data belongs to the corresponding category. The vector is the probability distribution B of the third output data.

[0108] And the probability distribution of the fourth output data The output layer of the second classifier is: neurons, corresponding to K categories. For the input target domain data, the second classifier will output a vector of length K. Each element of the vector is the probability that the target domain data belongs to the corresponding category. Then the vector is the probability distribution of the fourth output data. .

[0109] S8 minimizes classification differences: minimizes classification differences As the adversarial loss, the parameters of the BERT sub-model and the MLP sub-model are updated alternately until the minimum adversarial loss is obtained. The criterion for determining the minimum adversarial loss is: after multiple iterations, the classification difference No longer decrease; or preset a minimum threshold of adversarial loss, when the classification difference When it is less than the preset minimum adversarial loss threshold, it is deemed that the minimum adversarial loss has been obtained.

[0110] Update the parameters of the BERT sub-model and the MLP sub-model alternately, specifically:

[0111] Fix the MLP sub-model and update the BERT sub-model: First, fix the parameters of the MLP sub-model and focus on optimizing the parameters of the BERT sub-model. By adjusting the parameters of BERT, try to make the features extracted by the BERT sub-model more conducive to reducing classification differences, and use optimization algorithms such as gradient descent to update the parameters of the BERT sub-model with the goal of minimizing adversarial loss.

[0112] Fix the BERT sub-model and update the MLP sub-model: After the parameters of BERT are optimized to a certain extent, fix the parameters of the BERT sub-model and start optimizing the parameters of the MLP sub-model. By adjusting the parameters of the MLP sub-model, try to make the classification model produce more consistent and accurate classification results when the MLP sub-model receives the same feature input. Similarly, use the optimization algorithm to update the parameters of the MLP sub-model to further reduce the adversarial loss.

[0113] Alternating iterations: The above two steps are performed alternately to form an iterative optimization process. In each iteration, the parameters of the BERT sub-model and the MLP sub-model are adjusted according to the current adversarial loss value. As the iterations proceed, the classification difference gradually decreases, and the adversarial loss also decreases accordingly. When the adversarial loss is reduced to a certain level or reaches the preset number of iterations, it is considered that the classification model has been sufficiently optimized and the iteration can be stopped.

[0114] This embodiment adds category labels to part of the target domain data, and retrains the trained classification model with the target domain data with the category labels, so that the model is more suitable for the classification of the target domain data. When retraining the classification model, the output data of the two classifiers, i.e., the third output data and the fourth output data, are extracted respectively. The classification difference between the third output data and the fourth output data is calculated to obtain the difference in classification of the target domain data by the two classifiers. In addition, this embodiment also provides a step of minimizing the classification difference. In this step, the classification difference is used as adversarial loss, and the parameters of the BERT sub-model and the MLP sub-model are updated alternately until the minimum classification difference is obtained, thereby improving the classification performance of the classification model, and encouraging the classification model to reach a consensus between multiple classifiers, so as to obtain more accurate classification results.

[0115] Embodiment 3 is different from Embodiment 2 in that:

[0116] like Figure 3 As shown, after executing S5 to add labels and before executing S6 to retrain, it also includes:

[0117] S9 label evaluation, including S91 obtaining category center, S92 calculating distance, S93 threshold filtering and S94 deletion.

[0118] S91 obtains the category center: obtains the category center of all samples contained in the k-th category (k-th category label), recorded as the fifth data, and the calculation model is as follows:

[0119] ;

[0120] in, is the category center of all samples contained in the kth category, the category center Each dimension of represents the average value of the corresponding feature of the kth class on all samples; is the number of all samples in the kth class; is the i-th sample of the k-th class.

[0121] It means to sum the vectors of all samples of the kth class, and the result is an m-dimensional vector, each element of which is the sum of all sample values ​​on the corresponding feature.

[0122] The category center is the average position of all samples of the category in the feature space, which is crucial for the accuracy of subsequent evaluation labels because it provides a reference point for measuring the degree of deviation between a single sample and the center of the category to which it belongs.

[0123] S92 Calculate distance: Calculate the distance between the i-th third data and the fifth data , the calculation model is as follows:

[0124] ;

[0125] in, is the dimension of the feature space of the kth class; is the j-th eigenvalue of the i-th third data in the target domain.

[0126] This step quantifies the degree of match between the category label and the target domain data, that is, checks whether the third data truly reflects the category attributes of the target domain data.

[0127] S93 Threshold Filter: Determine Distance Is it greater than a preset threshold? If so, execute S94 to delete; if not, execute S92 to calculate the distance based on the next third data.

[0128] When the distance When it is not greater than the preset threshold, this step further determines whether all the third data have executed the step of calculating the distance in S92. If so, execute S10 data collection; if not, execute S92 distance calculation.

[0129] This step sets a preset threshold to determine whether the distance between the category label of the target domain data and its category center is too large, which may indicate the mis-addition of the category label or the abnormality of the target domain data itself.

[0130] S94 deletion: Delete the current category label of the target domain data, and delete the target domain data corresponding to the current category label. After S94 deletion, only those target domain data and category labels with accurate category labels and high data quality are retained for S6 retraining.

[0131] After completing S6 retraining and before executing S4 classification, it also includes:

[0132] S10 Data Collection: Collect the target domain data deleted in S94 and organize them into a data set, which is recorded as the first data set. These data are considered to have label errors or data quality problems because they are too far away from the category center of the category they belong to, or because other evaluation criteria are not met. After collecting these deleted target domain data, they are organized and cleaned, and the processed data set is named the first data set to provide necessary input for subsequent steps.

[0133] S11 re-adds labels: inputs the target domain data in the first data set into the retrained classification model, outputs the category labels of the target domain data in the first data set, and records the category labels of the target domain data in the first data set as sixth data.

[0134] By inputting the target domain data into the retrained classification model for prediction, the performance of the retrained classification model on unseen (or new) data can be evaluated.

[0135] S12 recalculates the distance: calculates the distance H between the i-th sixth data and the fifth data, and the calculation model is as follows:

[0136] ;

[0137] in, is the dimension of the feature space of the kth class; is the j-th eigenvalue of the i-th sixth data.

[0138] S13 determines the distance: determines whether the distance H is greater than a preset threshold, if so, executes S6 to retrain; if not, executes S12 to recalculate the distance based on the (i+1)th sixth data.

[0139] When the distance H is not greater than the preset threshold, this step also determines whether all the sixth data have executed the step of recalculating the distance in S12. If so, execute S4 classification; if not, execute S12 to recalculate the distance.

[0140] In this embodiment, the target domain data deleted in S94 is input into the retrained classification model, its category label is output, and the category label is evaluated, so as to indirectly evaluate the performance of the retrained classification model. If the output of the retrained classification model still does not meet the expectations (i.e., the distance H is greater than the preset threshold), S6 retraining is re-executed to continue to optimize the performance of the trained classification model.

[0141] Embodiment 4: This embodiment discloses a big data information processing system based on deep learning, the system comprising:

[0142] The data collection module is used to collect source domain data and target domain data respectively. Source domain data refers to data that has been accompanied by clear category labels. The clear category labels are correct category labels after judgment. These source domain data play a supervisory and guiding role in subsequent model training. Target domain data refers to data that lacks category labels and is the object that the system ultimately needs to classify and label. The source domain and target domain are different fields, but the category labels are category labels uniformly named for the source domain and the target domain, that is, the category labels of the source domain and the target domain are the same.

[0143] A model module is established to establish a classification model, which includes: a BERT sub-model, an MLP sub-model, a first classifier and a second classifier. The BERT sub-model is based on a pre-trained deep bidirectional model and is good at capturing rich contextual information of the data; the MLP sub-model is used to further extract and transform features; the two classifiers classify and predict the data from different angles to improve the generalization ability and classification accuracy of the model.

[0144] The model training module is connected to the data acquisition module and the model building module in communication, and includes a first feature extraction unit, a first training unit, a second training unit, and a mean solution unit. The input end of the first feature extraction unit is connected to the output end of the data acquisition module; the output end of the first feature extraction unit is connected to the input end of the first training unit, and the output end of the first feature extraction unit is connected to the input end of the second unit; the output end of the first training unit and the output end of the second unit are both connected to the input end of the solution unit.

[0145] The first feature extraction unit is used to send the source domain data to the BERT sub-model for preliminary processing to obtain the output of the BERT sub-model; the first feature extraction unit is also used to use the obtained output of the BERT sub-model as the input of the MLP sub-model, and finally output the feature representation of the source domain data (source domain feature).

[0146] The first training unit is used to use the source domain features and their category labels as inputs of the first classifier to obtain output data of the first classifier.

[0147] The second training unit is used to use the source domain features and their category labels as inputs of the second classifier to obtain output data of the second classifier.

[0148] The mean solving unit is used to calculate the mean of the output data of the first classifier and the output data of the second classifier. Taking the mean as the output data of the classification model helps to fuse the prediction results of the two classifiers, reduce the impact of domain shift on the classification results, and improve the stability and accuracy of the classification model.

[0149] The classification module is connected to the model training module and the data acquisition module for inputting the target domain data into the trained classification model and outputting the category label of the target domain data.

[0150] This embodiment collects source domain data with category labels for subsequent model training. In the model building stage, the powerful feature extraction capability of the BERT sub-model and the nonlinear transformation characteristics of the MLP sub-model are utilized, combined with the design of the dual classifier. The dual classifier uses classifiers of different types or complementary performances to enhance the robustness and diversity of the classification model and reduce the impact of domain shift on the classification results. The model training module effectively integrates the advantages of different classification strategies by training two classifiers in parallel and calculating their output means, thereby improving the accuracy and stability of classification. Finally, the classification module applies the trained model to the target domain data, achieves high-precision category label prediction, and provides strong support for practical applications.

[0151] Embodiment 5: This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is processed by a processor, it is used to implement the method described.

[0152] Embodiment 6: This embodiment provides a server, on which the system is loaded, and the server includes a processor and a memory in communication with the processor;

[0153] A computer-readable storage medium is provided in the memory, and a computer program is stored in the computer-readable storage medium;

[0154] When the processor processes the computer program stored on the computer-readable storage medium, the method described above is implemented.

[0155] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for processing big data information based on deep learning, comprising: data collection: collecting source domain data and target domain data respectively, the source domain data has a category label, and the target domain data has no category label; characterized in that, The method further comprises: Establishing a model: establishing a classification model based on a deep learning algorithm, wherein the classification model includes: a BERT sub-model, an MLP sub-model, a first classifier and a second classifier, wherein the first classifier and the second classifier have complementary performances; Model training: including first feature extraction, first training, second training and solving the mean; First feature extraction: Input the source domain data into the BERT sub-model, use the BERT sub-model to deeply understand the text content of the data, extract the high-level semantic features of the source domain data, obtain the first data, use the first data as the input of the MLP sub-model, and output the source domain features; First training: taking the source domain features and the corresponding category labels as the input of the first classifier to obtain the first output data; Second training: Use the source domain features and the corresponding category labels as the input of the second classifier to obtain the second output data; Solving the mean: Calculate the mean of the first output data and the second output data, record it as the second data, and use the second data as the output data of the classification model; Add labels: select part of the target domain data, add category labels to each target domain data in turn, and record the category labels as the third data; Retraining: including second feature extraction, third training and fourth training; Second feature extraction: input the target domain data into the BERT sub-model of the trained classification model to obtain fourth data, and input the fourth data into the MLP sub-model of the trained classification model to obtain target domain features; Third training: using the target domain features and the third data as inputs of the first classifier of the trained classification model to obtain third output data; Fourth training: using the target domain features and the third data as inputs of a second classifier of the trained classification model to obtain fourth output data; Calculate classification differences: Calculate the classification differences between the third output data and the fourth output data. The calculation model is as follows: ； ； ； in, is a classification difference between the third output data and the fourth output data; is the probability distribution of the third output data; is the probability value of the third output data of the i-th category; is the probability distribution of the fourth output data; is the probability value of the fourth output data of the i-th category; D is the probability distribution of the third output data and the average distribution of the probability distribution C of the fourth output data; is the probability value of the average distribution of the i-th category, and the calculation model is as follows: ； Minimize the difference between categories: As the adversarial loss, the parameters of the BERT sub-model and the MLP sub-model are updated alternately until the minimum adversarial loss is obtained; Classification: Input the target domain data into the trained classification model and output the category label of the target domain data.

2. The big data information processing method based on deep learning according to claim 1 is characterized in that: In the third training and fourth training steps, the Dropout algorithm is used to train the first classifier and the second classifier.

3. The big data information processing method based on deep learning according to claim 1 or 2, characterized in that: After the labeling step and before the retraining step, the following steps are also included: Label evaluation: including obtaining category center, calculating distance, threshold filtering and deletion; Get the category center: Get the category center of all samples contained in the kth category, recorded as the fifth data, and the calculation model is as follows: ； in, is the category center of all samples contained in the kth category; is the number of all samples in the kth class; is the i-th sample of the k-th class; Calculate distance: Calculate the distance between the third and fifth data of the i-th , the calculation model is as follows: ； in, is the dimension of the feature space of the kth class; is the j-th eigenvalue of the i-th third data; Threshold filtering: judging distance Is it greater than a preset threshold? If so, execute the step of deleting; if not, execute the step of calculating the distance based on the i+1th third data; Delete: Delete the current category label. In the selected target domain data, delete the target domain data corresponding to the current category label.

4. The big data information processing method based on deep learning according to claim 3 is characterized in that: After the retraining step and before the classification step, the following steps are also included: Data collection: Collect the deleted target domain data and organize them into a data set, which is recorded as the first data set; Re-labeling: inputting the target domain data in the first data set into the retrained classification model, outputting the category label of the target domain data in the first data set, and recording the category label of the target domain data in the first data set as the sixth data; Recalculate the distance: Calculate the distance H between the i-th sixth data and the fifth data. The calculation model is as follows: ； in, is the dimension of the feature space of the kth class; is the j-th eigenvalue of the i-th sixth data; Determine the distance: determine whether the distance H is greater than a preset threshold. If so, execute the retraining step; if not, execute the distance recalculation step based on the (i+1)th sixth data.

5. A big data information processing system based on deep learning, comprising: a data acquisition module for respectively acquiring source domain data and target domain data, wherein the source domain data has a category label and the target domain data has no category label; characterized in that: The system further comprises: A model building module is used to build a classification model, wherein the classification model includes: a BERT sub-model, an MLP sub-model, a first classifier and a second classifier, wherein the first classifier and the second classifier have complementary performances; The model training module is connected to the data acquisition module and the model building module in communication, and includes a first feature extraction unit, a first training unit, a second training unit and a mean value solving unit; The first feature extraction unit is used to input the source domain data into the BERT sub-model, use the BERT sub-model to deeply understand the text content of the data, extract high-level semantic features of the source domain data, obtain the output of the BERT sub-model, use the output of the BERT sub-model as the input of the MLP sub-model, and output the source domain features; A first training unit is used to use the source domain features and the corresponding category labels as inputs of a first classifier to obtain output data of the first classifier; A second training unit is used to use the source domain features and the corresponding category labels as inputs of a second classifier to obtain output data of the second classifier; A mean solving unit, used for calculating the mean of the output data of the first classifier and the output data of the second classifier, and taking the mean as the output data of the classification model; A label adding module is used to select part of the target domain data, add a category label to each target domain data in turn, and record the category label as the third data; A retraining module, comprising a second feature extraction unit, a third training unit and a fourth training unit; A second feature extraction unit is used to input the target domain data into the BERT sub-model of the trained classification model to obtain fourth data, and input the fourth data into the MLP sub-model of the trained classification model to obtain target domain features; A third training unit is used to use the target domain feature and the third data as inputs of a first classifier of the trained classification model to obtain third output data; a fourth training unit, configured to use the target domain feature and the third data as inputs of a second classifier of the trained classification model to obtain fourth output data; The classification difference calculation module is used to calculate the classification difference between the third output data and the fourth output data. The calculation model is as follows: ； ； ； in, is a classification difference between the third output data and the fourth output data; is the probability distribution of the third output data; is the probability value of the third output data of the i-th category; is the probability distribution of the fourth output data; is the probability value of the fourth output data of the i-th category; D is the probability distribution of the third output data and the average distribution of the probability distribution C of the fourth output data; is the probability value of the average distribution of the i-th category, and the calculation model is as follows: ； Minimize the classification difference module, which is used to minimize the classification difference As the adversarial loss, the parameters of the BERT sub-model and the MLP sub-model are updated alternately until the minimum adversarial loss is obtained; The classification module is connected to the model training module and the data acquisition module for inputting the target domain data into the trained classification model and outputting the category label of the target domain data.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is processed by a processor, it is used to implement the method according to any one of claims 1 to 4.

7. A server, on which the system according to claim 5 is loaded, characterized in that: The server includes: a processor, and a memory communicatively connected to the processor; The memory is provided with a computer-readable storage medium according to claim 6, and the computer-readable storage medium stores a computer program; When the processor processes the computer program stored on the computer-readable storage medium, the method according to any one of claims 1 to 4 is implemented.

Images