Industrial dynamic tag identification method combining big data and ROBERTa

Abstract

The invention discloses an industry dynamic tag identification method combining big data and ROBERTa, and belongs to the technical field of AIOT, and the method comprises the steps: collecting text data related to the industry from a plurality of big data sources through an automatic means, carrying out the cleaning and preprocessing of the collected data, preparing the data for subsequent processing, and carrying out the recognition of the industry dynamic tag. Through the big data processing technology and the RoBERTa pre-training language model, efficient and accurate label identification of industry data is realized, and powerful support is provided for industry analysis, market prediction and the like. According to the method, the universality of big data and the deep understanding ability of RoBERTa in text processing are combined, and the accuracy and efficiency of tag identification are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and specifically refers to an industry dynamic label recognition method combining big data and ROBERTa. Background Art

[0002] With the rapid development of big data technology, various industries have accumulated massive amounts of data. How to extract valuable information from this data has become the key to industry analysis, market forecasting and other work. As an advanced pre-trained language model, RoBERTa has performed well in the field of text processing. How to effectively combine it with big data technology and apply it to industry dynamic label recognition requires further research;

[0003] However, existing traditional label recognition methods often rely on manually defined rules or simple machine learning models, and have problems such as low recognition efficiency and poor accuracy. Manual labeling relies on the annotator's understanding and judgment of information, which is inevitably affected by the annotator's subjective factors. Different annotators or the same annotator from different observation angles may have different understandings and labeling results for the same video or text information. For large video or text databases, manual labeling requires a lot of manpower, material resources and time. With the continuous increase in the amount of data, the labeling efficiency of this method is particularly low, and it cannot meet the needs of quickly processing large amounts of data. For this reason, a method for industry dynamic label recognition combining big data and ROBERTa is proposed. Summary of the invention

[0004] The purpose of the present invention is to provide an industry dynamic tag recognition method combining big data and ROBERTa to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for identifying industry dynamic labels combining big data and ROBERTa, comprising the following steps:

[0006] S1. Collect industry-related text data from multiple big data sources through automated means, and perform cleaning and preprocessing operations on the collected data to prepare the data for subsequent processing;

[0007] S2. Use the RoBERTa pre-trained language model to deeply encode the pre-processed text data, capture the semantic features and contextual information of the text through the model's multi-layer Transformer structure, generate a deep semantic representation of the text, and combine the statistical features of big data to construct a multi-dimensional feature vector.

[0008] S3. Based on the semantic representation and statistical features output by the RoBERTa model, deep learning algorithms and pre-trained model RoBERTa are used to automatically identify and generate dynamic tags related to the industry. Dynamic tags can reflect key information, trend changes and hot topics in text data, are time-sensitive, and can be dynamically adjusted as data is updated.

[0009] S4. By introducing user feedback mechanism and automatic evaluation algorithm, the generated dynamic tags are continuously optimized and evaluated. User feedback is used to correct incorrect tags, supplement missing tags and adjust tag weights. The automatic evaluation algorithm can quantitatively evaluate the tag quality according to the preset evaluation criteria.

[0010] S5. Apply the optimized dynamic tags to multiple fields such as industry analysis, market forecasting, information retrieval, and content recommendation to provide users with valuable industry insights and decision support, while displaying the tag results through visualization technology.

[0011] Among them, the S1 collects various types of enterprise-related data from big data sources, cleans the collected data, removes noise, missing values ​​and irrelevant information, and performs text preprocessing on the cleaned data, including word segmentation, stop word removal, stem extraction, and part-of-speech tagging, in preparation for subsequent processing. The data collection and preprocessing steps also include a real-time data update mechanism to ensure that the label recognition system can continuously capture the latest industry trends.

[0012] Among them, in the step of feature extraction and encoding, the feature extraction uses a convolutional neural network to extract a feature map from the preprocessed image and set it as X. After the convolutional layer of the convolutional neural network, the dimension of the feature map is further reduced by a pooling layer and the main features are extracted. The convolution operation in the convolutional neural network is implemented by the formula:

[0013]

[0014] In the formula, X is the input feature map, w is the convolution kernel, b is the bias term, and Y is the output feature map. Feature encoding is to reduce the dimension and encode the extracted feature map through the fully connected layer to obtain a feature vector of fixed length. The feature vector is further optimized using principal component analysis and linear discriminant analysis dimensionality reduction techniques. Finally, the feature vector is converted into a compact encoding form using the hash function method. The implementation formula of the fully connected layer is:

[0015] z=WY ij + b,

[0016] In the formula, Y ijis the input feature vector, W is the weight matrix, b is the bias vector, z is the output feature vector, and the encoded output of the RoBERTa model is pooled as needed to generate a fixed-length sentence representation vector.

[0017] Among them, S3, according to the method described in claim 1, first selects a pre-trained model, converts the pre-processed text data into a Token ID sequence format, uses the RoBERTa model to extract features from the text data, and captures the semantic features and contextual information of the text through the multi-layer pre-trained model structure of the model. On the basis of RoBERTa, an attention mechanism is further introduced to strengthen the extraction of key information. The attention weight calculation implementation formula is:

[0018]

[0019] In the formula, Q, K, and V represent the matrix representation of query, key, and value, respectively. k is the dimension of the key, which is used to scale the dot product result to prevent the gradient from disappearing. The softmax function is used to normalize the attention score to a probability distribution, and then normalize it using the softmax function to get the attention weight of each element. The feature vector processed by the attention mechanism is passed to the output layer, and the probability of each label is calculated by the softmax classification function. The implementation formula of the softmax function to calculate the probability of each label is:

[0020]

[0021] In the formula, z i is the original output of the model for the input x belonging to the i-th label, p(y i |x) is the probability that the input x belongs to the i-th label. Finally, the model is trained using the labeled dataset and the model parameters are optimized through the back propagation algorithm.

[0022] Among them, the time sensitivity of the dynamic label is realized by considering the changing trend, frequency of occurrence and correlation with other labels of the label in the time series data. First, the labeled data set is divided into a training set, a validation set and a test set to ensure that each set contains samples to cover the diversity of the data. Then, the model parameters are initialized according to the classifier of the attention mechanism, and the model is trained using the training set data. In each training batch, the forward propagation input data passes through the model, and the loss between the predicted label and the true label is calculated. At the same time, the back propagation algorithm is used to calculate the gradient of the loss function with respect to the model parameters, and the model parameters are updated to minimize the loss. The implementation of the loss function is shown as follows:

[0023]

[0024] In the formula, N is the number of samples, C is the number of categories, and y is ic is the true label of whether the i-th sample belongs to the c-th class, p ic is the probability that the i-th sample belongs to the c-th class predicted by the model. The optimization algorithm is used to further adjust the model structure and training strategy according to the test results. The optimization algorithm implementation formula is:

[0025]

[0026] In the formula, θ t is the model parameter at the tth iteration, η is the learning rate, It is the gradient of the loss function with respect to the model parameters. Finally, the model is used to predict the new input data, generate dynamic labels, and deploy the trained model to the actual application scenario.

[0027] Among them, in said S4, in the label optimization and evaluation step, the user feedback mechanism allows the user to directly participate in the label optimization process by means of label correction, label addition, and label weight adjustment.

[0028] Among them, the S4, user feedback mechanism ensures that the model can be continuously optimized and adjusted according to the actual application effect. First, the user compares the collected data with the preset benchmark value and historical data through the system to evaluate the changing trend of the model performance. Through data analysis, the reasons why the model performance does not meet expectations are identified. According to the data analysis results, the data preprocessing process is adjusted, the training data set is updated, and the model architecture is replaced to optimize the model performance. The effect of the optimized model is evaluated to verify whether it has achieved the expected optimization goal.

[0029] Among them, the S4 needs to randomly divide the data set into two mutually exclusive subsets, one as a training set and the other as a test set, train the model on the training set and evaluate the model performance on the test set; and randomly extract samples of the same size as the original data set from the original data set with replacement to form a training set, and use the unselected samples as a test set, train the model on the training set, and evaluate the model performance on the test set, repeat multiple times, each sampling forms a different training set and test set, so as to obtain multiple model performance evaluation results, take the average of the model performance of multiple iterations as the final performance evaluation result, and finally process the evaluation results.

[0030] Among them, in the step of applying and displaying labels in S5, the visual display method supports multi-dimensional and multi-perspective data exploration, allowing users to customize the display content and layout according to their needs.

[0031] Among them, the S5 is applied to data classification and label recognition tasks in other fields. First, the data source, data format, data quality, classification goals and requirements of the new field are understood. According to the data characteristics of the new field, the original data is cleaned, format converted, and feature extracted. At the same time, according to the classification task requirements of the new field, the RoBERTa model is adjusted, the pre-trained corpus is replaced, the model parameters are adjusted, and new features are introduced. The adjusted model is trained using the data from the new field, and the performance of the model is evaluated through cross-validation. The trained model is deployed to the actual application scenario, and the data in the new field is classified and label recognized. Finally, a feedback mechanism is established to continuously optimize and adjust the model according to the actual application effect.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. This invention can automatically and accurately identify and annotate industry dynamic labels through big data and the RoBERTa model, greatly improving data processing efficiency. The RoBERTa model can better adapt to natural language processing tasks in different fields through larger training data sets and richer language representations, and has stronger adaptability and stability in different environments and conditions;

[0034] 2. In order to solve the problem that manual labeling depends on the annotator's understanding and judgment of information, which will be affected by the annotator's subjective factors, the present invention uses deep learning and natural language processing technology to achieve intelligent label recognition and labeling, reduce manual intervention, and reduce the influence of subjective factors;

[0035] 3. The present invention introduces a feedback mechanism and a dynamic update mechanism. Users can compare the collected data with the preset benchmark value and historical data through the system, adjust the data preprocessing process, update the training data set, and replace the model architecture according to the data analysis results to optimize the model, ensure the accuracy and timeliness of the label recognition algorithm, and meet the changes in market demand and technological development;

[0036] 4. The present invention has made a number of innovative optimizations and improvements on the basis of the BERT model through the RoBERTa model, including a dynamic masking strategy, a larger training data set and a larger number of model parameters. These innovations have significantly improved the performance of the RoBERTa model. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is the operating process of the industry dynamic tag recognition method combining big data and ROBERTa in the present invention Figure 1 ;

[0038] Figure 2 This is the operating process of the industry dynamic tag recognition method combining big data and ROBERTa in the present invention Figure 2 ;

[0039] Figure 3 This is the operating process of the industry dynamic tag recognition method combining big data and ROBERTa in the present invention Figure 3 . DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Example

[0042] See also Figure 1-Figure 3 As shown, the present invention provides a technical solution: comprising the following steps:

[0043] S1. Collect industry-related text data from multiple big data sources through automated means, and perform cleaning and preprocessing operations on the collected data to prepare the data for subsequent processing;

[0044] S2. Use the RoBERTa pre-trained language model to deeply encode the pre-processed text data, capture the semantic features and contextual information of the text through the model's multi-layer Transformer structure, generate a deep semantic representation of the text, and combine the statistical features of big data to construct a multi-dimensional feature vector.

[0045] S3. Based on the semantic representation and statistical features output by the RoBERTa model, deep learning algorithms and pre-trained model RoBERTa are used to automatically identify and generate dynamic tags related to the industry. Dynamic tags can reflect key information, trend changes and hot topics in text data, are time-sensitive, and can be dynamically adjusted as data is updated.

[0046] S4. By introducing user feedback mechanism and automatic evaluation algorithm, the generated dynamic tags are continuously optimized and evaluated. User feedback is used to correct incorrect tags, supplement missing tags and adjust tag weights. The automatic evaluation algorithm can quantitatively evaluate the tag quality according to the preset evaluation criteria.

[0047] S5. Apply the optimized dynamic tags to multiple fields such as industry analysis, market forecasting, information retrieval, and content recommendation to provide users with valuable industry insights and decision support, while displaying the tag results through visualization technology.

[0048] Among them, the S1 collects various types of enterprise-related data from big data sources, cleans the collected data, removes noise, missing values ​​and irrelevant information, and performs text preprocessing on the cleaned data, including word segmentation, stop word removal, stem extraction, and part-of-speech tagging, in preparation for subsequent processing. The data collection and preprocessing steps also include a real-time data update mechanism to ensure that the label recognition system can continuously capture the latest industry trends.

[0049] Among them, in the step of feature extraction and encoding, the feature extraction uses a convolutional neural network to extract a feature map from the preprocessed image and set it as X. After the convolutional layer of the convolutional neural network, the dimension of the feature map is further reduced by a pooling layer and the main features are extracted. The convolution operation in the convolutional neural network is implemented by the formula:

[0050]

[0051] In the formula, X is the input feature map, e is the convolution kernel, n is the bias term, and Y is the output feature map. Feature encoding is to reduce the dimension and encode the extracted feature map through the fully connected layer to obtain a feature vector of fixed length. The feature vector is further optimized using principal component analysis and linear discriminant analysis dimensionality reduction techniques. Finally, the feature vector is converted into a compact encoding form using the hash function method. The implementation formula of the fully connected layer is:

[0052] z=WY ij + b,

[0053] In the formula, Y ij is the input feature vector, W is the weight matrix, b is the bias vector, z is the output feature vector, and the encoded output of the RoBERTa model is pooled as needed to generate a fixed-length sentence representation vector.

[0054] Among them, S3, according to the method described in claim 1, first selects a pre-trained model, converts the pre-processed text data into a Token ID sequence format, uses the RoBERTa model to extract features from the text data, and captures the semantic features and contextual information of the text through the multi-layer pre-trained model structure of the model. On the basis of RoBERTa, an attention mechanism is further introduced to strengthen the extraction of key information. The attention weight calculation implementation formula is:

[0055]

[0056] In the formula, Q, K, and V represent the matrix representation of query, key, and value, respectively. kis the dimension of the key, which is used to scale the dot product result to prevent the gradient from disappearing. The softmax function is used to normalize the attention score to a probability distribution, and then normalize it using the softmax function to get the attention weight of each element. The feature vector processed by the attention mechanism is passed to the output layer, and the probability of each label is calculated by the softmax classification function. The implementation formula of the softmax function to calculate the probability of each label is:

[0057]

[0058] In the formula, z i is the original output of the model for the input x belonging to the i-th label, p(y i |x) is the probability that the input x belongs to the i-th label. Finally, the model is trained using the labeled dataset and the model parameters are optimized through the back propagation algorithm.

[0059] Among them, the time sensitivity of the dynamic label is realized by considering the changing trend, frequency of occurrence and correlation with other labels of the label in the time series data. First, the labeled data set is divided into a training set, a validation set and a test set to ensure that each set contains samples to cover the diversity of the data. Then, the model parameters are initialized according to the classifier of the attention mechanism, and the model is trained using the training set data. In each training batch, the forward propagation input data passes through the model, and the loss between the predicted label and the true label is calculated. At the same time, the back propagation algorithm is used to calculate the gradient of the loss function with respect to the model parameters, and the model parameters are updated to minimize the loss. The implementation of the loss function is shown as follows:

[0060]

[0061] In the formula, N is the number of samples, C is the number of categories, and y is ic is the true label of whether the i-th sample belongs to the c-th class, p ic is the probability that the i-th sample belongs to the c-th class predicted by the model. The optimization algorithm is used to further adjust the model structure and training strategy according to the test results. The optimization algorithm implementation formula is:

[0062]

[0063] In the formula, θ t is the model parameter at the tth iteration, η is the learning rate, It is the gradient of the loss function with respect to the model parameters. Finally, the model is used to predict the new input data, generate dynamic labels, and deploy the trained model to the actual application scenario.

[0064] Among them, in said S4, in the label optimization and evaluation step, the user feedback mechanism allows the user to directly participate in the label optimization process by means of label correction, label addition, and label weight adjustment.

[0065] Among them, the S4, user feedback mechanism ensures that the model can be continuously optimized and adjusted according to the actual application effect. First, the user compares the collected data with the preset benchmark value and historical data through the system to evaluate the changing trend of the model performance. Through data analysis, the reasons why the model performance does not meet expectations are identified. According to the data analysis results, the data preprocessing process is adjusted, the training data set is updated, and the model architecture is replaced to optimize the model performance. The effect of the optimized model is evaluated to verify whether it has achieved the expected optimization goal.

[0066] Among them, the S4 needs to randomly divide the data set into two mutually exclusive subsets, one as a training set and the other as a test set, train the model on the training set and evaluate the model performance on the test set; and randomly extract samples of the same size as the original data set from the original data set with replacement to form a training set, and use the unselected samples as a test set, train the model on the training set, and evaluate the model performance on the test set, repeat multiple times, each sampling forms a different training set and test set, so as to obtain multiple model performance evaluation results, take the average of the model performance of multiple iterations as the final performance evaluation result, and finally process the evaluation results.

[0067] Among them, in the step of applying and displaying labels in S5, the visual display method supports multi-dimensional and multi-perspective data exploration, allowing users to customize the display content and layout according to their needs.

[0068] Among them, the S5 is applied to data classification and label recognition tasks in other fields. First, the data source, data format, data quality, classification goals and requirements of the new field are understood. According to the data characteristics of the new field, the original data is cleaned, format converted, and feature extracted. At the same time, according to the classification task requirements of the new field, the RoBERTa model is adjusted, the pre-trained corpus is replaced, the model parameters are adjusted, and new features are introduced. The adjusted model is trained using the data from the new field, and the performance of the model is evaluated through cross-validation. The trained model is deployed to the actual application scenario, and the data in the new field is classified and label recognized. Finally, a feedback mechanism is established to continuously optimize and adjust the model according to the actual application effect.

[0069] In this example, specifically: S2, in the feature extraction and encoding step, the feature extraction uses a convolutional neural network to extract a feature map from the preprocessed image and set it to X. After the convolutional neural network convolution layer, the pooling layer is used to further reduce the dimension of the feature map and extract the main features. The convolution operation in the convolutional neural network is implemented as follows:

[0070]

[0071] In the formula, X is the input feature map, w is the convolution kernel, b is the bias term, and Y is the output feature map. Feature encoding is to reduce the dimension and encode the extracted feature map through the fully connected layer to obtain a feature vector of fixed length. The feature vector is further optimized using principal component analysis and linear discriminant analysis dimensionality reduction techniques. Finally, the feature vector is converted into a compact encoding form using the hash function method. The implementation formula of the fully connected layer is:

[0072] z=WY ij +b,

[0073] In the formula, Y ij is the input feature vector, W is the weight matrix, b is the bias vector, z is the output feature vector, and the encoded output of the RoBERTa model is pooled as needed to generate a fixed-length sentence representation vector.

[0074] In this example, specifically: S2, in the feature extraction and encoding step, the feature extraction uses a convolutional neural network to extract a feature map from the preprocessed image and set it to X. After the convolutional neural network convolution layer, the pooling layer is used to further reduce the dimension of the feature map and extract the main features. The convolution operation in the convolutional neural network is implemented as follows:

[0075]

[0076] In the formula, X is the input feature map, w is the convolution kernel, b is the bias term, and Y is the output feature map. Feature encoding is to reduce the dimension and encode the extracted feature map through the fully connected layer to obtain a feature vector of fixed length. The feature vector is further optimized using principal component analysis and linear discriminant analysis dimensionality reduction techniques. Finally, the feature vector is converted into a compact encoding form using the hash function method. The implementation formula of the fully connected layer is:

[0077] z=WY ij +b,

[0078] In the formula, Y ij is the input feature vector, W is the weight matrix, b is the bias vector, z is the output feature vector, and the encoded output of the RoBERTa model is pooled as needed to generate a fixed-length sentence representation vector.

[0079] In this example, specifically: S3, according to the method described in claim 1, first selects a pre-trained model, converts the pre-processed text data into a Token ID sequence format, uses the RoBERTa model to extract features from the text data, and captures the semantic features and contextual information of the text through the multi-layer pre-trained model structure of the model. On the basis of RoBERTa, an attention mechanism is further introduced to strengthen the extraction of key information. The attention weight calculation implementation formula is:

[0080]

[0081] In the formula, Q, K, and V represent the matrix representation of query, key, and value, respectively. k is the dimension of the key, which is used to scale the dot product result to prevent the gradient from disappearing. The softmax function is used to normalize the attention score to a probability distribution, and then normalize it using the softmax function to get the attention weight of each element. The feature vector processed by the attention mechanism is passed to the output layer, and the probability of each label is calculated by the softmax classification function. The implementation formula of the softmax function to calculate the probability of each label is:

[0082]

[0083] In the formula, z i is the original output of the model for the input x belonging to the i-th label, p(y i |x) is the probability that the input x belongs to the i-th label. Finally, the model is trained using the labeled dataset and the model parameters are optimized through the back propagation algorithm.

[0084] 5. In this example, specifically: S4, in which the time sensitivity of dynamic labels is achieved by considering the changing trend, frequency of occurrence and correlation with other labels of labels in time series data. First, the labeled data set is divided into training set, validation set and test set, ensuring that each set contains samples to cover the diversity of data. Then, the model parameters are initialized according to the classifier of the attention mechanism, and the model is trained using the training set data. In each training batch, the forward propagation input data passes through the model, and the loss between the predicted label and the true label is calculated. At the same time, the back-propagation algorithm is used to calculate the gradient of the loss function with respect to the model parameters, and the model parameters are updated to minimize the loss. The implementation of the loss function is shown as follows:

[0085]

[0086] In the formula, N is the number of samples, C is the number of categories, and y is ic is the true label of whether the i-th sample belongs to the c-th class, p icis the probability that the i-th sample belongs to the c-th class predicted by the model. The optimization algorithm is used to further adjust the model structure and training strategy according to the test results. The optimization algorithm implementation formula is:

[0087]

[0088] In the formula, θ t is the model parameter at the tth iteration, η is the learning rate, It is the gradient of the loss function with respect to the model parameters. Finally, the model is used to predict the new input data, generate dynamic labels, and deploy the trained model to the actual application scenario.

[0089] Working principle: By deploying cameras, sensors and other equipment in the monitoring area, collecting data at a preset frequency based on large data sets, and integrating them according to timestamps and location information, a multi-dimensional data set is formed, the data set is cleaned, missing values, outliers, and duplicate data are identified and processed to ensure data quality, and data is compressed according to data type and compression requirements, and the quality and compression ratio of the compressed data are checked to optimize storage and transmission efficiency, and then the processed data is converted into a unified format for subsequent analysis and processing;

[0090] Feature maps are extracted from the preprocessed multi-dimensional data set. These features are independent of specific situations and are universal. A convolutional neural network method is introduced to extract features from each data source and obtain feature encoding through weighted averaging. The weights of different data sources are dynamically adjusted according to the real-time data analysis results of the device and user feedback to optimize the accuracy of feature extraction. The RoBERTa model is used to deeply understand text data, including industry dynamic reports, social media comments, etc., to extract key information and semantic features. The weighted data is combined with the text features extracted by RoBERTa as the input of the abnormal behavior prediction model. On the basis of RoBERTa, the attention mechanism is further introduced to strengthen the extraction of key information. The parameters of the policy network are continuously updated using the reinforcement learning algorithm, so that the model can gradually learn the ability to take optimal actions in complex environments. The trained policy network parameters are integrated into the abnormal behavior prediction model to form the final dynamic label recognition model. The model is continuously optimized and updated according to the learning results of the policy network and the continuous feedback from users.

[0091] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0092] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A method for identifying industry dynamic labels by combining big data and ROBERTa, characterized in that: The following steps are involved: S1. Collect industry-related text data from multiple big data sources through automated means, and perform cleaning and preprocessing operations on the collected data to prepare the data for subsequent processing; S2. Use the RoBERTa pre-trained language model to deeply encode the pre-processed text data, capture the semantic features and contextual information of the text through the model's multi-layer Transformer structure, generate a deep semantic representation of the text, and combine the statistical features of big data to construct a multi-dimensional feature vector. S3. Based on the semantic representation and statistical features output by the RoBERTa model, deep learning algorithms and pre-trained model RoBERTa are used to automatically identify and generate dynamic tags related to the industry. Dynamic tags can reflect key information, trend changes and hot topics in text data, are time-sensitive, and can be dynamically adjusted as data is updated. S4. By introducing user feedback mechanism and automatic evaluation algorithm, the generated dynamic tags are continuously optimized and evaluated. User feedback is used to correct incorrect tags, supplement missing tags and adjust tag weights. The automatic evaluation algorithm can quantitatively evaluate the tag quality according to the preset evaluation criteria. S5. Apply the optimized dynamic tags to multiple fields such as industry analysis, market forecasting, information retrieval, and content recommendation to provide users with valuable industry insights and decision support, while displaying the tag results through visualization technology.

2. The industry dynamic tag recognition method combining big data and ROBERTa according to claim 1 is characterized by: The S1 collects various enterprise-related data from big data sources, cleans the collected data, removes noise, missing values ​​and irrelevant information, and performs text preprocessing on the cleaned data, including word segmentation, stop word removal, stem extraction, and part-of-speech tagging, in preparation for subsequent processing. The data collection and preprocessing steps also include a real-time data update mechanism to ensure that the label recognition system can continuously capture the latest industry trends.

3. The industry dynamic tag recognition method combining big data and ROBERTa according to claim 1 is characterized by: In the step of feature extraction and encoding, the feature extraction uses a convolutional neural network to extract a feature map from the preprocessed image, which is set to X. After the convolutional neural network convolution layer, the dimension of the feature map is further reduced through the pooling layer and the main features are extracted. The convolution operation in the convolutional neural network is implemented as follows: In the formula, X is the input feature map, e is the convolution kernel, n is the bias term, and Y is the output feature map. Feature encoding is to reduce the dimension and encode the extracted feature map through the fully connected layer to obtain a feature vector of fixed length. The feature vector is further optimized using principal component analysis and linear discriminant analysis dimensionality reduction techniques. Finally, the feature vector is converted into a compact encoding form using the hash function method. The implementation formula of the fully connected layer is: z=WY ij +b, In the formula, Y ij is the input feature vector, W is the weight matrix, b is the bias vector, z is the output feature vector, and the encoded output of the RoBERTa model is pooled as needed to generate a fixed-length sentence representation vector.

4. The industry dynamic tag recognition method combining big data and ROBERTa according to claim 3 is characterized by: Said S3, according to the method described in claim 1, first selects a pre-trained model, converts the pre-processed text data into a Token ID sequence format, uses the RoBERTa model to extract features from the text data, and captures the semantic features and contextual information of the text through the multi-layer pre-trained model structure of the model. On the basis of RoBERTa, an attention mechanism is further introduced to strengthen the extraction of key information. The attention weight calculation implementation formula is: In the formula, Q, K, and V represent the matrix representation of query, key, and value, respectively. k is the dimension of the key, which is used to scale the dot product result to prevent the gradient from disappearing. The softmax function is used to normalize the attention score to a probability distribution, and then normalize it using the softmax function to get the attention weight of each element. The feature vector processed by the attention mechanism is passed to the output layer, and the probability of each label is calculated by the softmax classification function. The implementation formula of the softmax function to calculate the probability of each label is: In the formula, z i is the original output of the model for the input x belonging to the i-th label, p(y i |x) is the probability that the input x belongs to the i-th label. Finally, the model is trained using the labeled dataset and the model parameters are optimized through the back propagation algorithm.

5. The industry dynamic tag recognition method combining big data and ROBERTa according to claim 4 is characterized by: The S4, in which the time sensitivity of the dynamic label is realized by considering the changing trend, frequency of occurrence and correlation with other labels of the label in the time series data, firstly divides the labeled data set into a training set, a validation set and a test set, and ensures that each set contains samples to cover the diversity of the data, and then initializes the model parameters according to the classifier of the attention mechanism, and trains the model using the training set data. In each training batch, the forward propagation input data passes through the model, and the loss between the predicted label and the true label is calculated. At the same time, the back propagation algorithm is used to calculate the gradient of the loss function with respect to the model parameters, and the model parameters are updated to minimize the loss, wherein the implementation of the loss function is publicized as: In the formula, N is the number of samples, C is the number of categories, and y is ic is the true label of whether the i-th sample belongs to the c-th class, p ic is the probability that the i-th sample belongs to the c-th class predicted by the model. The optimization algorithm is used to further adjust the model structure and training strategy according to the test results. The optimization algorithm implementation formula is: In the formula, θ t is the model parameter at the tth iteration, η is the learning rate, It is the gradient of the loss function with respect to the model parameters. Finally, the model is used to predict the new input data, generate dynamic labels, and deploy the trained model to the actual application scenario.

6. The industry dynamic tag recognition method combining big data and ROBERTa according to claim 1 is characterized by: In the step of S4, in which the label optimization and evaluation is performed, the user feedback mechanism allows the user to directly participate in the label optimization process by means of label modification, label addition, and label weight adjustment.

7. The industry dynamic tag recognition method combining big data and ROBERTa according to claim 6 is characterized by: The S4, user feedback mechanism ensures that the model can be continuously optimized and adjusted according to the actual application effect. First, the user compares the collected data with the preset benchmark value and historical data through the system to evaluate the changing trend of the model performance. The reason why the model performance does not meet expectations is identified through data analysis. According to the data analysis results, the data preprocessing process is adjusted, the training data set is updated, and the model architecture is replaced to optimize the model performance. The effect of the optimized model is evaluated to verify whether it has achieved the expected optimization goal.

8. The industry dynamic tag recognition method combining big data and ROBERTa according to claim 6 is characterized by: The S4 needs to randomly divide the data set into two mutually exclusive subsets, one as a training set and the other as a test set, train the model on the training set and evaluate the model performance on the test set; and randomly extract samples of the same size as the original data set from the original data set with replacement to form a training set, and use the unselected samples as a test set, train the model on the training set, and evaluate the model performance on the test set, and repeat this process multiple times, each sampling forming a different training set and test set, thereby obtaining multiple model performance evaluation results, taking the average of the model performance of multiple iterations as the final performance evaluation result, and finally processing the evaluation results.

9. The industry dynamic tag recognition method combining big data and ROBERTa according to claim 1 is characterized by: In S5, in the label application and display step, the visual display method supports multi-dimensional and multi-perspective data exploration, allowing users to customize the display content and layout according to their needs.

10. The industry dynamic tag recognition method combining big data and ROBERTa according to claim 1 is characterized by: The S5 is applied to data classification and label recognition tasks in other fields. First, the data source, data format, data quality, classification objectives and requirements of the new field are understood. According to the data characteristics of the new field, the original data is cleaned, format converted, and features are extracted. At the same time, according to the classification task requirements of the new field, the RoBERTa model is adjusted, the pre-trained corpus is replaced, the model parameters are adjusted, and new features are introduced. The adjusted model is trained using the data from the new field, and the performance of the model is evaluated through cross-validation. The trained model is deployed to the actual application scenario, and the data in the new field is classified and label recognized. Finally, a feedback mechanism is established to continuously optimize and adjust the model according to the actual application effect.

Images