Text analysis method and system based on data model

Through keyword guidance and meta-learning technology, the generalization performance of text analysis models in specific fields is optimized, and the problem of poor performance of existing technologies in specific fields is solved, and the high performance of the task is achieved under the few samples and multi-label tasks Accuracy classification and efficient text analysis.

CN118966239AInactive Publication Date: 2024-11-15YIMING UNIVERSAL TECH (BEIJING) CO LTD

Patent Information

Application Number
CN202411451449.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-17
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing text analysis methods perform poorly in data analysis in specific fields, especially in the case of small samples and multi-label, which cannot achieve efficient classification results, and relying on a large amount of supervised data leads to high labor and time costs.

Method used

The text analysis method based on data model is adopted to optimize the generalization performance of the model through keyword guidance and meta-learning technology. The specific steps include: collecting unsupervised data, building a keyword database, performing keyword guidance and shading, fine-tuning the pre-trained model, updating model parameters with meta-learning algorithms, performing multi-label classification, and optimizing model performance through semantic similarity analysis.

Benefits of technology

Improve the accuracy and efficiency of text analysis models in specific fields, reduce the dependence on large amounts of labeled data, realize high-precision classification performance under small samples and multi-label tasks, and provide more explanatory through semantic similarity analysis analysis results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118966239A_ABST
    Figure CN118966239A_ABST
Patent Text Reader

Abstract

The invention discloses a text analysis method and system based on a data model, and relates to the technical field of text analysis. A text analysis system based on a data model comprises a data acquisition and processing module, a keyword library generation module, an unsupervised field adaptation module, a grammar analysis module and an evaluation optimization module. According to the method, high-frequency keywords in a specific field are focused in the fine tuning process of the pre-training model through a keyword-guided covering method, so that semantic features of a target field are better captured; by combining the meta-learning technology, the model can quickly learn and adapt to different tasks under the condition of a small number of training samples, and by utilizing the model after meta-learning optimization, each label can be independently predicted, and high-precision classification performance is ensured to be realized under a multi-label task with a small number of samples.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of text analysis, and in particular to a text analysis method and system based on a data model. Background Art

[0002] With the rapid development of big data and artificial intelligence technologies, text analysis has been increasingly used in multiple fields, especially in the field of natural language processing (NLP). However, existing text analysis methods show poor generalization performance when faced with text data in specific fields. Although general pre-trained models perform well in open fields, they are not ideal in terms of accuracy in semantic understanding and classification in data analysis tasks in specific fields due to large differences in data features and few training samples. In addition, traditional multi-label classification methods cannot achieve efficient classification results when faced with few samples and multiple labels.

[0003] In order to solve this problem, domain adaptation and meta-learning have gradually become important directions for optimizing the generalization ability of models in recent years. Domain adaptation integrates unsupervised data in the target domain so that the pre-trained model can better adapt to the specific semantic features of the domain. Meta-learning technology updates model parameters through a small number of training samples, allowing the model to quickly adjust to different tasks, thereby improving its performance under few-sample conditions. Summary of the invention

[0004] Existing text analysis technologies usually rely on a large amount of supervised data, and the process of obtaining labeled data requires a lot of manpower and time costs. In order to improve the performance of text analysis models in small sample, multi-label tasks and reduce the dependence on a large amount of labeled data, this paper proposes a text analysis method based on a data model. By combining keyword guidance and meta-learning technology, the generalization performance of the model is optimized, making it more accurate and efficient in text analysis tasks in specific fields.

[0005] A text analysis method based on a data model, comprising: S1. Collecting unsupervised data in a target domain, performing preliminary preprocessing operations on the collected data, including data merging, data cleaning, data segmentation and data screening, to obtain standard text data; extracting high-frequency words in the target domain from the standard text data by means of word frequency statistics, and constructing a keyword library according to the frequency of the words, wherein the keyword library can effectively represent the core concepts and terms in the target domain; S2. Performing keyword-guided masking on the standard text data through the keyword library, replacing high-frequency keywords with masking marks, and preferentially selecting and masking high-frequency keywords in the keyword library, so that the model can better capture the semantic features of the target domain during the training process; using the standard text after masking and replacement as a masking language model task to fine-tune the pre-trained model, learn the lexical semantics and contextual relationships of the target domain, and optimize the generalization performance of the model in a specific domain; S3. Combining a meta-learning algorithm with the pre-trained model Perform optimization training to obtain a text analysis model, use meta-learning to update model parameters so that the model has the ability to quickly adjust in different tasks, and apply the optimized model parameters to multi-label classification tasks to improve task performance in few-sample, multi-label situations; send the text in the target field that needs to be analyzed into the text analysis model to perform multi-label classification tasks and obtain text analysis results. The model makes independent predictions for each label to ensure high-precision classification performance in multi-label tasks with few samples; S4. Perform semantic similarity analysis on the generated text analysis results and the preset reference text. By comparing with the reference text, comprehensively calculate the semantic similarity between the texts, map the similarity to the preset scoring range, and obtain an analytical evaluation of the degree of understanding of the concepts in the target field. The analytical evaluation is used for subsequent optimization of the model; S5. Output the text analysis results, including multi-label classification results and analytical evaluation, as a visualization report for user reference.

[0006] As a preferred technical solution of the present invention, it also includes a pre-training model; the pre-training model is composed of a multi-head self-attention unit, a feedforward neural unit and a masked language task unit. Each multi-head self-attention unit and feedforward neural unit includes a normalization layer and a residual connection layer. A total of 12 groups of multi-head self-attention units and feedforward neural units constitute the encoder structure. The word embedding vector is processed by the encoder, and the obtained data is output to the pooling layer of the masked language task unit of the pre-training model. The semantics corresponding to the masked mark is extracted and input to the fully connected layer of the model, and normalized classification is performed on the entire vocabulary; the standard text data is split into a vocabulary-level list using word segmentation, and full-word tags are added to the words that can form a vocabulary with the first word, and [cls] tags and [sep] tags are added at the beginning and end of the sentence respectively to generate a column with full-word tags in units of words. table; add full-word tags to the keywords in the keyword library to generate a keyword list with full-word tags; perform keyword-guided masking on the preprocessed unsupervised data list and keyword list to generate input data with masked tags, input the standard text data into the pre-trained model through word embedding, and perform calculation of the multi-head self-attention mechanism. During the calculation of the masked tag vector, the vector itself is calculated as the semantic vector of the context. After the text vector is output through the BERT layer, it will enter the pooling layer, extract the masked semantic vector in the text vector in the pooling layer, and perform cross-entropy classification on the vector in the fully connected layer, find the word corresponding to the masked tag in the vocabulary, and the word replaced by the masked tag is the masked label, calculate the loss and perform back propagation training; as a preferred technical solution of the present invention, it also includes keyword-guided masking; According to the difference between the characteristics of keywords and general knowledge domain vocabulary, the pre-training model is fine-tuned using vocabulary-level segmentation and keyword recognition masking; in the word segmentation process, the input standard text data is segmented into vocabulary using the word segmentation method, and the vocabulary is split into individual characters. The full-word label "#" is added before the Chinese characters except the first Chinese character to indicate that the Chinese character is connected to the previous character to form a vocabulary; in the masking process, according to the characteristics of the keywords, the vocabulary in the keyword library is extracted one by one. For the input text after the above segmentation, the vocabulary in the keyword library is compared with the vocabulary in the test text to determine whether the keyword exists in the input text and its location, record the location information, and replace the keywords in the sentence with mask labels. For a sentence, the minimum sentence length is set to 10 characters. If the sentence is not long enough, it will not participate in the training. The maximum number of masks is set to 20, that is, when the mask mark in a sentence reaches a maximum of 20, stop marking subsequent keywords.

[0007] As a preferred technical solution of the present invention, a text analysis model; a data input unit, consisting of a meta-learning channel and a multi-label learning channel. The meta-learning channel performs binary classification on three different tasks, so the data is a combination of three binary classification tasks. In the multi-label channel, the three binary classification tasks are combined into a multi-label classification task, so the data is multi-label data. The two channels input the model respectively at different stages; the data of the three tasks are randomly distributed to each batch, so that the three classification data are randomly arranged. After all the data are distributed, they are split into training set and validation set. In the training set, the data is split into support set and query set. In the input data of the multi-label classification task, multiple binary classification data are merged into multi-label classification data and input into the multi-label classification model together with the data set; in the meta-learning optimization unit, the text analysis model is initialized using the parameters of the pre-trained model, BCEloss is used as the classification loss, the [cls] vector represents the semantic information of the entire sentence, and the [cls] label in the pooling layer is used as the classification vector of the fully connected layer; in the multi-label classification unit, the model parameters optimized by meta-learning are retained, and a multi-label classification method combining BCEloss and Sigmoid is used at the output of the model. The generated model parameters are used as the weights of the final text analysis model.

[0008] As a preferred technical solution of the present invention, the training of the text analysis model: for the first batch of data, the support set is input into the model, and the model parameters are updated through a step of gradient descent. The parameters of the model are updated as follows: ; where θ is the initialization parameter of the original model, represents the performance evaluation value of the model on the first task set, represents the loss value on the first task set, Represents the gradient of the loss value of the first task set to θ; the query set of the current batch is sent to the model after the parameter update to obtain the query set loss of the current batch , and save it; use the initial model parameters θ to calculate the second batch, and so on, until the model is obtained in the last query set ; Calculate the total loss of the query set: ; where i represents the index of the batch. The gradient descent performed by the model on the support set does not directly participate in the final model parameter update. The model only updates the model parameters based on the total loss generated on the query set, as follows: , where α is a meta-parameter, Represented as the updated parameters on the support set, Represents the total loss value generated on the query task set; update the parameters to The model input validation set, the number of gradient descents in the validation set is N times, and the total loss in the validation set Querry set is , and update the model parameters, and verify that the updated model parameters are: ; where β represents the learning rate of meta-learning, Indicated by θ t is the performance evaluation value of the model with the parameters on the validation set, Represents the corresponding loss value, Denotes the parameter θ Meat The gradient of As the latest parameters, and trained with a multi-label dataset, the absolute output vector is calculated using the Sigmod activation function: ; Where x represents the input data, that is, the multi-label dataset; combined with BCEloss, calculate the single data loss of the multi-label classification task: ; where l is the number of labels, y i is the true value of label i, x i is the model’s predicted output on label i, Represents the predicted value x i Use the sigmoid function; where l represents the number of classifications of a piece of data, and the total loss in a batch is: ; where N represents all the data in a batch, and the loss is used to update the model parameters. After M rounds of model update, the final parameters are .

[0009] As a preferred technical solution of the present invention, it also includes a loss function, which uses a Sigmoid function to map the predicted value. The mapped value has been standardized between 0 and 1, and then BCELoss is used to calculate to obtain the loss value of multi-label classification, and then back propagation is performed.

[0010] As a preferred technical solution of the present invention, preprocessing operations are performed; data merging, merging all data into the same txt file; data cleaning, deleting unresolvable symbols, blank lines and spaces; data splitting, detecting symbols in the data, including symbols such as periods, exclamation marks and question marks, using the detected symbols as sentence symbols to split the data into sentences with complete semantics; data screening, checking the size of the sentence, and adjusting the sentences larger than 256 bytes and smaller than 20 bytes.

[0011] As a preferred technical solution of the present invention, it also includes the generation of a keyword library; obtaining a vocabulary standard table of the target field, performing word segmentation on the standard text data to obtain an analysis text vocabulary table, performing word frequency statistics on the analysis text vocabulary table, extracting words that appear frequently in the target field, screening the extracted high-frequency words, eliminating words that are not in the vocabulary standard table and noise words, sorting the screened high-frequency words, giving priority to incorporating words with high frequency into the keyword library, and constructing a keyword library that reflects the core concepts and terms in the target field and serves as a priority masking object in the subsequent language masking process.

[0012] As a preferred technical solution of the present invention, it also includes semantic similarity analysis, using a contrastive learning algorithm to compare the user text with the reference text one by one, and extract the semantic features and context structure in the text; based on the similarity of multi-anchor reference texts, the semantic similarity between the user text and the reference text is comprehensively calculated through context, vocabulary and semantic levels; the similarity calculation result is mapped to a preset scoring range to generate a final analysis and evaluation result, which is used to evaluate the degree of understanding of the generated content to the concepts of the target field.

[0013] A text analysis system based on a data model includes: a data collection and processing module, which is used to collect unsupervised data in a target field, and perform preliminary preprocessing operations on the collected data, including data merging, data cleaning, data segmentation and data screening, to obtain standard text data; a keyword library generation module, which is used to extract high-frequency words in the target field from the standard text data by means of word frequency statistics, and to construct a keyword library according to the frequency of the words; an unsupervised domain adaptation module, which is used to perform keyword-guided masking on the standard text data through the keyword library, and replace the high-frequency keywords with masking marks, so that the model can better capture the semantic features of the target field during the training process; the standard text after masking and replacement is used as a masking language model task to fine-tune the pre-trained model, learn the lexical semantics and contextual relationships of the target field, and optimize the generalization performance of the model in a specific field; a grammar analysis module, which is used to combine The meta-learning algorithm optimizes the pre-trained model to obtain a text analysis model, and uses meta-learning to update the model parameters so that the model has the ability to quickly adjust in different tasks. The optimized model parameters are applied to multi-label classification tasks to improve task performance in few-sample, multi-label situations. The text in the target field that needs to be analyzed is sent to the text analysis model to perform multi-label classification tasks to obtain text analysis results. The model makes independent predictions for each label to ensure high-precision classification performance in multi-label tasks with few samples. The evaluation and optimization module is used to perform semantic similarity analysis on the generated text analysis results and the preset reference text. By comparing with the reference text, the semantic similarity between the texts is comprehensively calculated, and the similarity is mapped to the preset scoring range to obtain an analytical evaluation of the degree of understanding of the concepts in the target field. The analytical evaluation is used for subsequent optimization of the model.

[0014] The present invention has the following advantages: 1. The present invention focuses on high-frequency keywords in a specific field during the fine-tuning process of the pre-trained model through a keyword-guided masking method, thereby better capturing the semantic features of the target field; by combining meta-learning technology, the model can quickly learn and adapt to different tasks with a small number of training samples, and by using the model optimized by meta-learning, each label can be independently predicted, ensuring high-precision classification performance in multi-label tasks with a small number of samples.

[0015] 2. The present invention uses an unsupervised domain adaptation method to effectively process unlabeled data using keywords, reducing the reliance on a large amount of labeled data. This greatly reduces the time and cost of manually labeling data, and enables the model to work effectively with the support of unsupervised data.

[0016] 3. Through semantic similarity analysis, the system can compare the text analysis results with the preset reference text and comprehensively evaluate the semantic similarity between the texts. This function can not only provide users with more explanatory analysis results, but also serve as a feedback mechanism for further optimization of the model and improve the accuracy of analysis and evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A structural diagram of a text analysis system based on a data model adopted in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to enable persons skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0019] Embodiment 1, a text analysis method based on a data model, comprises: S1. collecting unsupervised data in a target field, performing preliminary preprocessing operations on the collected data, including data merging, data cleaning, data segmentation and data screening, to obtain standard text data; extracting high-frequency words in the target field from the standard text data by means of word frequency statistics, and constructing a keyword library according to the frequency of the words, wherein the keyword library can effectively represent the core concepts and terms in the target field; preprocessing operations; data merging, merging all data into the same txt file; data cleaning, deleting unresolvable symbols, blank lines and spaces; data splitting, detecting symbols in the data, the detected symbols including symbols such as periods, exclamation marks and question marks, splitting the data using the detected symbols as sentence symbols, and segmenting out sentences with complete semantics; data screening, checking the size of the sentence, adjusting the sentences larger than 256 bytes and smaller than 20 bytes, directly deleting the sentences that are too small, and splitting the sentences that are too large again according to the middle punctuation marks.

[0020] Generation of keyword library; obtaining the vocabulary standard table of the target field, segmenting the standard text data into words, obtaining the analysis text vocabulary table, performing word frequency statistics on the analysis text vocabulary table, extracting the words that appear frequently in the target field, screening the extracted high-frequency words, eliminating the words that are not in the vocabulary standard table and noise words, sorting the screened high-frequency words, giving priority to the words that appear frequently into the keyword library, and constructing a keyword library that reflects the core concepts and terms in the target field and serves as the priority masking object in the subsequent language masking process.

[0021] S2. Perform keyword-guided masking on standard text data through the keyword library, replace high-frequency keywords with masking marks, give priority to and mask high-frequency keywords in the keyword library, so that the model can better capture the semantic features of the target domain during training; use the standard text after masking as the masking language model task to fine-tune the pre-trained model, learn the lexical semantics and contextual relationships of the target domain, and optimize the generalization performance of the model in specific domains; pre-trained model; the pre-trained model consists of a multi-head self-attention unit, a feedforward neural unit, and a masking language task unit. Each multi-head self-attention unit and feedforward neural unit contains a normalization layer and a residual connection layer, for a total of 12 groups of multi-head self-attention units. The encoder structure is composed of a word and a feedforward neural unit. The word embedding vector is processed by the encoder, and the obtained data is output to the pooling layer of the masked language task unit of the pre-trained model. The semantics corresponding to the masked mark is extracted and input to the fully connected layer of the model, and normalized classification is performed on the entire vocabulary. The standard text data is split into a vocabulary-level list using word segmentation, and full-word marks are added to the words that can form a vocabulary with the first word, and [cls] tags and [sep] tags are added to the beginning and end of the sentence respectively to generate a word-based list with full-word marks. Full-word marks are added to the keywords in the keyword library to generate a keyword list with full-word marks. Keyword guidance is performed on the preprocessed unsupervised data list and keyword list Masking, generating input data with masking marks, inputting standard text data into the pre-trained model through word embedding, and calculating the multi-head self-attention mechanism. During the calculation of the masking mark vector, the vector itself is calculated from the semantic vector of the context. After the text vector is output through the BERT layer, it will enter the pooling layer, extract the masking semantic vector in the text vector in the pooling layer, and perform cross-entropy classification on the vector in the fully connected layer. Find the word corresponding to the masking mark in the vocabulary, and the word replaced by the masking mark is the masked label. Calculate the loss and perform back-propagation training; the pre-training weights are trained using the Chinese full-word masking model, which already has text information in the field of general knowledge, including the grammatical structure of the language. , this type of information is equally important when predicting keywords; the Chinese full-word masking method used is also applicable to the keyword-guided masking method designed in this project, so that the model retains the source domain information while improving the generalization ability in the target domain; the keyword-guided method enables the model to focus more on the understanding of special vocabulary in the target domain. BERT uses a self-attention mechanism, and the semantic information of the vocabulary is represented by its context. Therefore, this method can grasp the keyword information of the target domain more quickly, improving the efficiency of unsupervised domain adaptation; keyword-guided masking; according to the difference between the characteristics of keywords and general knowledge domain vocabulary, use vocabulary-level segmentation and keyword recognition masking to fine-tune the pre-trained model;In the word segmentation process, the input standard text data is divided into words using the word segmentation method, and the words are divided into individual characters. The full word label "#" is added before the Chinese characters except the first Chinese character to indicate that the Chinese character is connected to the previous character to form a word; in the masking process, according to the characteristics of the keywords, the words in the keyword library are extracted one by one. For the input text after the above word segmentation, the words in the keyword library are compared with the words in the test text to determine whether the keywords exist in the input text and their positions, and the position information is recorded. The keywords in the sentence are replaced with masked labels. For a sentence, the minimum sentence length is set to 10 characters. If the sentence length is insufficient, it will not participate in the training. The maximum number of masks is set to 20, that is, when the mask marks in a sentence reach a maximum of 20, the subsequent keywords are stopped from being marked. The reason is that when there are too many mask marks in the sentence, there will also be more mask marks around the masked words, which makes the model unable to use the existing information to predict the mask mark, thereby increasing the difficulty of the model's prediction and causing the model to predict incorrectly. Accurate; S3. Combine the meta-learning algorithm to optimize the pre-trained model and obtain the text analysis model. Use meta-learning to update the model parameters so that the model has the ability to quickly adjust in different tasks. Apply the optimized model parameters to multi-label classification tasks to improve the task performance in the context of few samples and many labels. Send the text in the target field that needs to be analyzed into the text analysis model to perform the multi-label classification task and obtain the text analysis result. The model makes independent predictions for each label to ensure high-precision classification performance in the multi-label task with few samples. Text analysis model; The data input unit consists of a meta-learning channel and a multi-label learning channel. The meta-learning channel performs binary classification on three different tasks, so the data is a combination of three binary classification tasks. In the multi-label channel, the three binary classification tasks are merged into a multi-label classification task, so the data is multi-label data. The two channels input the model at different stages. The data of the three tasks are randomly assigned to each batch, so that the three classification data are randomly arranged. After all data are distributed, they are split into training sets and validation sets. In the training set, the data is split into support sets and query sets. In the input data of the multi-label classification task, multiple binary classification data are merged into multi-label classification data and input into the multi-label classification model together with the data set. The meta-learning optimization unit uses the parameters of the pre-trained model to initialize the text analysis model. The meta-learning training algorithm does not conflict with the model, but changes the input order and back propagation order of the data during training. Therefore, the previously trained initialized BERT weights are fully available. BCEloss is used as the classification loss, the [cls] vector represents the semantic information of the entire sentence, and the [cls] label in the pooling layer is used as the classification vector of the fully connected layer.The multi-label classification unit retains the model parameters optimized by meta-learning, uses a multi-label classification method combining BCEloss and Sigmoid at the model output, and the generated model parameters are used as the weights of the final text analysis model. ;

[0022] Training of the text analysis model: For the first batch of data, the support set is input into the model, and the model parameters are updated through a step of gradient descent. The model parameters are updated as follows: ; where θ is the initialization parameter of the original model, represents the performance evaluation value of the model on the first task set, represents the loss value on the first task set, Represents the gradient of the loss value of the first task set to θ; the query set of the current batch is sent to the model after the parameter update to obtain the query set loss of the current batch , and save it; use the initial model parameters θ to calculate the second batch, and so on, until the model is obtained in the last query set ; Calculate the total loss of the query set: ; where i represents the index of the batch. The gradient descent performed by the model on the support set does not directly participate in the final model parameter update. The model only updates the model parameters based on the total loss generated on the query set, as follows: , where α is a meta-parameter, Represented as the updated parameters on the support set, Represents the total loss value generated on the query task set; update the parameters to The model input validation set, the number of gradient descents in the validation set is N times, and the total loss in the validation set Querry set is , and update the model parameters, and verify that the updated model parameters are: ;Will As the latest parameters, and trained with a multi-label dataset, the absolute output vector is calculated using the Sigmod activation function: ; Combined with BCEloss, calculate the single data loss of multi-label classification tasks: ; where l is the number of labels, y i is the true value of label i, x i is the model’s predicted output on label i, Represents the predicted value x i Use the sigmoid function; where l represents the number of classifications of a piece of data, and the total loss in a batch is: ; where β represents the learning rate of meta-learning, Indicated by θ t is the performance evaluation value of the model with the parameters on the validation set, Represents the corresponding loss value, Denotes the parameter θ Meat Gradient; where N represents all the data in a batch, the loss is used to update the model parameters, and the final parameters of the model after M rounds are updated to ; Loss function, use Sigmoid function to map the predicted value, the mapped value has been standardized between 0 and 1, and then use BCELoss to calculate, get the loss value of multi-label classification, and then perform back propagation.

[0023] BCELoss is a loss function for binary classification. This method uses the cross entropy loss function, which is expressed as: ; Where N represents the number of data, x is the output of the model, and y is the true label; Sigmoid is a differentiable bounded function with non-negative derivatives at each point. It is often used in binary classification problems and the activation function of neural networks. The Sigmoid function form is: , where x represents the input data, i.e., a multi-label dataset; S4. Perform semantic similarity analysis on the generated text analysis results and the preset reference text. By comparing with the reference text, the semantic similarity between the texts is comprehensively calculated, and the similarity is mapped to a preset scoring interval to obtain an analytical evaluation of the degree of understanding of the concepts in the target domain. The analytical evaluation is used for subsequent optimization of the model; semantic similarity analysis uses a contrastive learning algorithm to compare the user text with the reference text one by one to extract the semantic features and contextual structure in the text; based on the similarity of multi-anchor reference texts, the semantic similarity between the user text and the reference text is comprehensively calculated at the context, vocabulary and semantic levels; the similarity calculation result is mapped to a preset scoring interval to generate the final analytical evaluation result, which is used to evaluate the degree of understanding of the generated content on the concepts in the target domain.

[0024] S5. Output the text analysis results, including multi-label classification results and analysis evaluation, as a visual report for user reference.

[0025] Example 2, a text analysis system based on a data model, see Figure 1As shown, it includes: a data collection and processing module, which is used to collect unsupervised data in the target field, and perform preliminary preprocessing operations on the collected data, including data merging, data cleaning, data segmentation and data screening, to obtain standard text data; a keyword library generation module, which is used to extract high-frequency words in the target field from the standard text data by means of word frequency statistics, and build a keyword library according to the frequency of the words; an unsupervised domain adaptation module, which is used to perform keyword-guided masking on the standard text data through the keyword library, and replace the high-frequency keywords with masking marks, so that the model can better capture the semantic features of the target field during the training process; the standard text after masking replacement is used as a masking language model task to fine-tune the pre-trained model, learn the lexical semantics and contextual relationships in the target field, and optimize the generalization performance of the model in a specific field; a grammar analysis module, which is used to combine the meta-learning algorithm to The pre-trained model is optimized and trained to obtain a text analysis model. Meta-learning is used to update the model parameters so that the model has the ability to quickly adjust in different tasks. The optimized model parameters are applied to multi-label classification tasks to improve the task performance in few-sample, multi-label situations. The text in the target field that needs to be analyzed is sent to the text analysis model to perform multi-label classification tasks to obtain text analysis results. The model makes independent predictions for each label to ensure high-precision classification performance in multi-label tasks with few samples. The evaluation and optimization module is used to perform semantic similarity analysis on the generated text analysis results and the preset reference text. By comparing with the reference text, the semantic similarity between the texts is comprehensively calculated, and the similarity is mapped to the preset scoring range to obtain an analytical evaluation of the degree of understanding of the concepts in the target field. The analytical evaluation is used for subsequent optimization of the model.

[0026] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A text analysis method based on a data model, characterized in that: include: S1. Collect unsupervised data in the target field, perform preliminary preprocessing operations on the collected data, including data merging, data cleaning, data segmentation and data screening, to obtain standard text data; extract high-frequency words in the target field from the standard text data by word frequency statistics, and build a keyword library based on the frequency of the words, which can effectively represent the core concepts and terms in the target field; S2. Perform keyword-guided masking on standard text data through the keyword library, replace high-frequency keywords with masking marks, and give priority to and mask high-frequency keywords in the keyword library, so that the model can better capture the semantic features of the target domain during training; The standard text after masking and replacement is used as the masking language model task to fine-tune the pre-trained model, learn the lexical semantics and contextual relationships of the target field, and optimize the generalization performance of the model in a specific field; S3. Combine the meta-learning algorithm to optimize the pre-trained model to obtain a text analysis model, use meta-learning to update the model parameters, so that the model has the ability to quickly adjust in different tasks, and apply the optimized model parameters to multi-label classification tasks to improve the task performance in a few-sample, multi-label situation; The text in the target field that needs to be analyzed is sent to the text analysis model to perform multi-label classification tasks and obtain text analysis results. The model makes independent predictions for each label to ensure high-precision classification performance in multi-label tasks with few samples. S4. Perform semantic similarity analysis on the generated text analysis results and the preset reference text. By comparing with the reference text, comprehensively calculate the semantic similarity between the texts, map the similarity to the preset scoring range, and obtain an analytical evaluation of the degree of understanding of the concepts in the target domain. The analytical evaluation is used for subsequent optimization of the model. S5. Output the text analysis results, including multi-label classification results and analytical evaluation, as a visual report for user reference.

2. A text analysis method based on a data model according to claim 1, characterized in that: It also includes a pre-training model; the pre-training model consists of a multi-head self-attention unit, a feedforward neural unit and a masked language task unit. Each multi-head self-attention unit and feedforward neural unit contains a normalization layer and a residual connection layer. A total of 12 groups of multi-head self-attention units and feedforward neural units constitute an encoder structure. The word embedding vector is processed by the encoder, and the obtained data is output to the pooling layer of the masked language task unit of the pre-training model. The semantics corresponding to the masked mark is extracted and input to the fully connected layer of the model, and normalization and classification are performed on the entire vocabulary; the standard text data is split into a vocabulary-level list using word segmentation, and full-word tags are added to the words that can form a vocabulary with the first word, and [cls] tags and [sep] tags are added at the beginning and end of the sentence respectively, to generate a word-based list with full-word tags; Add full-word tags to the keywords in the keyword library to generate a keyword list with full-word tags; Keyword-guided masking is performed on the preprocessed unsupervised data list and keyword list to generate input data with masked marks. The standard text data is input into the pre-trained model through word embedding, and the multi-head self-attention mechanism is calculated. During the calculation of the masked mark vector, the vector itself is calculated from the semantic vector of the context. After the text vector is output through the BERT layer, it will enter the pooling layer. The masked semantic vector in the text vector is extracted in the pooling layer, and the vector is cross-entropy classified in the fully connected layer. The word corresponding to the masked mark is found in the vocabulary. The word replaced by the masked mark is the masked label. The loss is calculated and back-propagation training is performed.

3. A text analysis method based on a data model according to claim 2, characterized in that: It also includes keyword-guided masking; according to the difference between the characteristics of keywords and general knowledge domain vocabulary, the pre-trained model is fine-tuned using vocabulary-level segmentation and keyword recognition masking; in the word segmentation process, the input standard text data is divided into vocabulary using the word segmentation method, and the vocabulary is divided into individual characters. The full-word label "#" is added before other Chinese characters except the first Chinese character to indicate that the Chinese character and the previous character are connected to form a vocabulary; in the masking process, according to the characteristics of the keywords, the vocabulary in the keyword library is extracted one by one. For the input text after the above segmentation, the vocabulary in the keyword library is compared with the vocabulary in the test text to determine whether the keyword exists in the input text and its location, record the location information, and replace the keywords in the sentence with mask labels. For a sentence, the minimum sentence length is set to 10 characters. If the sentence is not long enough, it will not participate in the training. The maximum number of masks is set to 20, that is, when the mask mark in a sentence reaches a maximum of 20, stop marking subsequent keywords.

4. A text analysis method based on a data model according to claim 1, characterized in that: Text analysis model; data input unit, consisting of a meta-learning channel and a multi-label learning channel. The meta-learning channel performs binary classification on three different tasks, so the data is a combination of three binary classification tasks. In the multi-label channel, the three binary classification tasks are combined into a multi-label classification task, so the data is multi-label data. The two channels input the model at different stages; the data of the three tasks are randomly assigned to each batch, so that the three classification data are randomly arranged; after all the data are assigned, they are split into a training set and a validation set. In the training set, the data is split into a support set and a query set. In the input data of the multi-label classification task, multiple binary classification data are combined into multi-label classification data and input into the multi-label classification model together with the data set; The meta-learning optimization unit uses the parameters of the pre-trained model to initialize the text analysis model, uses BCEloss as the classification loss, the [cls] vector represents the semantic information of the entire sentence, and uses the [cls] label in the pooling layer as the classification vector of the fully connected layer; the multi-label classification unit retains the model parameters optimized by meta-learning, and uses a multi-label classification method combining BCEloss and Sigmoid at the model output. The generated model parameters are used as the weights of the final text analysis model.

5. A data model-based text analysis method according to claim 4, characterized in that: Training of the text analysis model: For the first batch of data, the support set is input into the model, and the model parameters are updated through a step of gradient descent. The model parameters are updated as follows: ; where θ is the initialization parameter of the original model, represents the performance evaluation value of the model on the first task set, represents the loss value on the first task set, Represents the gradient of the loss value of the first task set to θ; the query set of the current batch is sent to the model after the parameter update to obtain the query set loss of the current batch , and save it; use the initial model parameters θ to calculate the second batch, and so on, until the model is obtained in the last query set ; Calculate the total loss of the query set: ; where i represents the index of the batch. The gradient descent performed by the model on the support set does not directly participate in the final model parameter update. The model only updates the model parameters based on the total loss generated on the query set, as follows: , where α is a meta-parameter, Represented as the updated parameters on the support set, Represents the total loss value generated on the query task set; update the parameters to The model input validation set, the number of gradient descents in the validation set is N times, and the total loss in the validation set Querry set is , and update the model parameters. The model parameters updated in the verification part are: ; where β represents the learning rate of meta-learning, Indicated by θ t is the performance evaluation value of the model with the parameters on the validation set, represents the corresponding loss value, Denotes the parameter θ Meat The gradient of As the latest parameters, and trained with a multi-label dataset, the absolute output vector is calculated using the Sigmod activation function: ; Where x represents the input data, that is, the multi-label dataset; combined with BCEloss, calculate the single data loss of the multi-label classification task: ; where l is the number of labels, y i is the true value of label i, x i is the model’s predicted output on label i, Represents the predicted value x i Use the sigmoid function; where l represents the number of classifications of a piece of data, and the total loss in a batch is: ; where N represents all the data in a batch, the model parameters are updated using the loss, and after M rounds of model parameters are finally updated to .

6. A text analysis method based on a data model according to claim 4, characterized in that: It also includes a loss function, which uses the Sigmoid function to map the predicted value. The mapped value has been standardized between 0 and 1. Then BCELoss is used to calculate the loss value of multi-label classification, and then backpropagation is performed.

7. A text analysis method based on a data model according to claim 1, characterized in that: Preprocessing operation; data merging, merging all data into the same txt file; Data cleaning: delete unparseable symbols, blank lines, and spaces; data splitting: detect symbols in the data, including periods, exclamation marks, and question marks, and use the detected symbols as sentence symbols to split the data into sentences with complete semantics; data screening: check the size of the statement and adjust the statements larger than 256 bytes and smaller than 20 bytes.

8. A data model-based text analysis method according to claim 1, characterized in that: It also includes the generation of a keyword library; obtaining a standard vocabulary table for the target field, segmenting the standard text data into words, obtaining an analysis text vocabulary table, performing word frequency statistics on the analysis text vocabulary table, extracting words that appear frequently in the target field, screening the extracted high-frequency words, eliminating words that are not in the standard vocabulary table and noise words, sorting the screened high-frequency words, giving priority to incorporating words with high frequency into the keyword library, and constructing a keyword library that reflects the core concepts and terms in the target field and serves as a priority masking object in the subsequent language masking process.

9. A text analysis method based on a data model according to claim 1, characterized in that: It also includes semantic similarity analysis, which uses a contrastive learning algorithm to compare user text with reference text one by one to extract semantic features and contextual structure in the text; based on the similarity of multi-anchor reference texts, the semantic similarity between user text and reference text is comprehensively calculated through context, vocabulary and semantic levels; the similarity calculation results are mapped to a preset scoring range to generate the final analysis and evaluation results, which are used to evaluate the degree of understanding of the generated content on the concepts of the target domain.

10. A text analysis system based on a data model, characterized in that: The system applies a data model-based text analysis method as described in any one of claims 1 to 9, including: a data collection and processing module, which is used to collect unsupervised data in a target field, and perform preliminary preprocessing operations on the collected data, including data merging, data cleaning, data segmentation and data screening, to obtain standard text data; a keyword library generation module, which is used to extract high-frequency words in the target field from the standard text data by means of word frequency statistics, and construct a keyword library according to the frequency of the words; an unsupervised domain adaptation module, which is used to perform keyword-guided masking on the standard text data through the keyword library, and replace high-frequency keywords with masking marks, so that the model can better capture the semantic features of the target field during the training process; the standard text after masking and replacement is used as a masking language model task to fine-tune the pre-trained model, learn the lexical semantics and contextual relationships of the target field, and optimize the generalization performance of the model in a specific field; The grammar analysis module is used to optimize the training of the pre-trained model in combination with the meta-learning algorithm to obtain a text analysis model, and use meta-learning to update the model parameters so that the model has the ability to quickly adjust in different tasks. The optimized model parameters are applied to multi-label classification tasks to improve the task performance in the context of few samples and many labels. The text in the target field that needs to be analyzed is sent to the text analysis model to perform the multi-label classification task to obtain the text analysis results. The model makes independent predictions for each label to ensure high-precision classification performance in the multi-label task of few samples. The evaluation and optimization module is used to perform semantic similarity analysis on the generated text analysis results and the preset reference text. By comparing with the reference text, the semantic similarity between the texts is comprehensively calculated, and the similarity is mapped to the preset scoring range to obtain an analytical evaluation of the degree of understanding of the concepts in the target field. The analytical evaluation is used for subsequent optimization of the model.

Citation Information

Patent Citations

  • Softmax-based text multi-label classification method

    CN113688911A

  • Financial event multi-label classification method and system

    CN115099307A

  • System and method for generating a multi-label classifier for textual communications

    US20230306051A1

Cited By

  • Keyword generation method and system based on big data

    CN118886416A

  • Visual information monitoring method and system based on talent management

    CN120031526A

  • Unsupervised enhanced large model ultra-long text set data comparative analysis method and system

    CN120181068A

  • Function identification method and system based on AI enhancement

    CN120181095A

  • Text specification marking method and system based on rule corpus

    CN121072515A