Model training method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a model training method and device, electronic equipment and a storage medium, and relates to the technical field of deep learning, and the method comprises the steps: carrying out the word segmentation of a theme document, and obtaining a word segmentation list; a sliding window for the word segmentation list is obtained, word frequency statistics is conducted on the word segmentation list according to the sliding window, and a global word frequency list corresponding to the word segmentation list is obtained; performing feature statistics on the global word frequency table to obtain corresponding sparse features; performing feature fusion on the sparse features according to the global word frequency table to obtain a dense matrix; obtaining a domain dictionary corresponding to the subject document, and performing weighting processing on the dense matrix according to the domain dictionary to obtain a corresponding mixed feature matrix; and training the initial classifier to be trained according to the mixed feature matrix to obtain a trained target classifier, thereby effectively reducing dependence on manual annotation, forming an end-to-end cold start solution, reducing noise interference in a model training process, and shortening a training period.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, in particular to a model training method, a model training device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] With the development of traditional machine learning and large model fields, more and more model training needs to be used to meet the model classification scene requirements such as user intent recognition in daily development and production. However, the increasingly refined and specialized scenarios and requirements may encounter situations where the business side does not have much target data accumulation or the source data has low value for the current requirements, making it impossible to perform data annotation work or the cost of data annotation is extremely high. Detailed and specialized scenarios also lack available open source data sets for transfer learning. Ultimately, it leads to the predicament of lacking a trainable data set for model training.

[0003] The training difficulty of the current classification model lies in: on the one hand, the construction cost of high-quality annotated data sets continues to rise, and there is strong dependence on domain experts, long annotation period and other bottlenecks; on the other hand, a large amount of low-value text data lacks an effective utilization mechanism, resulting in that its potential value is not fully tapped.

[0004] In related technologies, multiple focuses are on semi-supervised learning or transfer learning, but there are significant challenges when dealing with low correlation text: the method based on pseudo-label is easily disturbed by noise data, and the error propagation rate is high; when the theme correlation is low, the model performance of cross-domain transfer learning will be greatly degraded. In addition, although the large model has certain zero sample learning ability, when the input text lacks clear theme direction, its classification accuracy is generally 3 to 5 percentage points lower than the benchmark value, and the large model used for classification model will consume a large amount of GPU (Graphics Processing Unit, graphics processing unit) resources. This technical bottleneck is particularly prominent in vertical fields. Due to data sensitivity, financial, communication and other industries have few available annotated data, and non-structured data processing technology is difficult to extract effective training signals from a large amount of weakly related text; traditional text enhancement methods have insufficient key feature capture rate and are easy to introduce semantic distortion when processing loosely structured text. SUMMARY

[0005] The embodiments of the present application provide a model training method, device, electronic device and computer readable storage medium to solve or partially solve the problem that the annotated data in a specific field has strong dependence on manual annotation and is difficult to process low-value text, and the model training process is easily disturbed by noise and has a long training period.

[0006] The embodiments of the present application disclose a model training method, comprising:

[0007] segmenting the topic document of different categories to obtain a segmentation list corresponding to the topic document;

[0008] obtaining a sliding window for the segmentation list, and performing word frequency statistics on the segmentation list according to the sliding window to obtain a global word frequency table corresponding to the segmentation list;

[0009] performing feature statistics on the global word frequency table to obtain a sparse feature;

[0010] performing feature fusion on the sparse feature according to the global word frequency table to obtain a dense matrix;

[0011] obtaining a domain dictionary corresponding to the topic document, and performing weighted processing on the dense matrix according to the domain dictionary to obtain a hybrid feature matrix;

[0012] training an initial classifier to be trained according to the hybrid feature matrix to obtain a target classifier after training.

[0013] In some possible implementation manners, the performing word frequency statistics on the segmentation list according to the sliding window to obtain the global word frequency table corresponding to the segmentation list includes:

[0014] obtaining a window step corresponding to the sliding window;

[0015] segmenting the segmentation list according to the sliding window and the window step to obtain a candidate phrase, and counting the frequency of each candidate phrase in the topic document;

[0016] constructing the global word frequency table corresponding to the segmentation list based on the candidate phrase and the corresponding frequency.

[0017] In some possible implementation manners, the performing feature statistics on the global word frequency table to obtain the sparse feature includes:

[0018] obtaining a total number of documents corresponding to the topic document;

[0019] obtaining a number of documents containing the candidate phrase;

[0020] obtaining a smoothing hyperparameter, and calculating the inverse document probability corresponding to each candidate phrase by using the total number of documents, the number of documents, and the smoothing hyperparameter;

[0021] performing feature fusion on the frequency and the inverse document probability corresponding to each candidate phrase to obtain a sparse matrix.

[0022] In some possible implementation manners, the performing feature fusion on the sparse feature according to the global word frequency table to obtain the dense matrix includes:

[0023] performing theme feature fusion on the global word frequency table and the sparse feature to obtain a corresponding theme enhanced weight matrix;

[0024] performing word vector embedding processing on the global word frequency table and the theme enhanced weight matrix to obtain a corresponding dense matrix.

[0025] In some possible implementation manners, the performing theme feature fusion on the global word frequency table and the sparse feature to obtain a corresponding theme enhanced weight matrix comprises:

[0026] obtaining a theme fusion model;

[0027] converting the global word frequency table into a word item frequency vector;

[0028] inputting the word item frequency vector into the theme fusion model to perform probability mapping, to obtain a document level probability and a word item level probability;

[0029] performing tensor point multiplication operation on the document level probability, the word item level probability and the sparse matrix to obtain a word item weight matrix, and performing normalization on the word item weight matrix to obtain a corresponding theme enhanced weight matrix.

[0030] In some possible implementation manners, the performing word vector embedding processing on the global word frequency table and the theme enhanced weight matrix to obtain a corresponding dense matrix comprises:

[0031] obtaining a word vector model and a word table corresponding to the word vector model;

[0032] taking a candidate phrase in the global word frequency table and existing in the word table as a first candidate phrase, and obtaining a first word vector corresponding to the first candidate phrase from the word table;

[0033] taking a candidate phrase in the global word frequency table and not existing in the word table as a second candidate phrase, and splitting the second candidate phrase into subwords;

[0034] obtaining a second word vector corresponding to each of the subwords from the word table;

[0035] performing weighted average processing based on the second word vector corresponding to each of the subwords to obtain a third word vector corresponding to the second candidate phrase;

[0036] performing word vector embedding processing on the first word vector, the third word vector and the theme enhanced weight matrix to obtain a corresponding dense matrix.

[0037] In some possible implementation manners, the weighting the dense matrix according to the domain dictionary to obtain a corresponding mixed feature matrix comprises:

[0038] extracting a domain keyword list from the domain dictionary;

[0039] matching the candidate phrases in the global word frequency table with the domain keyword list, and weighting the word items corresponding to the candidate phrases in the dense matrix according to a matching result to obtain an enhanced weight matrix;

[0040] performing dimension reduction processing on the enhanced weight matrix to obtain a corresponding mixed feature matrix.

[0041] In some possible implementation manners, the weighting the word items corresponding to the candidate phrases in the dense matrix according to the matching result to obtain the enhanced weight matrix comprises:

[0042] taking the candidate phrases in the global word frequency table and present in the domain keyword list as third candidate phrases, and increasing the weights of the first word items corresponding to the third candidate phrases in the dense matrix;

[0043] taking the candidate phrases in the global word frequency table and not present in the domain keyword list as fourth candidate phrases, and performing smoothing calibration processing on the second word items corresponding to the fourth candidate phrases in the dense matrix;

[0044] obtaining the enhanced weight matrix based on the weight increasing processing and / or the smoothing calibration processing performed on the dense matrix.

[0045] In some possible implementation manners, the performing dimension reduction processing on the enhanced weight matrix to obtain the corresponding mixed feature matrix comprises:

[0046] performing normalization on the enhanced weight matrix to obtain a normalized matrix;

[0047] calculating a covariance matrix corresponding to the normalized matrix;

[0048] performing singular value decomposition on the covariance matrix to obtain corresponding eigenvalues;

[0049] obtaining a cumulative contribution rate corresponding to each of the eigenvalues, and selecting an eigenvalue with a cumulative contribution rate greater than or equal to a first preset threshold as a target eigenvector, and constructing a projection matrix based on each of the target eigenvectors;

[0050] projecting the projection matrix into a low-dimensional orthogonal space to obtain a corresponding mixed feature matrix.

[0051] In some possible implementation manners, the training of the initial classifier to be trained according to the mixed feature matrix comprises:

[0052] obtaining an evaluation data set;

[0053] inputting the mixed feature matrix into the initial classifier to be trained to obtain a prediction result, and inputting the evaluation data set into the initial classifier to obtain an evaluation result;

[0054] comparing the prediction result with the evaluation result to obtain a corresponding error value;

[0055] iterating the initial classifier based on the error value until the error value is less than or equal to a second preset threshold, stopping iteration, and obtaining a target classifier after training.

[0056] In some possible implementation manners, the word segmentation of the topic document of different categories comprises:

[0057] obtaining topic documents of different categories;

[0058] segmenting the topic documents according to a domain dictionary to obtain a corresponding word segmentation list.

[0059] The embodiment of the application further discloses a model training device, comprising:

[0060] a word segmentation module configured to perform word segmentation on topic documents of different categories to obtain a corresponding word segmentation list of the topic documents;

[0061] a word frequency statistics module configured to obtain a sliding window for the word segmentation list, and perform word frequency statistics on the word segmentation list according to the sliding window to obtain a global word frequency table corresponding to the word segmentation list;

[0062] a feature statistics module configured to perform feature statistics on the global word frequency table to obtain corresponding sparse features;

[0063] a feature fusion module configured to perform feature fusion on the sparse features according to the global word frequency table to obtain a dense matrix;

[0064] a weighting module configured to obtain a domain dictionary corresponding to the topic documents, and perform weighting processing on the dense matrix according to the domain dictionary to obtain a corresponding mixed feature matrix;

[0065] a training module configured to train an initial classifier to be trained according to the mixed feature matrix to obtain a target classifier after training.

[0066] In some possible implementation manners, the word frequency counting module is specifically configured to:

[0067] obtain a window step length corresponding to the sliding window;

[0068] divide the word segmentation list according to the sliding window and the window step length, to obtain corresponding candidate phrases, and count frequencies of the candidate phrases in the theme document;

[0069] construct a global word frequency table corresponding to the word segmentation list based on the candidate phrases and the corresponding frequencies.

[0070] In some possible implementation manners, the word frequency counting module is specifically configured to:

[0071] obtain a total number of documents corresponding to the theme document;

[0072] obtain a number of documents containing the candidate phrases;

[0073] obtain a smoothing hyperparameter, and calculate the inverse document probability corresponding to each candidate phrase by using the total number of documents, the number of documents, and the smoothing hyperparameter;

[0074] perform feature fusion by using the frequency and the inverse document probability corresponding to each candidate phrase, to obtain a sparse matrix.

[0075] In some possible implementation manners, the feature fusion module is specifically configured to:

[0076] perform theme feature fusion by using the global word frequency table and the sparse feature, to obtain a theme enhanced weight matrix;

[0077] perform word vector embedding processing by using the global word frequency table and the theme enhanced weight matrix, to obtain a dense matrix.

[0078] In some possible implementation manners, the feature fusion module is specifically configured to:

[0079] obtain a theme fusion model;

[0080] convert the global word frequency table into a word item frequency vector;

[0081] input the word item frequency vector into the theme fusion model to perform probability mapping, to obtain a document-level probability and a word item-level probability;

[0082] perform tensor point multiplication operation on the document-level probability, the word item-level probability, and the sparse matrix, to obtain a word item weight matrix, and normalize the word item weight matrix to obtain a corresponding theme enhanced weight matrix.

[0083] In some possible implementation manners, the feature fusion module is specifically configured to:

[0084] obtain a word vector model and a word list corresponding to the word vector model;

[0085] take a candidate phrase in the global word frequency list and existing in the word list as a first candidate phrase, and obtain a first word vector corresponding to the first candidate phrase from the word list;

[0086] take a candidate phrase in the global word frequency list and not existing in the word list as a second candidate phrase, and split the second candidate phrase into subwords;

[0087] obtain a second word vector corresponding to each of the subwords from the word list;

[0088] perform weighted average processing based on the second word vector corresponding to each of the subwords to obtain a third word vector corresponding to the second candidate phrase;

[0089] perform word vector embedding processing on the first word vector, the third word vector, and the topic enhancement weight matrix to obtain a corresponding dense matrix.

[0090] In some possible implementation manners, the weighting module is specifically configured to:

[0091] extract a domain keyword list from the domain dictionary;

[0092] match a candidate phrase in a global word frequency list with the domain keyword list, and perform weighted processing on a word item corresponding to the candidate phrase in the dense matrix according to a matching result to obtain an enhanced weight matrix;

[0093] perform dimension reduction processing on the enhanced weight matrix to obtain a corresponding hybrid feature matrix.

[0094] In some possible implementation manners, the weighting module is specifically configured to:

[0095] take a candidate phrase in a global word frequency list and existing in the domain keyword list as a third candidate phrase, and perform weight increase processing on a first word item corresponding to the third candidate phrase in the dense matrix;

[0096] take a candidate phrase in the global word frequency list and not existing in the domain keyword list as a fourth candidate phrase, and perform smoothing calibration processing on a second word item corresponding to the fourth candidate phrase in the dense matrix;

[0097] obtain an enhanced weight matrix based on the weight increase processing and / or the smoothing calibration processing performed on the dense matrix.

[0098] In some possible implementation manners, the weighting module is specifically configured to:

[0099] normalizing the enhanced weight matrix to obtain a normalized matrix;

[0100] calculating a covariance matrix corresponding to the normalized matrix;

[0101] performing singular value decomposition on the covariance matrix to obtain corresponding eigenvalues;

[0102] obtaining cumulative contribution rates corresponding to each of the eigenvalues, and selecting an eigenvalue with a cumulative contribution rate greater than or equal to a first preset threshold as a target eigenvector, and constructing a projection matrix based on each of the target eigenvectors;

[0103] projecting the projection matrix into a low-dimensional orthogonal space to obtain a corresponding mixed feature matrix.

[0104] In some possible implementation manners, the training module is specifically configured to:

[0105] obtaining an evaluation data set;

[0106] inputting the mixed feature matrix into an initial classifier to be trained to obtain a prediction result, and inputting the evaluation data set into the initial classifier to obtain an evaluation result;

[0107] comparing the prediction result with the evaluation result to obtain a corresponding error value;

[0108] iterating the initial classifier based on the error value until the error value is less than or equal to a second preset threshold, stopping iteration, and obtaining a target classifier after training.

[0109] In some possible implementation manners, the word segmentation module is specifically configured to:

[0110] obtaining subject documents of different categories;

[0111] segmenting words in the subject documents according to a domain dictionary to obtain a corresponding word segmentation list.

[0112] An electronic device is also disclosed in the embodiments of the present application, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus;

[0113] The memory is configured to store a computer program.

[0114] The processor is configured to execute the program stored in the memory, and implement the method according to the embodiments of the present application.

[0115] The embodiment of the present application also discloses a computer readable storage medium, which stores instructions, and when executed by one or more processors, causes the processor to execute the method according to the embodiment of the present application.

[0116] The embodiment of the present application has the following advantages:

[0117] In the embodiment of the present application, by performing word segmentation on the subject documents of different categories, a word segmentation list corresponding to the subject documents is obtained, a sliding window for the word segmentation list is acquired, word frequency statistics of the word segmentation list are performed according to the sliding window, a global word frequency table corresponding to the word segmentation list is obtained, feature statistics of the global word frequency table are performed, corresponding sparse features are obtained, the sparse features are fused according to the global word frequency table, a dense matrix is obtained, a domain dictionary corresponding to the subject documents is acquired, and the dense matrix is weighted according to the domain dictionary, a corresponding hybrid feature matrix is obtained, and finally the initial classifier to be trained is trained according to the hybrid feature matrix, and a trained target classifier is obtained, so that by optimizing the feature extraction process, not only the local word frequency features are extracted, but also the subject features are fused and weighted, the dependence on artificial labeling is effectively reduced, the low-value text can be converted into the hybrid feature matrix that can be used for model training through multi-dimensional feature engineering, an end-to-end cold start solution is formed, noise interference in the model training process is effectively reduced, the training period is shortened, and the effect of model training is improved. BRIEF DESCRIPTION OF DRAWINGS

[0118] Figure 1 is a step flowchart of a model training method provided in the embodiment of the present application;

[0119] Figure 2 is a processing flow diagram of a classifier cold start provided in the embodiment of the present application;

[0120] Figure 3 is a structural block diagram of a model training device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0121] In order to make the above-mentioned objects, features and advantages of the present application more apparent and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0122] As an example, the current classification model training difficulty lies in: on the one hand, the construction cost of high-quality labeled data set continues to rise, and there is strong dependence on domain experts and long labeling period; on the other hand, a large amount of low-value text data lacks effective utilization mechanism, so that its potential value is not fully tapped.

[0123] In the related art, multi-focus is on semi-supervised learning or transfer learning, but there are significant challenges in processing low correlation text: the pseudo-label-based method is easily disturbed by noisy data, and the error propagation rate is high; when the theme correlation is low, the model performance of cross-domain transfer learning will be greatly degraded. In addition, although the large model has certain zero sample learning ability, when the input text lacks clear theme direction, its classification accuracy is generally 3 to 5 percentage points lower than the benchmark value, and the large model used for the classification model will consume a large amount of GPU resources.

[0124] This technical bottleneck is particularly prominent in vertical fields. Due to data sensitivity, financial, communication and other industries have limited labeled data available, and existing unstructured data processing techniques cannot extract effective training signals from massive weakly correlated text. Traditional text enhancement methods have insufficient key feature capture rates and are prone to introducing semantic distortion. Therefore, developing a cold start technology for low-value unstructured text and establishing a mapping mechanism from weakly correlated data to strong semantic space may be another path to break through the current AI model training data dilemma.

[0125] To this end, in the present application, by segmenting different categories of theme documents, a segmentation list corresponding to the theme documents is obtained, and a sliding window for the segmentation list is obtained, and the word frequency of the segmentation list is counted according to the sliding window, to obtain a global word frequency table corresponding to the segmentation list. Then, feature statistics are performed on the global word frequency table to obtain corresponding sparse features, and then the sparse features are fused according to the global word frequency table to obtain a dense matrix. Then, a domain dictionary corresponding to the theme document is obtained, and the dense matrix is weighted according to the domain dictionary to obtain a corresponding hybrid feature matrix. Finally, the initial classifier to be trained is trained according to the hybrid feature matrix to obtain the trained target classifier. Thus, by optimizing the feature extraction process, not only the local word frequency features are extracted, but also the theme features are fused and weighted, effectively reducing the dependence on manual labeling. Through multi-dimensional feature engineering, low-value text can be converted into a hybrid feature matrix that can be used for model training, forming an end-to-end cold start solution, effectively reducing noise interference in the model training process, shortening the training period, and improving the effect of model training.

[0126] In order for those skilled in the art to better understand the technical solutions in the embodiments of the present application, some technical features involved in the embodiments of the present application are explained and described as follows:

[0127] TF-IDF (Term Frequency-Inverse Document Frequency, i.e. term frequency-inverse document frequency): a statistical method for evaluating the importance of words in a document. TF (term frequency) measures the number of times a word appears in a document, and IDF (inverse document frequency) reduces the weight of common words.

[0128] Low-value unstructured text data: refers to text data that is associated with a specific topic but has low relevance and loose structure, making it difficult to form an effective labeled data set. For example, chat records, comments, and statistical information with low relevance to the topic.

[0129] Cold start: before the classifier is formally trained using the labeled data set, a certain training method is used to make the model learn certain weight parameters and have a certain classification ability.

[0130] Smooth IDF: a smoothing parameter is introduced in the IDF formula to avoid extreme values of IDF for non-existent words or low-frequency words, and to improve numerical stability.

[0131] LDA (Latent Dirichlet Allocation): a topic model algorithm that assumes that a document is generated by mixing multiple topics, and each topic is a probability distribution of words.

[0132] Domain knowledge weighting: based on a domain dictionary (such as medical terminology), the TF-IDF weight of the keyword is amplified to enhance the importance of domain-related features. For example, the weight of "diabetes" in medical text is increased by 1.5 times.

[0133] PCA (Principal Component Analysis): a linear dimension reduction method that projects high-dimensional data into a low-dimensional space by retaining the direction with the largest variance, commonly used to compress the dimension of word vectors.

[0134] Reference Figure 1 , shows a step flowchart of a model training method provided in an embodiment of the application, which can specifically include the following steps:

[0135] Step 101: performing word segmentation on the topic documents of different categories to obtain a word segmentation list corresponding to the topic documents;

[0136] For a topic document, it can be a text in a specific field, and the topic document can contain content describing the corresponding field. The topic document can be represented by the extracted words, for example, for a medical text, the words "diabetes" can be extracted from the medical text to represent that the medical text is related to diabetes.

[0137] In some possible implementation manners, after obtaining the subject documents of different categories, a corresponding domain dictionary can also be obtained, by loading the domain dictionary, the unregistered word recognition capability of the word segmenter is dynamically optimized, so that when the subject documents are segmented, the complete segmentation of the compound professional vocabulary can be ensured, to obtain an effective and accurate word segmentation list. The domain dictionary can be a dictionary composed of professional technical vocabulary corresponding to different technical fields, and the application does not limit this.

[0138] In some examples, for the processing procedure of the subject documents, the processing procedure can include text cleaning, word segmentation filtering, and domain dictionary expansion, etc., wherein the text cleaning can filter out the garbled codes and meaningless character sequences in the subject documents, the word segmentation filtering can filter out irrelevant words, and the domain dictionary expansion can ensure the complete segmentation of the compound professional vocabulary, etc. For example, the processing procedure of the subject documents can include:

[0139] 1. Text cleaning

[0140] First, the original subject documents of different categories are subjected to text cleaning, a regular expression matching and replacing mechanism is adopted to remove HTML (HyperText Markup Language) tags, URL (Uniform Resource Locator) links and non-text symbols (such as special characters such as @, #, etc.), and at the same time, a preset text pattern recognition algorithm is used to detect and filter garbled codes and meaningless character sequences.

[0141] 2. Word segmentation filtering

[0142] Subsequently, a multi-granularity word segmentation engine based on the Jieba word segmenter is used for word segmentation, a part-of-speech tagging is simultaneously performed and a part-of-speech filtering rule library is constructed, an extensible stop word list (containing 12 types of virtual words such as pronouns, prepositions and auxiliary words) and a dynamic part-of-speech whitelist (retaining nouns, verbs, adjectives and their derived part-of-speech) are configured, and a double filtering mechanism based on rules and statistical models is realized.

[0143] 3. Domain dictionary expansion

[0144] In view of the problem of insufficient domain adaptability, a domain dictionary expansion interface is introduced, a predefined professional term library is loaded, the unregistered word recognition capability of the word segmenter is dynamically optimized, and the complete segmentation of the compound professional vocabulary is ensured.

[0145] Finally, a structured word segmentation list corresponding to the subject documents can be output, such as [word segmentation 1, word segmentation 2,...], etc., wherein each word segmentation is subjected to semantic validity verification and morphological normalization processing, and provides high-quality basic corpus for subsequent feature extraction and semantic analysis.

[0146] In step 102, a sliding window is obtained for the word segmentation list, and word frequency statistics are performed on the word segmentation list according to the sliding window to obtain a global word frequency table corresponding to the word segmentation list.

[0147] After obtaining the corresponding word segmentation list through word segmentation, a sliding window for the word segmentation list can be further obtained, and word frequency statistics can be performed on the word segmentation list according to the sliding window to obtain a global word frequency table corresponding to the word segmentation list. Through the sliding window, the frequency of each word in the overall word segmentation list can be obtained.

[0148] The sliding window can be the extraction number corresponding to each word segmentation. Through the sliding window, a certain number of words in the word segmentation list can be extracted to form a corresponding candidate phrase in each word segmentation. Therefore, through the sliding window, the word segmentation can meet the needs of the classification model for continuous phrase classification and judgment, and more context semantic information can be obtained.

[0149] In some possible implementation manners, a window step corresponding to the sliding window can be obtained, and then the word segmentation list can be segmented according to the sliding window and the window step to obtain a corresponding candidate phrase, and the frequency of each candidate phrase in the theme document can be counted. Then, based on the candidate phrase and the corresponding frequency, a global word frequency table corresponding to the word segmentation list can be constructed. The candidate phrase can be a word group composed of a plurality of words, and the window step can be the distance of window movement in each word segmentation. For example, if the window step is 1, it means that after each word segmentation is completed, the sliding window moves one word according to the segmentation order.

[0150] In some examples, by using the n-gram dynamic sliding window for word frequency statistics, the statistical result can be more in line with the needs of the classification model for continuous phrase classification and judgment, and more context semantic information can be obtained from the theme document. In a specific implementation, the preprocessed word segmentation list is dynamically n-gram segmented, and the sliding window mechanism is used to traverse the content of the word segmentation list. For example, the window size is set to n (usually 2-3 according to actual conditions), the window step is 1, and continuous n word units are extracted to form a candidate phrase. In this way, the frequency of n-gram word groups in all theme documents can be counted through the sliding window to construct a global word frequency table. Therefore, through the n-gram dynamic sliding window segmentation, both local word sequence features and cross-window context associations can be captured, and the problem of semantic fragmentation caused by fixed segmentation can be avoided.

[0151] In step 103, feature statistics are performed on the global word frequency table to obtain corresponding sparse features.

[0152] After the corresponding candidate phrases are obtained by segmentation, feature statistics can be performed on the global word frequency table to obtain corresponding sparse features. Optionally, the total number of documents corresponding to the theme document can be obtained, and the number of documents containing the candidate phrase can be obtained. Then, a smoothing hyperparameter is obtained, and the total number of documents, the number of documents, and the smoothing hyperparameter are used for calculation to obtain the inverse document probability corresponding to each candidate phrase. Finally, the frequency corresponding to each candidate phrase and the inverse document probability are fused to obtain a corresponding sparse matrix.

[0153] wherein the total number of documents represents the number of all theme documents; and the number of documents represents the number of documents containing the candidate phrase, which can be obtained by searching for the corresponding candidate phrase in the theme document and counting the number of theme documents containing the candidate phrase. The present application does not limit this.

[0154] In some possible implementation manners, to alleviate the weight explosion problem of low-frequency words and avoid a zero denominator, the calculation formula can be modified during the calculation process of the inverse document probability, and a corresponding smoothing hyperparameter can be introduced. Optionally, the modified formula can be as follows:

[0155]

[0156] wherein N is the total number of theme documents, N t is the number of documents containing the term t, and the smoothing hyperparameters a and b are used to smooth the distribution of low-frequency words, so that the formula alleviates the statistical deviation caused by a small document size through the numerator term a, and suppresses the weight overflow of an unregistered word through the denominator term b.

[0157] After the frequency and the inverse document probability are counted, the term frequency (TF) and the improved IDF can be further combined to calculate the maximum weight of each term in the theme document:

[0158] TF-IDF(t, d) = TF(t, d) * IDF(t)

[0159] wherein the term frequency TF(t, d) is the standardized frequency, that is, the number of occurrences of the term t in the theme document d divided by the total number of words in the document. Finally, based on the calculated frequency and inverse document probability and the like, a corresponding sparse matrix is constructed, and the matrix dimension is M x K (M is the number of theme documents, and K is the total number of n-gram terms after deduplication). Optionally, the term is a candidate phrase, and the present application does not limit this.

[0160] In an example, it is assumed that:

[0161] Document 1: ["machine learning", "algorithm", "research"];

[0162] Document 2: ["deep learning", "algorithm", "application"].

[0163] After tokenization of document 1 and document 2, assuming that the size of the sliding window is 2 and the sliding step is 1, during the process of traversing the tokenization list, each time, 2 continuous word units can be extracted as candidate phrases, and the candidate phrases shown below can be output:

[0164] 2-gram of document 1: ["machine learning algorithm", "algorithm research"];

[0165] 2-gram of document 2: ["deep learning algorithm", "algorithm application"].

[0166] Then, the number of occurrences of each candidate phrase in all topic documents can be obtained, and then a corresponding global term frequency table can be constructed based on the candidate documents and the number of occurrences, as shown in Table 1 below:

[0167] n-gram occurrence "machine learning algorithm" 1 "algorithm research" 1 "deep learning algorithm" 1 "algorithm application" 1

[0168] Table 1

[0169] Then, assuming that N = 2, a = 1, and β = 0.5, IDF("algorithm research") can be calculated as log((2+1) / (1+0.5))≈0.301, and other candidate phrases can be calculated in the same way, so as to obtain the inverse document probability corresponding to each candidate phrase.

[0170] In addition, TF("algorithm research", document 1) can be calculated as 1 / 3≈0.333, and TF-IDF("algorithm research", document 1) can be calculated as 0.333x0.301≈0.1, so as to obtain the frequency corresponding to each candidate phrase.

[0171] Finally, based on the above calculation results, a corresponding sparse matrix can be constructed, as shown in Table 2 below:

[0172] document machine learning algorithm algorithm research deep learning algorithm algorithm application document 1 0.2 0.1 0 0 document 2 0 0 0.2 0.1

[0173] Table 2

[0174] Through the above process, on the one hand, the appearance frequency of n-gram word groups in all topic documents is counted through the sliding window to construct a global term frequency table, effectively capturing both local word sequence features and cross-window context associations, and avoiding the semantic fragmentation problem caused by fixed segmentation. On the other hand, by improving the calculation formula of TF-IDF, the weight explosion problem of low-frequency words is effectively alleviated, and the statistical deviation caused by small document size is alleviated.

[0175] In step 104, the sparse features are fused according to the global term frequency table to obtain a dense matrix.

[0176] After the corresponding sparse matrix is extracted, the sparse features can be further fused according to the global word frequency table to obtain a dense matrix, so as to fuse the global topic distribution features through the global word frequency table, enhance the semantic coherence and field theme distinguishability of the text features, and improve the sufficiency of the local word frequency features in capturing long-range semantic association.

[0177] In some possible implementation manners, the topic feature fusion can be performed by using the global word frequency table and the sparse features to obtain a corresponding topic enhanced weight matrix, and then word vector embedding processing can be performed by using the global word frequency table and the topic enhanced weight matrix to obtain a corresponding dense matrix, so as to fuse the global topic distribution features through the global word frequency table, enhance the semantic coherence and field theme distinguishability of the text features, improve the sufficiency of the local word frequency features in capturing long-range semantic association, and introduce word vector embedding, so that high-frequency and field-specific word items occupy a dominant position in the vector space.

[0178] For the fusion process of the topic features, a topic fusion model can be obtained, then the global word frequency table can be converted into a word item frequency vector, and the word item frequency vector can be input into the topic fusion model for probability mapping to obtain a document-level probability and a word item-level probability, then tensor point multiplication operation can be performed on the document-level probability, the word item-level probability and the sparse matrix to obtain a word item weight matrix, and normalization can be performed on the word item weight matrix to obtain a corresponding topic enhanced weight matrix, so that in the case of capturing short-range context information by using the improved TF-IDF (n-gram dynamic window), the global topic distribution of the document is further extracted by the topic model, the long-range semantic association missing problem is effectively solved, and the long-range semantic association is improved.

[0179] The document-level probability can be used to represent the association degree between the topic document and the topic, and the documents can be clustered by the probability distribution similarity; the word item-level probability can be used to represent the association degree between the word item and different topics, and the fusion of the topic distribution features on the sparse matrix by the document-level probability and the word item-level probability can enhance the semantic coherence and field theme distinguishability of the text representation, and solve the problem of insufficient capturing of long-range semantic association by the local word frequency features.

[0180] In some examples, the semantic coherence and field theme distinguishability of the text representation are enhanced by fusing the global topic distribution features, and the problem of insufficient capturing of long-range semantic association by the local word frequency features is solved. Specifically, the global word frequency table generated in the foregoing embodiments is converted into a bag-of-words format as the input of an LDA model (i.e., a topic fusion model), each document is represented as a word item frequency vector, and the co-occurrence mode of the n-gram word group is retained.

[0181] After the number of topics K is determined, for each topic document, the model outputs a double-layer probability mapping:

[0182] Document level: Document-topic probability matrix (i.e., document-level probability) P doc ∈R M×K , where M is the total number of documents, K is the number of topics, and each element P mk represents the probability that the mth document belongs to the kth topic.

[0183] Term level: Topic-term probability matrix (i.e., term-level probability) P word ∈R K×N , N is the total number of n-gram terms, and element P kn represents the generation probability of the nth term in the kth topic.

[0184] Then, the double-layer probability distribution matrix can be tensor-point multiplied with the TF-IDF matrix F tfidf ∈R M×N (i.e., a sparse matrix) to generate a topic-weighted term weight matrix:

[0185] F fusion =P doc *P word ⊙F tfidf

[0186] where ⊙ represents Hadamard product (element-wise multiplication), and through this operation, global topic weights can be injected into local term features. Finally, the term weight matrix F fusion is normalized to eliminate the influence of document length differences on feature scales, thereby obtaining the corresponding topic-enhanced weight matrix. Thus, in the case of capturing short-range context information using the improved TF-IDF (n-gram dynamic window), the global topic distribution of the document is further extracted through the topic model, effectively solving the problem of missing long-range semantic associations and improving the relevance of long-range semantics.

[0187] For the process of word vector embedding, the word vector model and the corresponding vocabulary of the word vector model can be obtained. Then, the candidate phrases that exist in the global word frequency table and are in the vocabulary are taken as the first candidate phrases, the first word vector corresponding to the first candidate phrases is obtained from the vocabulary, the candidate phrases that do not exist in the global word frequency table and are in the vocabulary are taken as the second candidate phrases, the second candidate phrases are split into subwords, and the second word vectors corresponding to the subwords are obtained from the vocabulary. Then, the third word vector corresponding to the second candidate phrases is obtained through weighted average processing based on the second word vectors corresponding to the subwords. Finally, the first word vector, the third word vector, and the topic-enhanced weight matrix are subjected to word vector embedding processing to obtain the corresponding dense matrix. Thus, by element-wise fusion of the weight and the word vector, high-frequency words and domain keywords occupy a dominant position in the vector space.

[0188] In some examples, a pre-trained Word2Vec word vector model is employed as a semantic representation basis.

[0189] For the characteristics of n-gram phrases, for each n-gram phrase, its word vector is generated in the following way: if the candidate phrase exists in the vocabulary of the pre-trained model, its corresponding word vector is directly loaded; if the candidate phrase is an out-of-vocabulary word, the candidate phrase is disassembled into multiple subwords, the corresponding word vectors of each subword are loaded from the vocabulary, and then the vectors of each independent subword in the candidate phrase are dimensionally weighted and averaged. Then, the dense matrix F fusion Deeply fused with the word vector, for the mth word vector t j in the document d m , its F fusion weight w jm and word vector v m are extracted, and element-wise scalar multiplication is performed to generate a semantic enhanced vector:

[0190] e jm = w jm * v m

[0191] Through the above process, the statistical weight can be coupled with the semantic space to form a dense matrix with dimensions (number of topic documents x n-gram word x embedding dimension), so that high-frequency and domain-specific words occupy a dominant position in the vector space.

[0192] In step 105, a domain dictionary corresponding to the topic document is obtained, and the dense matrix is weighted according to the domain dictionary to obtain a corresponding hybrid feature matrix.

[0193] After obtaining the dense matrix, the domain dictionary corresponding to the topic document can be combined, and then the dense matrix is weighted according to the domain dictionary. The deep fusion of semantic features and statistical features is realized through domain knowledge guided feature enhancement, and data dimension compression is performed to save computing power cost and computational complexity, so as to obtain a corresponding hybrid feature matrix.

[0194] In some feasible implementations, a list of domain keywords can be extracted from the domain dictionary first, then the candidate phrases in the global word frequency table are matched with the list of domain keywords, and the word items corresponding to the candidate phrases in the dense matrix are weighted according to the matching results to obtain an enhanced weight matrix, and then the enhanced weight matrix is dimensionally reduced to obtain a corresponding hybrid feature matrix, so as to realize the deep fusion of semantic features and statistical features through domain knowledge guided feature enhancement, and enhance the domain knowledge features.

[0195] Wherein, for the process of weight processing, the candidate phrase existing in the field keyword list in the global word frequency table can be taken as the third candidate phrase, the first word item corresponding to the third candidate phrase in the dense matrix is subjected to weight increasing processing, and the candidate phrase not existing in the field keyword list in the global word frequency table is taken as the fourth candidate phrase, the second word item corresponding to the fourth candidate phrase in the dense matrix is subjected to smoothing calibration processing, then based on the weight increasing processing and / or the smoothing calibration processing performed on the dense matrix, an enhanced weight matrix is obtained, so as to dynamically amplify the keyword weight based on the predefined field dictionary, and prevent overfitting in combination with the adaptive smoothing factor, and ensure the significance of the professional terms.

[0196] For example, the keyword list can be loaded from the field dictionary first, and the hash index structure is constructed to realize the word item query with O(1) time complexity. For each n-gram word item in the document, if it exists in the field dictionary or the word group composed of the continuous n word items exists in the field dictionary, the weight enhancement function is activated: the weight w of the word item is increased by γ, that is, w b is dynamically multiplied by the field enhancement coefficient γ, that is, W f = W b ×(1+γ).

[0197] Wherein, γ is an adjustable hyperparameter, which is optimized in the interval [0.5, 2.0] (or other intervals) through grid search. The operation improves the semantic density of the field keyword in the vector space, so that the vector module length of the professional term is significantly higher than that of the general vocabulary. Meanwhile, in order to avoid feature deviation caused by excessive enhancement, an adaptive smoothing factor is introduced to dynamically calibrate the enhanced weight:

[0198] λ=1 / (1+logN fre )

[0199] Wherein, N fre is the occurrence frequency of the word item in the field dictionary. Based on the above process, the enhanced weight matrix with field differentiation can be finally generated.

[0200] Further, after obtaining the enhanced weight matrix, principal component space projection dimension reduction can be performed to compress the data dimension and save the computing cost and complexity. Specifically, the enhanced weight matrix can be normalized to obtain a normalized matrix, then a covariance matrix corresponding to the normalized matrix is calculated, and then singular value decomposition is performed on the covariance matrix to obtain corresponding eigenvalues, and the cumulative contribution rate corresponding to each eigenvalue is obtained, and the eigenvalues with a cumulative contribution rate greater than or equal to a first preset threshold are selected as target eigenvectors, a projection matrix is constructed based on each target eigenvector, and finally the projection matrix is projected into a low-dimensional orthogonal space to obtain a corresponding mixed feature matrix, so that the high-dimensional coefficient feature is compressed into a low-dimensional orthogonal space by retaining key principal components, thereby significantly reducing the computational complexity while retaining most of the effective information.

[0201] For example, the enhanced weight matrix can be normalized again, and PCA dimension reduction can be performed on the n-gram word number dimension according to the actual computing power. Specifically, a covariance matrix can be constructed, principal component directions with a cumulative contribution rate greater than or equal to 85% or 92% are selected, a feature subspace is obtained by singular value decomposition, and the original features are projected into a low-dimensional orthogonal space, and the output dimension is reduced to (number of topic documents x k x embedding dimension), where k is the target dimension (usually 128-1024 dimensions), so that a mixed feature matrix fused with multi-dimensional value features can be obtained, so as to input the mixed feature matrix into a classifier and train the classifier.

[0202] In some examples, the PCA dimension reduction process can include: ① data flattening: converting a 3D matrix into a 2D matrix: [number of documents x (number of features x embedding dimension)], to adapt to the 2D input requirement of PCA, for example, converting 1000 x 5000 x 300 into a 1000 x 1,500,000 matrix; ② the corresponding covariance matrix can be calculated by the following formula:

[0203]

[0204] where N can be the total number of documents, and D can be the total number of features (dimension after flattening).

[0205] ③ After obtaining the covariance matrix, eigenvalue decomposition can be performed by the following formula:

[0206] Cov = U∑V T

[0207] where U is the left singular vector (i.e., the principal component direction), ∑V T is a diagonal matrix (used for eigenvalue descending arrangement), V is the right singular vector, and T is the matrix transpose operation (i.e., row and column interchanging).

[0208] IV. Principal component screening, the corresponding cumulative contribution rate can be calculated by the following formula:

[0209] Cumulative contribution rate

[0210] Wherein, σ i The i-th largest eigenvalue can be calculated based on the above formula to determine the target dimension k by selecting the minimum k such that the cumulative contribution rate is greater than or equal to a predetermined threshold (such as 85% or the like). For example, if the contribution rate of the first 512 dimensions is 90%, then k = 512, and the like.

[0211] V. High-dimensional features can be projected into a low-dimensional space by the following formula:

[0212] X PCA = X·V k (V k ∈R D×k )

[0213] Wherein, V k The projection matrix composed of the first k eigenvectors can be used to realize dimension reduction and output the corresponding mixed feature matrix, so as to retain the key principal components by PCA, compress high-dimensional sparse features into low-dimensional orthogonal space, significantly reduce the calculation complexity, and retain most of the effective information.

[0214] Step 106, training the initial classifier according to the mixed feature matrix to obtain the trained target classifier.

[0215] In the embodiment of the present application, after obtaining the corresponding mixed features, the evaluation data set can be further obtained, and then the mixed feature matrix is input into the initial classifier to be trained to obtain the prediction result, and the evaluation data set is input into the initial classifier to obtain the evaluation result. Then, the prediction result is compared with the evaluation result to obtain the corresponding error value, and the initial classifier is iterated based on the error value until the error value is less than or equal to a second predetermined threshold, and the iteration is stopped to obtain the trained target classifier. Thus, by optimizing the feature extraction process, not only the local word frequency features are extracted, but also the theme features are integrated for weighted processing, effectively reducing the dependence on manual labeling. Through multi-dimensional feature engineering, low-value text can be converted into a mixed feature matrix that can be used for model training, forming an end-to-end cold start solution, effectively reducing noise interference in the model training process, shortening the training period, and improving the effect of model training.

[0216] It should be noted that the embodiments of the present application include but are not limited to the above examples. It can be understood that those skilled in the art can set it according to the actual needs under the guidance of the idea of the embodiments of the present application, and the present application does not limit it.

[0217] In the embodiment of the present application, by segmenting the topic documents of different categories, the segmentation list corresponding to the topic documents is obtained, and the sliding window for the segmentation list is obtained, and the word frequency of the segmentation list is counted according to the sliding window, and the global word frequency table corresponding to the segmentation list is obtained, then the feature statistics of the global word frequency table is performed, and the corresponding sparse feature is obtained, then the sparse feature is fused according to the global word frequency table, and the dense matrix is obtained, then the domain dictionary corresponding to the topic document is obtained, and the dense matrix is weighted according to the domain dictionary, and the corresponding hybrid feature matrix is obtained, and finally the initial classifier to be trained is trained according to the hybrid feature matrix, and the trained target classifier is obtained, so that the feature extraction process is optimized, not only the local word frequency feature is extracted, but also the topic feature is fused and weighted, the dependence on manual annotation is effectively reduced, the low-value text can be converted into a hybrid feature matrix that can be used for model training through multi-dimensional feature engineering, an end-to-end cold start solution is formed, noise interference in the model training process is effectively reduced, the training period is shortened, and the effect of model training is improved.

[0218] In order to make the technical personnel in the art better understand the technical solutions in the embodiments of the present application, the following will be exemplarily described through corresponding examples:

[0219] As an example, referring to Figure 2 , a processing flow diagram of the classifier cold start provided in the embodiment of the present application is shown, which can specifically include the following processes:

[0220] Step 1: data preprocessing and segmentation

[0221] In this process, noise removal and key information retention can be achieved through a multi-step text processing flow, and the specific implementation method is as follows:

[0222] Firstly, the original topic documents of various categories are subjected to text cleaning, and the regular expression matching and replacement mechanism is adopted to remove HTML tags, URL links and non-text symbols (such as @, # and other special characters), and at the same time, the preset text pattern recognition algorithm is used to detect and filter garbled characters and meaningless character sequences.

[0223] Subsequently, a multi-granularity segmentation engine based on Jieba segmenter is used for word segmentation, and part-of-speech tagging is performed synchronously and a part-of-speech filtering rule library is constructed, and through the configuration of an extensible stop word list (containing 12 types of virtual words such as pronouns, prepositions and auxiliary words) and a dynamic part-of-speech whitelist (retaining nouns, verbs, adjectives and their derived part-of-speech), a double filtering mechanism based on rules and statistical models is realized.

[0224] To solve the problem of insufficient domain adaptability, a domain dictionary extension interface is introduced to dynamically optimize the recognition ability of the word segmenter by loading a predefined professional term library, ensuring the complete segmentation of complex professional terms.

[0225] The final output is structured as a standardized segmentation list of [segment1, segment2,...], where each segment has been verified for semantic validity and normalized in form, providing high-quality basic corpus for subsequent feature extraction and semantic analysis.

[0226] Step 2: Implementation of TF-IDF basic weight

[0227] 2.1 n-gram dynamic sliding window

[0228] To better meet the needs of the classification model for continuous phrase classification and to obtain more context semantic information, this method uses n-gram dynamic sliding window for word frequency statistics. The preprocessed segmentation list is dynamically divided into n-grams, and a sliding window mechanism is used to traverse the segmentation list content. The window size is set to n (usually 2-3 according to actual conditions), and the window step is 1. The continuous n-gram is extracted to form a candidate phrase. The frequency of n-gram in all topic documents is counted by sliding window to build a global word frequency table. This method can capture both local word sequence features and cross-window context associations, avoiding the semantic fragmentation problem caused by fixed segmentation in traditional n-gram methods.

[0229] 2.2 Improved IDF smoothing calculation

[0230] To alleviate the weight explosion problem of low-frequency words and avoid zero denominator, a modified formula is used for smoothing:

[0231]

[0232] where N is the total number of topic documents, Nt is the number of documents containing term t, and hyperparameters α and β are used to smooth the low-frequency word distribution. This formula alleviates the statistical bias caused by small document size through the numerator term α, and suppresses the weight overflow of out-of-vocabulary words through the denominator term β.

[0233] 2.2 TF-IDF weight fusion and matrix generation

[0234] Combine the term frequency (TF) with the improved IDF to calculate the maximum weight of each term in the topic document:

[0235] TF-IDF(t, d) = TF(t, d) * IDF(t)

[0236] where the term frequency TF(t, d) adopts the normalized frequency, i.e., the number of occurrences of the term t in the document d divided by the total number of words in the document. Finally, a sparse matrix is constructed, with the matrix dimension being M x K (M is the number of topic documents, and K is the total number of n-gram terms after deduplication).

[0237] Step 3: LDA topic model fusion

[0238] By fusing the global topic distribution feature, the semantic coherence and domain topic distinguishability of the text representation are enhanced, and the problem of insufficient capture of long-range semantic association by local term frequency features is solved.

[0239] Based on the global term frequency table generated in step 2, it is converted into a bag-of-words format as the input of the LDA model. Each document is represented as a term frequency vector, preserving the co-occurrence pattern of n-gram terms. The number of topics K is determined, and the model is trained.

[0240] For each topic document, the model outputs a double-layer probability mapping:

[0241] Document level: document-topic probability matrix P doc ∈R M×K , where M is the total number of documents, K is the number of topics, and each element P mk represents the probability that the mth document belongs to the kth topic.

[0242] Term level: topic-term probability matrix P word ∈R K×N , N is the total number of n-gram terms, and element P kn represents the generation probability of the nth term in the kth topic.

[0243] The double-layer probability distribution matrix is multiplied by the TF-IDF matrix F tfidf ∈R M×N to generate a topic-weighted term weight matrix:

[0244] F fusion = P doc * P word ⊙ F tfidf

[0245] where ⊙ represents Hadamard product (element-wise multiplication), which injects global topic weights into local term features.

[0246] Finally, F fusion is normalized to eliminate the influence of document length difference on feature scale.

[0247] Step 4: word embedding weighting

[0248] Pre-trained Word2Vec word vector model is used as the semantic representation basis. For the characteristics of n-gram, for each n-gram, its word vector is generated in the following way:

[0249] 1. If the word group exists as a whole in the word table of the pre-trained model, directly load its corresponding word vector.

[0250] 2. If it is an out-of-vocabulary word, it is disassembled into subwords, and the vectors of each independent subword in the word group are dimensionally weighted and averaged.

[0251] Then F fusion is generated in step 3 j is deeply fused with the word vector. For the mth word vector t m in the document d fusion , its F jm weight w m and word vector v jm are extracted, and element-by-element scalar multiplication is performed to generate a semantic enhanced vector:

[0252] e jm = w m *v n

[0253] This operation couples the statistical weight with the semantic space to form a dense matrix with dimensions (number of topic documents x n-gram word x embedding dimension), so that high-frequency and domain-specific words occupy a dominant position in the vector space.

[0254] Step 5: Domain knowledge weighting and dimensionality reduction

[0255] This step further realizes the deep fusion of semantic features and statistical features through domain knowledge guided feature enhancement, and performs data dimensionality compression to save computing power cost and computational complexity.

[0256] 5.1 Domain knowledge dynamic weighting mechanism

[0257] The knowledge enhancement module is established based on the domain dictionary constructed in step 1. The system loads the keyword list from the domain dictionary and constructs a hash index structure to realize O(1) time complexity of word query. For each n-gram word in the document, if it exists in the domain dictionary or the word group composed of consecutive n words exists in the domain dictionary, the weight enhancement function is activated: the weight w n of the word is dynamically multiplied by the domain enhancement coefficient γ, i.e. W f = W b ×(1+γ). Where γ is an adjustable hyperparameter, which is optimized in the interval [0.5, 2.0] through grid search. This operation increases the semantic density of domain keywords in the vector space, making the vector norm of professional terms significantly higher than that of general words.

[0258] At the same time, in order to avoid feature deviation caused by excessive enhancement, an adaptive smoothing factor is introduced to dynamically calibrate the enhanced weight:

[0259] λ = 1 / (1 + logN fre )

[0260] Where N fre is the frequency of the term in the domain dictionary. Finally, an enhanced weight matrix with domain discriminability is generated.

[0261] 5.2 Principal component space projection dimension reduction

[0262] Based on the normalization of the mixed feature matrix again, according to the actual situation of computing power, PCA dimension reduction is performed on the n-gram word number dimension, by constructing a covariance matrix, selecting the principal component direction with cumulative contribution rate ≥85% or 92%, and through singular value decomposition, the original feature is projected into a low-dimensional orthogonal space, and the output dimension is reduced to (theme document number × k × embedding dimension), where k is the target dimension (usually 128-1024 dimensions).

[0263] From this, a mixed feature matrix that integrates multi-dimensional value features can be obtained, which can be directly input into the classifier.

[0264] Step 6: Model training and effectiveness

[0265] Collect and construct 8 low-value discrete theme document data, about 10w words per theme document, form a mixed feature matrix (dimension reduction: 512, embedding dimension: 1024) after the above steps, and train different classifiers, and construct 200 evaluation data sets for evaluation, stop when validloss tends to be stable, and get the trained classifier.

[0266] In the above process, first, the original data is deeply cleaned and segmented, then the local semantic features are extracted by the improved TF-IDF algorithm, and the global theme distribution information is captured by combining the LDA theme model. On this basis, the semantic space features are fused by word embedding weighting, and a dynamic weighting mechanism is introduced to enhance the discriminability of professional terms. Finally, PCA dimension reduction is used to realize the concentration and optimization of high-dimensional features. This framework does not need to rely on labeled data, but directly converts low-value text into trainable mixed feature matrix through multi-step feature engineering, forming an end-to-end cold start solution.

[0267] Further, the potential value of low correlation text is fully mined through a multi-dimensional feature fusion mechanism. Specifically, it includes: 1. Local and global feature combination: short-range context information is captured by using an improved TF-IDF (n-gram dynamic window), and the global theme distribution of the document is extracted by an LDA theme model to solve the problem of missing long-range semantic association. 2. Coupling of statistical weight and semantic space: the weight and the Word2Vec word vector are fused element by element, so that high-frequency words and domain keywords occupy a dominant position in the vector space. 3. Feature enhancement guided by domain knowledge: the weight of the keyword is dynamically amplified based on a pre-defined domain dictionary, and an adaptive smoothing factor is combined to prevent overfitting, ensuring the prominence of professional terms. 4. Efficient dimension reduction and information condensation: the high-dimensional sparse features are compressed to a low-dimensional orthogonal space by PCA, which significantly reduces the computational complexity while retaining most of the effective information.

[0268] It should be noted that for the method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited by the order of the described actions, because according to the embodiments of the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily necessary for the embodiments of the present application.

[0269] Referring to Figure 3 , a structural block diagram of a model training device provided in an embodiment of the present application is shown, which can specifically include the following modules:

[0270] The word segmentation module 301 is configured to perform word segmentation on the topic documents of different categories to obtain a word segmentation list corresponding to the topic documents.

[0271] The word frequency statistics module 302 is configured to obtain a sliding window for the word segmentation list, and perform word frequency statistics on the word segmentation list according to the sliding window to obtain a global word frequency table corresponding to the word segmentation list.

[0272] The feature statistics module 303 is configured to perform feature statistics on the global word frequency table to obtain corresponding sparse features.

[0273] The feature fusion module 304 is configured to perform feature fusion on the sparse features according to the global word frequency table to obtain a dense matrix.

[0274] The weighting module 305 is configured to obtain a domain dictionary corresponding to the topic documents, and perform weighting processing on the dense matrix according to the domain dictionary to obtain a corresponding hybrid feature matrix.

[0275] The training module 306 is configured to train an initial classifier to be trained according to the mixed feature matrix to obtain a trained target classifier.

[0276] In some possible implementation manners, the word frequency statistics module 302 is specifically configured to:

[0277] obtain a window step length corresponding to the sliding window;

[0278] divide the word segmentation list according to the sliding window and the window step length to obtain corresponding candidate phrases, and count frequencies of the candidate phrases in the theme document;

[0279] construct a global word frequency table corresponding to the word segmentation list based on the candidate phrases and the corresponding frequencies.

[0280] In some possible implementation manners, the word frequency statistics module 302 is specifically configured to:

[0281] obtain a total number of documents corresponding to the theme document;

[0282] obtain a number of documents containing the candidate phrases;

[0283] obtain a smoothing hyperparameter, and calculate the inverse document probability corresponding to each candidate phrase by using the total number of documents, the number of documents, and the smoothing hyperparameter;

[0284] perform feature fusion on the frequency and the inverse document probability corresponding to each candidate phrase to obtain a corresponding sparse matrix.

[0285] In some possible implementation manners, the feature fusion module 304 is specifically configured to:

[0286] perform theme feature fusion on the global word frequency table and the sparse feature to obtain a corresponding theme enhanced weight matrix;

[0287] perform word vector embedding processing on the global word frequency table and the theme enhanced weight matrix to obtain a corresponding dense matrix.

[0288] In some possible implementation manners, the feature fusion module 304 is specifically configured to:

[0289] obtain a theme fusion model;

[0290] convert the global word frequency table into a word item frequency vector;

[0291] input the word item frequency vector into the theme fusion model to perform probability mapping to obtain a document-level probability and a word item-level probability;

[0292] The document-level probability, the word-level probability, and the sparse matrix are subjected to a tensor point product operation to obtain a word weight matrix, and the word weight matrix is normalized to obtain a corresponding topic-enhanced weight matrix.

[0293] In some possible implementation manners, the feature fusion module 304 is specifically configured to:

[0294] obtain a word vector model and a word list corresponding to the word vector model;

[0295] obtain, as a first candidate phrase, a candidate phrase that exists in the word list in the global word frequency table, obtain a first word vector corresponding to the first candidate phrase from the word list;

[0296] obtain, as a second candidate phrase, a candidate phrase that does not exist in the word list in the global word frequency table, and split the second candidate phrase into subwords;

[0297] obtain a second word vector corresponding to each of the subwords from the word list;

[0298] perform weighted average processing based on the second word vector corresponding to each of the subwords to obtain a third word vector corresponding to the second candidate phrase;

[0299] perform word vector embedding processing on the first word vector, the third word vector, and the topic-enhanced weight matrix to obtain a corresponding dense matrix.

[0300] In some possible implementation manners, the weighting module 305 is specifically configured to:

[0301] extract a domain keyword list from the domain dictionary;

[0302] match a candidate phrase in a global word frequency table with the domain keyword list, and perform weighting processing on a word corresponding to the candidate phrase in the dense matrix according to a matching result to obtain an enhanced weight matrix;

[0303] perform dimension reduction processing on the enhanced weight matrix to obtain a corresponding hybrid feature matrix.

[0304] In some possible implementation manners, the weighting module 305 is specifically configured to:

[0305] obtain, as a third candidate phrase, a candidate phrase that exists in the domain keyword list in the global word frequency table, and perform weight increasing processing on a first word corresponding to the third candidate phrase in the dense matrix;

[0306] a candidate phrase not existing in the domain keyword list in the global word frequency table is taken as a fourth candidate phrase, and a second word item corresponding to the fourth candidate phrase in the dense matrix is subjected to smoothing calibration processing;

[0307] Based on the weight increasing processing and / or the smoothing calibration processing performed on the dense matrix, an enhanced weight matrix is obtained.

[0308] In some feasible implementation manners, the weighting module 305 is specifically configured to:

[0309] The enhanced weight matrix is normalized to obtain a normalized matrix;

[0310] A covariance matrix corresponding to the normalized matrix is calculated;

[0311] The covariance matrix is subjected to singular value decomposition to obtain corresponding eigenvalues;

[0312] Cumulative contribution rates corresponding to each of the eigenvalues are obtained, and eigenvalues with cumulative contribution rates greater than or equal to a first preset threshold are selected as target eigenvectors, and a projection matrix is constructed based on each of the target eigenvectors;

[0313] The projection matrix is projected into a low-dimensional orthogonal space to obtain a corresponding mixed feature matrix.

[0314] In some feasible implementation manners, the training module 306 is specifically configured to:

[0315] An evaluation data set is obtained;

[0316] The mixed feature matrix is input into an initial classifier to be trained to obtain a prediction result, and the evaluation data set is input into the initial classifier to obtain an evaluation result;

[0317] The prediction result is compared with the evaluation result to obtain a corresponding error value;

[0318] The initial classifier is iterated based on the error value until the error value is less than or equal to a second preset threshold, and the iteration is stopped to obtain a target classifier after training.

[0319] In some feasible implementation manners, the word segmentation module 301 is specifically configured to:

[0320] Different categories of theme documents are obtained;

[0321] The theme documents are subjected to word segmentation according to a domain dictionary to obtain a corresponding word segmentation list.

[0322] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts refer to the part of the method embodiment.

[0323] In addition, the embodiment of the present application also provides an electronic device, comprising: a processor, a memory, a computer program stored on the memory and executable on the processor, which realizes each process of the method embodiment of the model training method when executed by the processor, and can achieve the same technical effect, to avoid repetition, which will not be repeated here.

[0324] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program realizes each process of the method embodiment of the model training method when executed by the processor, and can achieve the same technical effect, to avoid repetition, which will not be repeated here. The computer readable storage medium includes read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0325] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts of each embodiment can be referred to each other.

[0326] Those skilled in the art should understand that the embodiments of the embodiment of the present application can be provided as a method, device or computer program product. Therefore, the embodiment of the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiment of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk memory, CD-ROM, optical memory, EEPROM, Flash and eMMC, etc.) containing computer usable program code.

[0327] The embodiments of the present application are described with reference to flowcharts and / or block diagrams according to the method, terminal device (system) and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce a machine that realizes the functions described in the flowcharts and / or block diagrams. Figure 1 one flow or multiple flows and / or blocks Figure 1means for performing the function specified by that block or blocks.

[0328] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow Figure 1 flow or flows and / or blocks Figure 1 means for performing the function specified by that block or blocks.

[0329] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 flow or flows and / or blocks Figure 1 means for performing the function specified by that block or blocks.

[0330] While preferred embodiments of the application have been described, modifications and variations can be apparent to those skilled in the art once aware of the general underlying concepts. Accordingly, the appended claims are intended to embrace all such modifications and variations as fall within the scope of the application.

[0331] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to nomenclature different components to distinguish one component from another component and do not necessarily imply or require any such actual relationship or order. Also, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element. The terms "an embodiment", "one embodiment", or similar phrases as used herein do not denote the same embodiment or indicate that a certain feature is essential to the present application, but rather that an alternative embodiment might include the feature.

[0332] The training method of a model and the training device of a model are described in detail above, and the principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for training a model, characterized in that, include: Segment the topic documents of different categories to obtain the word segmentation list corresponding to the topic documents; Obtain a sliding window for the word segmentation list, and perform word frequency statistics on the word segmentation list according to the sliding window to obtain the global word frequency table corresponding to the word segmentation list; Perform feature statistics on the global word frequency table to obtain the corresponding sparse features; The sparse features are fused according to the global term frequency table to obtain a dense matrix; Obtain the domain dictionary corresponding to the topic document, and perform weighted processing on the dense matrix according to the domain dictionary to obtain the corresponding hybrid feature matrix; The initial classifier to be trained is trained based on the hybrid feature matrix to obtain the trained target classifier.

2. The method according to claim 1, characterized in that, The step of performing word frequency statistics on the word segmentation list according to the sliding window to obtain the global word frequency table corresponding to the word segmentation list includes: Obtain the window step size corresponding to the sliding window; The word segmentation list is segmented according to the sliding window and the window step size to obtain the corresponding candidate phrases, and the frequency of each candidate phrase in the topic document is counted. Based on the candidate phrases and their corresponding frequencies, a global word frequency table corresponding to the word segmentation list is constructed.

3. The method according to claim 2, characterized in that, The step of performing feature statistics on the global word frequency table to obtain the corresponding sparse features includes: Get the total number of documents corresponding to the topic document; Obtain the number of documents containing the candidate phrase; Obtain the smoothing hyperparameter, and calculate the inverse document probability corresponding to each candidate phrase using the total number of documents, the number of documents, and the smoothing hyperparameter; The frequency and inverse document probability of each candidate phrase are used to perform feature fusion to obtain the corresponding sparse matrix.

4. The method according to any one of claims 1 to 3, characterized in that, The step of fusing the sparse features based on the global word frequency table to obtain a dense matrix includes: The global word frequency table and the sparse features are used to fuse topic features to obtain the corresponding topic enhancement weight matrix; The global word frequency table and the topic enhancement weight matrix are used to perform word vector embedding processing to obtain the corresponding dense matrix.

5. The method according to claim 4, characterized in that, The step of fusing topic features using the global term frequency table and the sparse features to obtain the corresponding topic enhancement weight matrix includes: Obtain the topic fusion model; Convert the global term frequency table into a term frequency vector; The term frequency vector is input into the topic fusion model for probability mapping to obtain document-level probability and term-level probability. The document-level probability, the term-level probability, and the sparse matrix are subjected to tensor dot product operation to obtain the term weight matrix. The term weight matrix is ​​then normalized to obtain the corresponding topic enhancement weight matrix.

6. The method according to claim 4, characterized in that, The step of using the global word frequency table and the topic enhancement weight matrix to perform word vector embedding processing to obtain the corresponding dense matrix includes: Obtain the word vector model and the vocabulary corresponding to the word vector model; The candidate phrases that are present in the entire global word frequency table are taken as the first candidate phrases, and the first word vectors corresponding to the first candidate phrases are obtained from the word table. Candidate phrases that do not exist in the global word frequency table are taken as second candidate phrases, and the second candidate phrases are split into sub-words; Obtain the second word vector corresponding to each of the sub-words from the vocabulary list; A weighted average is performed on the second word vectors corresponding to each of the sub-words to obtain the third word vector corresponding to the second candidate phrase. The first word vector, the third word vector, and the topic enhancement weight matrix are subjected to word vector embedding processing to obtain the corresponding dense matrix.

7. The method according to claim 1, characterized in that, The step of weighting the dense matrix according to the domain dictionary to obtain the corresponding mixed feature matrix includes: Extract a list of domain keywords from the domain dictionary; Candidate phrases in the global word frequency table are matched with the domain keyword list, and the terms corresponding to the candidate phrases in the dense matrix are weighted according to the matching results to obtain an enhanced weight matrix. The enhanced weight matrix is ​​then subjected to dimensionality reduction to obtain the corresponding hybrid feature matrix.

8. The method according to claim 7, characterized in that, The step of weighting the terms corresponding to candidate phrases in the dense matrix based on the matching results to obtain an enhanced weight matrix includes: The candidate phrases that exist in the domain keyword list in the global word frequency table are used as the third candidate phrases, and the weight of the first term corresponding to the third candidate phrase in the dense matrix is ​​increased. The candidate phrases that do not exist in the domain keyword list in the global word frequency table are taken as the fourth candidate phrases, and the second word items corresponding to the fourth candidate phrases in the dense matrix are subjected to smoothing calibration. An enhanced weight matrix is ​​obtained by performing weight augmentation and / or smoothing calibration on the dense matrix.

9. The method according to claim 7 or 8, characterized in that, The step of reducing the dimensionality of the enhanced weight matrix to obtain the corresponding hybrid feature matrix includes: The enhanced weight matrix is ​​normalized to obtain a normalized matrix; Calculate the covariance matrix corresponding to the normalized matrix; Singular value decomposition is performed on the covariance matrix to obtain the corresponding eigenvalues; Obtain the cumulative contribution rate corresponding to each of the aforementioned feature values, and select the feature values ​​whose cumulative contribution rate is greater than or equal to a first preset threshold as target feature vectors, and construct a projection matrix based on each of the aforementioned target feature vectors; The projection matrix is ​​projected into a low-dimensional orthogonal space to obtain the corresponding hybrid feature matrix.

10. The method according to claim 1, characterized in that, The step of training the initial classifier to be trained based on the mixed feature matrix to obtain the trained target classifier includes: Obtain the assessment dataset; The mixed feature matrix is ​​input into the initial classifier to be trained to obtain the prediction result, and the evaluation dataset is input into the initial classifier to train to obtain the evaluation result. The prediction results are compared with the evaluation results to obtain the corresponding error values; The initial classifier is iterated based on the error value until the error value is less than or equal to a second preset threshold, at which point the iteration stops and the trained target classifier is obtained.

11. The method according to claim 1, characterized in that, The step of segmenting different categories of topic documents to obtain a word segmentation list corresponding to the topic document includes: Obtain documents on different categories of topics; The subject document is segmented into words according to the domain dictionary to obtain the corresponding word segmentation list.

12. A training device for a model, characterized in that, include: The word segmentation module is used to segment different categories of topic documents into words and obtain a word segmentation list corresponding to the topic document; The word frequency statistics module is used to obtain a sliding window for the word segmentation list, and to perform word frequency statistics on the word segmentation list according to the sliding window to obtain the global word frequency table corresponding to the word segmentation list; The feature statistics module is used to perform feature statistics on the global word frequency table to obtain the corresponding sparse features; The feature fusion module is used to perform feature fusion on the sparse features according to the global word frequency table to obtain a dense matrix; The weighting module is used to obtain the domain dictionary corresponding to the topic document, and to perform weighting processing on the dense matrix according to the domain dictionary to obtain the corresponding hybrid feature matrix; The training module is used to train the initial classifier to be trained based on the hybrid feature matrix to obtain the trained target classifier.

13. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-11.

14. A computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-11.

Images