A data source automatic expansion method based on adaptive knowledge distillation

Abstract

The application provides a data source automatic expansion method based on adaptive knowledge distillation, uses general crawler technology to capture webpage text information of known categories, carries out pretreatment on the webpage text information, converts the webpage text information into a corresponding word list, takes the word list and the categories thereof as training and test data, and constructs training and test data sets; an Attention-BiLSTM is constructed as a data source classification model, model compression is realized through an adaptive knowledge distillation algorithm, and the convergence speed and recognition ability of the model are improved; the collected text information under unknown categories of data sources is pretreated, the constructed model is input to predict corresponding categories, according to the classification results, the urls of the data sources and the corresponding labels are recorded and stored, and automatic accumulation and expansion of the data sources are realized. The application can cooperate with general crawler technology to automatically expand various data sources.

Description

Technical Field

[0001] This invention relates to the field of network information processing, and specifically to a method for automatically expanding data sources based on adaptive knowledge distillation. Background Technology

[0002] With the rapid development and popularization of internet technology, the amount of online information is growing exponentially, bringing opportunities for comprehensive data collection across various fields. Effective website topic classification can automatically sort out the categories of massive websites based on website text information, thereby accumulating and expanding various data sources from small to large. Therefore, this invention combines natural language processing and web crawling technology to extensively collect and classify text information from multiple websites, expanding data sources while reducing manual intervention, and enabling more efficient organization of internet data in various fields. Unlike other application scenarios, data collection programs consume a large amount of computing and memory resources for a long time. Therefore, the classification model needs to be lightweight and efficient. Knowledge distillation can transfer knowledge from complex, large-scale models to lightweight models, achieving model compression. Existing knowledge distillation methods mostly involve manually setting cross-entropy loss weights, which are highly subjective, have limited flexibility, and are difficult to effectively balance the convergence speed and recognition ability of the model. Based on this, this invention proposes an automatic data source expansion method based on adaptive knowledge distillation. Summary of the Invention

[0003] The purpose of this invention is to provide an automatic data source expansion method based on adaptive knowledge distillation. TeacherModel and StudentModel respectively adopt BERT and a lightweight single-layer Attention-BiLSTM. According to the improvement rate of the model's recognition ability during training, the cross-entropy loss weight is adaptively adjusted, which can accelerate the convergence speed of the model and improve the model's recognition ability.

[0004] The technical solution to achieve the purpose of this invention is: an automatic data source expansion method based on adaptive knowledge distillation, comprising the following steps:

[0005] Step 1, Obtaining Website Text Information: Use general web crawling techniques to capture web page text information of known categories;

[0006] Step 2, Data Preprocessing: The webpage text information obtained in Step 1 is preprocessed and converted into a corresponding word list. The word list and its category are used as training and testing data to construct the training and testing datasets.

[0007] Step 3, Data source classification model construction: Construct a lightweight single-layer Attention-BiLSTM as the data source classification model;

[0008] Step 4, Adaptive Knowledge Distillation: Using the BERT model as the Teacher Model and the single-layer Attention-BiLSTM model constructed in Step 3 as the Student Model, adaptive knowledge distillation is used to train the Student Model, dynamically adjusting the cross-entropy loss weights to improve the convergence speed and recognition ability of the model, and obtaining an accurate, efficient, and lightweight data source classification model.

[0009] Step 5, Data Source Expansion: Preprocess the text information collected from unknown category data sources, input the data source classification model obtained from knowledge distillation training in Step 4 to predict the corresponding category, and record and store the URLs of each data source and their corresponding tags based on the classification results, thereby realizing the automatic accumulation and expansion of data sources.

[0010] Step 2, data preprocessing, the specific method is as follows:

[0011] 1) Denoise, segment, and build a dictionary for the webpage text information obtained in step 1, and convert it into a corresponding word list;

[0012] 2) Use the word list and its category as training and testing data to construct training and testing datasets.

[0013] Step 3, data source classification model construction, the specific method is as follows:

[0014] 1) Word Embeddings

[0015] By using Word Embeddings, each word in step 2 is mapped to a feature vector, which is then used as a high-quality feature input for the downstream website topic classification model. The specific calculation formula is as follows:

[0016] S = [s1, s2, ..., s] T ,] (1)

[0017] x t =Embeddings(s t (2)

[0018] Where S represents the word list of the input text, T represents the length of the word list, and s t To represent a word in a list, x t This represents the word vector encoded by Word Embeddings, where the subscript t indicates the position of the current word in the list, and 1≤t≤T;

[0019] 2) BiLSTM

[0020] BiLSTM consists of a forward-computed LSTM and a backward-computed LSTM, capturing contextual information from both directions;

[0021] After passing through the forward LSTM, the forward hidden state is obtained:

[0022]

[0023] After passing through a reverse LSTM, the reverse hidden state is obtained:

[0024]

[0025] Where, x t Representing word vectors, This represents the forward hidden vector, which is determined by the current word vector and the forward hidden vector from the previous time step. The inverse hidden vector is determined by the current word vector and the inverse hidden vector at the next time step. The subscript t indicates the position of the current word, and 1≤t≤T.

[0026] right and The hidden vector is obtained by merging the vectors using a weighted summation method:

[0027]

[0028] Among them, h t Represents the hidden vector. The weight matrix represents the positive output; b represents the weight matrix of the reverse output. t Represents the bias vector;

[0029] 3) Attention mechanism

[0030] The hidden vectors of BiLSTM serve as the input to the attention mechanism. The weights of each hidden state are calculated using the Softmax method, representing their contribution to the model output.

[0031] e t =utanh(Wh t +b) (6)

[0032]

[0033] g t =ω t h t (8)

[0034]

[0035] Where u represents the attention weight vector, W represents the weight matrix, b is the bias vector, and ω t G represents the weight of each hidden state. tLet represent the hidden state vector with attention weights, Q be the output layer mapping matrix, and p be the model's predicted probability distribution.

[0036] Step 4, Adaptive Knowledge Distillation, the specific method is as follows:

[0037] The BERT model is large and complex, but has strong learning capabilities. Therefore, it is used as the Teacher Model to guide the training of the Attention-BiLSTM, which is used as the Student Model, thus compressing the complex model. The overall loss of knowledge distillation is the weighted average of two losses: the cross-entropy loss between the Student Model's predicted probability distribution and the true label, and the KL divergence between the Student Model's predicted probability distribution and the Teacher Model's predicted probability distribution.

[0038]

[0039] Where Loss is the overall loss, y is the true label, and p is the predicted probability distribution of the Student Model. Let L be the predicted probability distribution of the Teacher Model. CE For cross-entropy loss, L KL The KL divergence is used, and temperature T is used for smoothing. λ is L CE Weight of Loss;

[0040] Adaptive knowledge distillation can automatically adjust L based on the improvement rate of the model's recognition ability without manual intervention. CE weight A larger boost rate will cause the model to automatically shrink. Increase the Teacher Model's contribution to model training to guide the model to converge quickly; a smaller boost rate will cause the model to automatically increase... To increase the contribution of real labels to model training and enhance the model's recognition ability, the j-th sample, during the k-th round of training, The calculation method is as follows:

[0041]

[0042] Where j is the sample number, and k is the current training round (k>2). The improvement rate of the model's recognition ability is limited to [0, 1]; α is the inertia coefficient, indicating... The degree of influence from previous values; (1-α) is the dynamic coefficient, indicating The degree to which it is affected by the improvement rate of recognition ability. As can be seen from the above analysis, Due to inertia factor and dynamic factors It consists of two parts: the former avoids randomness, and the latter can be dynamically corrected. If α = 1, then Let λ be a fixed value; if α = 0, then It depends entirely on the rate of improvement in the model's recognition capabilities;

[0043] With the improvement of equation (11), the overall loss of the adaptive knowledge distillation model is:

[0044]

[0045] in, The improved overall loss is defined as follows: j is the sample index, k is the current training epoch, and λ is the initial L value set. CE (p, y) weights The adaptive weights are an improvement on equation (11);

[0046] Using the Attention-BiLSTM model from step 3 as the Student Model and BERT as the Teacher Model, and Equation (12) as the overall loss function, a lightweight and efficient data source classification model can be obtained through adaptive knowledge distillation.

[0047] Step 5, expand the data source, the specific method is as follows:

[0048] 1) Obtain the data source to be identified

[0049] The web crawling framework Scrapy is used to make network requests to the website, extract the text content based on the web page structure, obtain the effective features of these newly added web pages, including title, summary, and body, and perform preprocessing.

[0050] 2) Determine the data source label

[0051] Using the adaptive knowledge distillation algorithm from step 4, the data source classification model from step 3 is trained. This lightweight and efficient classification model identifies the categories of these web pages and tags the websites to which they belong based on their categories.

[0052] C = func(S) (13)

[0053] Where S represents the text information of the webpage, which is input into the data source classification model func obtained through training, and outputs the category C to which the webpage belongs. If different webpages under a certain website correspond to multiple categories, then the website should also have multiple tags, and its tags should be stored in a list.

[0054] 3) Automated accumulation and expansion of data sources

[0055] Store the URL of the website homepage and the corresponding tag list to automate the accumulation and expansion of data sources.

[0056] An automatic data source expansion system based on adaptive knowledge distillation is characterized by achieving automatic expansion of data sources based on adaptive knowledge distillation through the aforementioned automatic data source expansion method.

[0057] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it automatically expands the data source based on adaptive knowledge distillation through the aforementioned method for automatic expansion of the data source based on adaptive knowledge distillation.

[0058] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the data source is automatically expanded based on adaptive knowledge distillation through the aforementioned method for automatically expanding the data source based on adaptive knowledge distillation.

[0059] Compared with existing technologies, the significant advantages of this invention are: 1) It proposes a method for identifying website categories based on NLP technology, which can automatically expand various data sources in conjunction with general crawler technology; 2) It uses adaptive weights to dynamically correct the cross-entropy loss weights of knowledge distillation, which can improve the convergence speed and recognition ability of the model, resulting in an accurate, efficient, and lightweight data source classification model that is more suitable for data collection application scenarios. Attached Figure Description

[0060] Figure 1 It is a framework for automatically expanding data sources based on adaptive knowledge distillation;

[0061] Figure 2 It is an adaptive knowledge distillation network model architecture. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0063] This invention proposes a method for identifying website categories based on NLP technology. It can automatically expand various data sources in conjunction with general-purpose web crawlers. Through adaptive knowledge distillation, it improves the training efficiency and recognition capability of the model, resulting in an accurate, efficient, and lightweight data source classification model, making this invention suitable for data acquisition applications. The specific steps are as follows:

[0064] Step 1. Obtain training corpus. Using general web crawling techniques, obtain website text information of known categories as training corpus. Each sample specifically includes web page text and its corresponding category.

[0065] Step 2. Data Preprocessing. After preprocessing operations such as noise reduction, word segmentation, and dictionary construction, the web page text information obtained in Step 1 is converted into a corresponding word list. The word list and its category are used as training and testing datasets and stored in the database.

[0066] Step 3. Building the Data Source Classification Model. A lightweight single-layer Attention-BiLSTM is constructed as the data source classification model, which involves the following steps:

[0067] 1) Word Embeddings

[0068] The input text is encoded using Word Embeddings. Each word from step 2 is mapped to a feature vector using Word Embeddings, which serves as high-quality feature input for the downstream website classification model. The specific calculation formula is as follows:

[0069] S = [s1, s2, ..., s] T ,] (1)

[0070] x t =Embeddings(s t (2)

[0071] Where S represents the word list of the input text, T represents the length of the word list, and s t To represent a word in a list, x t This represents the word vector encoded by Word Embeddings, where the subscript t indicates the position of the current word in the list, and 1≤t≤T.

[0072] 2) BiLSTM

[0073] BiLSTM links a forward-computed LSTM and a backward-computed LSTM, enabling it to capture contextual information more comprehensively from both directions.

[0074] After passing through the forward LSTM, the forward hidden state can be obtained:

[0075]

[0076] After passing through a reverse LSTM, the reverse hidden state can be obtained:

[0077]

[0078] Where, x tRepresenting word vectors, This represents the forward hidden vector, which is determined by the current word vector and the forward hidden vector from the previous time step. The vector represents the reverse hidden vector, which is determined by the current word vector and the reverse hidden vector at the next time step. The subscript t indicates the position of the current word, and 1≤t≤T.

[0079] right and By merging the vectors using a weighted summation method, the hidden vectors can be obtained:

[0080]

[0081] Among them, h t Represents the hidden vector. The weight matrix represents the positive output; b represents the weight matrix of the reverse output. t This represents the bias vector.

[0082] 3) Attention mechanism

[0083] To further improve the model's accuracy, an attention mechanism was introduced on top of BiLSTM. By assigning different weights to hidden features through probability allocation, the attention to important features was increased while the attention to other features was reduced, thereby improving the accuracy of the model's predictions.

[0084] The hidden vectors of BiLSTM serve as the input to the attention mechanism. The weights of each hidden state are calculated using the Softmax method, representing their contribution to the model output.

[0085] e t =utanh(Wh t +b) (6)

[0086]

[0087] g t =ω t h t (8)

[0088]

[0089] Where u represents the attention weight vector, W represents the weight matrix, b is the bias vector, and ω t Let gt represent the weights of each hidden state, gt represent the hidden state vector with attention weights, Q represent the output layer mapping matrix, and p represent the model's predicted probability distribution.

[0090] Step 4. Adaptive Knowledge Distillation. Using the BERT model as the Teacher Model and the single-layer Attention-BiLSTM model constructed in Step 3 as the Student Model, adaptive knowledge distillation is used to train the Student Model. The cross-entropy loss weights are dynamically adjusted to improve the convergence speed and recognition ability of the model, resulting in an accurate, efficient, and lightweight data source classification model.

[0091] The BERT model is large and complex, but it has strong learning capabilities. Therefore, it is used as the Teacher Model to guide the training of the Attention-BiLSTM, which is used as the Student Model, thus compressing the complex model. The overall loss of knowledge distillation is the weighted average of two losses: the cross-entropy loss between the Student Model's predicted probability distribution and the true label, and the KL divergence between the Student Model's predicted probability distribution and the Teacher Model's predicted probability distribution.

[0092]

[0093] Where Loss is the overall loss, y is the true label, and p is the predicted probability distribution of the Student Model. Let L be the predicted probability distribution of Teacher Mode1. CE For cross-entropy loss, L KL The KL divergence is used, and temperature T is used for smoothing. λ is L CE Weight of Loss.

[0094] Adaptive knowledge distillation can automatically adjust L based on the improvement rate of the model's recognition ability without manual intervention. CE weight A larger boost rate will cause the model to automatically shrink. Increase the Teacher Model's contribution to model training to guide the model to converge quickly; a smaller boost rate will cause the model to automatically increase... To increase the contribution of real labels to model training and enhance the model's recognition ability, the j-th sample, during the k-th round of training, The calculation method is as follows:

[0095]

[0096] Where j is the sample number, and k is the current training round (k>2). The improvement rate of the model's recognition ability is limited to [0, 1]; α is the inertia coefficient, indicating... The degree of influence from previous values; (1-α) is the dynamic coefficient, indicating The degree to which it is affected by the improvement rate of recognition ability. As can be seen from the above analysis, Due to inertia factor and dynamic factors It consists of two parts: the former avoids randomness, and the latter can be dynamically corrected. If α = 1, then Let λ be a fixed value; if α = 0, then It depends entirely on the rate of improvement in the model's recognition capabilities.

[0097] With the improvement of equation (11), the overall loss of the adaptive knowledge distillation model is:

[0098]

[0099] in, The improved overall loss is defined as follows: j is the sample index, k is the current training epoch, and λ is the initial L value set. CE (p, y) weights The adaptive weights are an improvement of equation (11).

[0100] Using the Attention-BiLSTM model from step 3 as the Student Model and BERT as the Teacher Model, and Equation (12) as the overall loss function, a lightweight and efficient data source classification model can be obtained through adaptive knowledge distillation.

[0101] Step 5. Data Source Expansion. Preprocess the text information collected from unknown category data sources, input the data source classification model trained by knowledge distillation in Step 4 to predict the corresponding category, and record and store the URLs of each data source and their corresponding tags based on the classification results, thus achieving automatic accumulation and expansion of data sources.

[0102] 1) Obtain the data source to be identified

[0103] The web crawling framework Scrapy is used to make network requests to the website, extract the text content based on the web page structure, obtain the effective features of these newly added web pages, including title, summary, and body, and perform preprocessing.

[0104] 2) Determine the data source label

[0105] Using the adaptive knowledge distillation algorithm from step 4, the data source classification model from step 3 is trained. This lightweight and efficient classification model identifies the categories of these web pages and tags the websites to which they belong based on their categories.

[0106] C = func(S) (13)

[0107] Where S represents the text information of the webpage, which is input into the data source classification model func obtained through training, and outputs the category C to which the webpage belongs. If different webpages under a certain website correspond to multiple categories, then the website should also have multiple tags, and its tags should be stored in a list.

[0108] 3) Automated accumulation and expansion of data sources

[0109] Store the URL of the website homepage and the corresponding tag list to automate the accumulation and expansion of data sources.

[0110] Example

[0111] To verify the effectiveness of the present invention, the following experiment was conducted.

[0112] Step 1. Obtain training corpus. A web crawler was developed for 13 energy websites, including those related to oil, natural gas, and electricity, to obtain the website text content and its category, as shown in Table 1.

[0113] Table 1. Webpage Text Content and Its Category

[0114]

[0115] Step 2. Data Preprocessing. The corpus obtained in Step 1 undergoes preprocessing operations such as denoising, word segmentation, and dictionary construction to convert the webpage text information into a corresponding word list. This word list and its categories are used as training and testing datasets and stored in the database. The word list and its categories are shown in Table 2.

[0116] Table 2. List of words and their categories

[0117]

[0118]

[0119] Step 3. Data Source Classification Model Construction. First, Word Embeddings accepts discrete index one-hot vectors as input. According to formula (2), a webpage text can be mapped to a 100-dimensional semantic feature vector:

[0120] [[-1.1907, -1.8287, ..., 1.1620],

[0121] [-1.0208, 0.9166, ..., -1.3042],

[0122] [-1.0574, -0.1188, ..., 0.9078],

[0123] [0.3452, -0.5713, ..., -0.2351],

[0124] …

[0125] [1.0076, -0.7529, ..., -0.2250]]

[0126] Then, according to formulas (3)-(9), the Attention-BiLSTM data source classification model is constructed.

[0127] Step 4. Adaptive Knowledge Distillation. First, a BERT model is trained using the labeled data obtained in Step 1. This model is used as the Teacher Model, and the Attention-BiLSTM data source classification model constructed in Step 3 is used as the Student Model. Then, L... CE The initial weight λ is set to 0.7, the inertia coefficient α is set to 0.4, and the temperature T is set to 2. Equation (12) is used as the objective function for adaptive knowledge distillation, and iterative training of the model begins. The data source classification model converges after iterative training with adaptive knowledge distillation, resulting in an accurate and efficient lightweight model Tunc. * .

[0128] Step 5. Data Source Expansion. Using web scraping technology, collect multiple webpage texts from an unknown category data source (url: www.XXX.com), and perform preprocessing operations to obtain the following word segmentation lists:

[0129]

[0130]

[0131] Input the information into func obtained in step 4. * The category of the data source can be predicted according to formula (13):

[0132]

[0133]

[0134] Based on the classification results, the URLs of each data source and their corresponding tags are recorded and stored in the data source repository (resources), enabling the automated accumulation and expansion of data sources.

[0135] dict={″www.XXX.com″:[″solar energy″,″wind energy″,…]}

[0136] resources.append(dict).

[0137] If users subsequently require data on solar or wind energy, they can directly collect data from this data source. Testing has shown that this invention significantly improves the targeting and efficiency of data collection.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data source automatic expansion method based on adaptive knowledge distillation, characterized in that, The method comprises the following steps: Step 1, website text information acquisition: using crawler technology to capture the text information of the known category web page; Step 2, data preprocessing: preprocessing the web text information obtained in step 1, converting it into a corresponding word list, taking the word list and its category as training and testing data, and constructing a training and testing data set; Step 3, data source classification model construction: constructing a lightweight single-layer Attention-BiLSTM as a data source classification model; Step 4, adaptive knowledge distillation: taking the BERT model as the Teacher Model and the single-layer Attention-BiLSTM model constructed in step 3 as the Student Model, training the Student Model by adaptive knowledge distillation, dynamically adjusting the cross-entropy loss weight, and obtaining an accurate and efficient lightweight data source classification model, wherein: The overall loss of knowledge distillation is the weighted average of two losses, which are the cross-entropy loss of the predicted probability distribution of the Student Model and the true label, and the KL divergence of the predicted probability distribution of the Student Model and the predicted probability distribution of the Teacher Model: where Loss is the overall loss, y is the true label, and p is the predicted probability distribution of the Student Model, is the predicted probability distribution of the Teacher Model, and L CE is the cross-entropy loss, L KL is the KL divergence, and temperature T is used to smooth is the predicted probability distribution of the Teacher Model, and L CE is the weight of L Adaptive knowledge distillation automatically adjusts L based on the improvement rate of the model's recognition ability. CE weight A larger boost rate will cause the model to automatically shrink. Increase the Teacher Model's contribution to model training to guide the model to converge quickly; a smaller boost rate will cause the model to automatically increase... To increase the contribution of real labels to model training and enhance the model's recognition ability, the j-th sample will be used in the k-th training round. The calculation method is as follows: wherein j is a sample serial number, k, k>2 is a current training round, is a model recognition ability improvement rate, the range is limited to [0, 1]; α is an inertia coefficient, indicating the degree of influence by the previous value; (1-α) is a dynamic coefficient, indicating the degree of influence by the recognition ability improvement rate; From the above analysis, is composed of two parts, the former avoids contingency, and the latter dynamically corrects two parts, the former avoids contingency, and the latter dynamically corrects If α = 1, then is a fixed value λ; if α = 0, then is completely dependent on the improvement rate of model identification ability;​ Through the improvement of formula (11), the overall loss of the adaptive knowledge distillation model is: wherein, L is the improved overall loss, λ is the set initial l CE (p, y) weights, is the improved adaptive weight of equation (11); Taking the Attention-BiLSTM model of step 3 as the Student Model, BERT as the Teacher Model, and formula (12) as the overall loss function, a lightweight and efficient data source classification model is obtained through adaptive knowledge distillation; Step 5, data source expansion: preprocessing the text information of the unknown category data source collected, inputting the data source classification model obtained by knowledge distillation in step 4 to predict the corresponding category, recording and storing the url and its corresponding label of each data source according to the classification result, and realizing the automatic accumulation and expansion of the data source.

2. The method of claim 1, wherein, Step 2, data preprocessing, the specific method is: 1) denoising, word segmentation and dictionary establishment are performed on the web text information obtained in step 1, and the web text information is converted into a corresponding word list; 2) taking the word list and its category as training and testing data, a training and testing data set is constructed. 3.The data source automatic expansion method based on adaptive knowledge distillation according to claim 1, wherein, Step 3, data source classification model construction, the specific method is: 1) Word Embeddings In the way of Word Embeddings, each word in step 2 is mapped to a feature vector, which is used as high-quality feature input for the downstream website topic classification model, and the specific calculation formula is as follows: S = [s1, s2,..., s T ,] (1) x t = Embeddings(s t ) (2) where S denotes a list of words of an input text, T denotes the length of the list of words, s t denotes a word in the list, x t denotes a word vector encoded by WordEmbeddings, the subscript t denotes the position of the current word in the list, 1≤t≤T; 2) BiLSTM BiLSTM includes a forward calculation LSTM and a reverse calculation LSTM, which captures context information from two directions; After forward LSTM, the forward hidden state is obtained: After reverse LSTM, the reverse hidden state is obtained: wherein x t represents a word vector, represents a forward hidden vector determined by the current word vector and the previous time forward hidden vector, represents a backward hidden vector determined by the current word vector and the next time backward hidden vector, and subscript t represents the position of the current word, 1≤t≤T; To and The hidden vector is obtained by merging in a weighted sum manner: where h t denotes a hidden vector, denotes a weight matrix for the forward output; denotes a weight matrix for the backward output, b t denotes a bias vector; 3) Attention mechanism The hidden vector of BiLSTM is the input of the attention mechanism, and the weight of each hidden state is calculated by the Softmax method, which is used as the contribution degree of the hidden state to the model output: e t = utanh(Wh t + b) (6) g t = ω t h t (8) where u denotes the attention weight vector, W denotes the weight matrix, b is the bias vector, and ω t denotes the weight of each hidden state, g t denotes the hidden state vector with attention weight, Q is the output layer mapping matrix, and p is the prediction probability distribution of the model. 4.The data source automatic expansion method based on adaptive knowledge distillation according to claim 1, wherein, Step 5, data source expansion, the specific method is: 1) Obtain the data source to be identified Through the web scraping framework Scrapy, the network request is made to the website, the text content is preliminarily extracted according to the webpage structure, the effective features of these new webpages are obtained, including the title, abstract, and body, and pretreatment is performed; 2) Determine the data source label The data source classification model of step 3 is trained by using the adaptive knowledge distillation algorithm of step 4, the category of these webpages is identified by using the lightweight and efficient classification model, and the label of the website is marked according to the category: C = func (S) (13) Wherein, S represents the webpage text information, which is input into the trained data source classification model func, and the category C of the webpage is output, if different webpages under a website correspond to multiple categories, the website should also have multiple labels, and the label list is stored; 3) Automatic accumulation and expansion of data source The url of the homepage of the website and the corresponding label list are stored, and the automatic accumulation and expansion of the data source are realized.

5. An adaptive knowledge distillation based data source automatic augmentation system, comprising: Through the data source automatic expansion method based on adaptive knowledge distillation of any one of claims 1-4, the data source automatic expansion based on adaptive knowledge distillation is realized.

6. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, when the processor executes the computer program, the data source automatic expansion method based on adaptive knowledge distillation of any one of claims 1-4 is realized, and the data source automatic expansion based on adaptive knowledge distillation is realized.

7. A computer readable storage medium having a computer program stored thereon, when the computer program is executed by a processor, the data source automatic expansion method based on adaptive knowledge distillation of any one of claims 1-4 is realized, and the data source automatic expansion based on adaptive knowledge distillation is realized.

Images