An adaptive text classification method in the field of scientific and technical information

Abstract

The application discloses a self-adaptive text classification method in the field of scientific information, which comprises the following steps: training a classifier and establishing an original clustering forest; selecting a suitable classifier and establishing an optimal clustering forest, wherein the classifier is a clustering tree algorithm; and testing the latest sample to obtain an integrated final result. The application can improve the classification effect and generalization ability of the text classification in the field of scientific information, thereby improving the accuracy of the text classification model and the accuracy of the text classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical information field, and particularly relates to a self-adaptive text classification method in the technical information field. BACKGROUND

[0002] The information technology refers to a method and equipment used for obtaining, processing, storing, retrieving and transmitting text, number, image and sound information. The text classification of the technical information can identify the category of the text. However, the existing classification method has low classification efficiency, is prone to error and wastes a large amount of manpower. SUMMARY

[0003] In order to solve the above problems, the present application provides a self-adaptive text classification method in the technical information field.

[0004] In order to achieve the above object, the technical scheme adopted by the present application is as follows:

[0005] A self-adaptive text classification method in the technical information field, comprising the following steps: training a classifier and establishing an original clustering forest.

[0006] The self-adaptive selection of a suitable classifier and the establishment of an optimal clustering forest, wherein the classifier is a clustering tree algorithm.

[0007] Testing the latest sample to obtain an integrated final result.

[0008] Optionally, the training of the classifier and the establishment of the original clustering forest comprises: defining a time window Mmax, and only the classifiers in the Mmax time period can adapt to the current concept. At the t time, the latest Mmax classifiers form an original clustering forest. If t<=Mmax, all the historical classifiers will be added to the original clustering forest. If t>Mmax, only the classifiers (f t-Mmax+1 ,f t-Mmax+2 ,...,f t ) are added to the original clustering forest.

[0009] Optionally, the clustering tree algorithm adopts a self-adaptive selection strategy and a voting strategy. The self-adaptive selection strategy is used for selecting the classifiers with good performance and meeting the current concept, and the voting strategy is used for integrated voting on the prediction results.

[0010] Optionally, in the classification of the text stream with concept drift, the number of classifiers participating in the final prediction is obviously inconsistent in different time periods. When the concept is stable, the historical data blocks have a greater impact on the current concept, and the classification algorithm makes the historical classifiers conforming to the current concept participate in the integrated classification. When the concept drifts, the probability distribution of the historical data blocks is different from the new concept, and the classification algorithm eliminates the influence of the historical classifiers in the integrated model. In order to adapt to the actual concept drift environment, the adaptive selection strategy of the DCF algorithm dynamically changes the size of the classifiers participating in the classification according to the occurrence of the concept drift.

[0011] Optionally, when the concept drifts, the historical data cannot adapt to the new concept, and the classifiers established by the historical data cannot be applied to the new test samples. Therefore, it is crucial to obtain the information of the test samples. The voting strategy takes into account the accuracy weight and the confidence weight of the classifier.

[0012] Optionally, the confidence weight is established by three classifiers, namely, classifier 1, classifier 2 and classifier 3, according to the corresponding training data blocks. At time i, when the concept does not drift, the three classifiers classify the test sample, calculate the distance (i.e., the similarity) between the cluster node center to which the test sample belongs and the test sample, denoted as d1, d2 and d3, and it can be obtained that d1

[0013] Optionally, the accuracy weight is that the adaptive selection strategy in the DCF algorithm dynamically divides the classifiers into two categories, namely, good performance and poor performance, and selects the good classifiers to join the optimal clustering forest. In order to obtain the accuracy weight at each time, the DCF algorithm calculates the accuracy of each classifier according to the validation set, and judges whether the concept drifts according to the change of the accuracy weight. If it is detected that the concept does not drift, the optimal clustering forest is expected to contain as many historical classifiers as possible. If the concept drifts, only the classifiers meeting the accuracy weight requirement participate in the final integrated classification.

[0014] Optionally, the voting strategy is used to integrate the prediction results of each classifier. For the i-th classifier, the confidence weight and the accuracy weight of the j-th test sample xj need to be calculated, and the two are combined to obtain the final voting weight:

[0015] v i (xj ) = φ i λ i，j

[0016] where φi is the accuracy weight of the classifier fi, and λi,j is the confidence weight of the classifier fi to the test sample xj.

[0017] Optionally, if the output function of the ith classifier is fi(xj,l), then:

[0018]

[0019] where Li,j is the output class label of the test sample xj in the ith classifier, and l represents all possible output results of xj.

[0020] Optionally, the integrated output function fE(xj,l) of the DCF model is calculated by the following formula:

[0021]

[0022] where vi(xj) is the voting weight, and fi(xj,l) is the output function of the classifier i

[0023] When f E (xj,l) takes the maximum value, the predicted class label Lj is calculated by the following formula:

[0024]

[0025] The integrated prediction result of the DCF is as follows:

[0026]

[0027] Compared with the prior art, the technical progress achieved by the present application is that:

[0028] The present application dynamically selects the classifiers participating in classification according to the adaptive selection strategy, and then obtains the final prediction result according to the integrated voting strategy. Different from the existing algorithm, the DCF algorithm introduces accuracy weight and confidence weight, which not only considers the influence of historical data on the classification algorithm, but also tries to use the latest data to solve the concept drift problem, so as to improve the classification effect and generalization ability of the text classification in the field of scientific and technical information, thereby improving the accuracy of the text classification model, and further improving the accuracy of the text classification. BRIEF DESCRIPTION OF DRAWINGS

[0029] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, illustrate the present application together with the embodiments thereof, and explain the present application, but do not constitute a limitation of the present application.

[0030] In the drawings:

[0031] Figure 1 Flowchart of the present application. DETAILED DESCRIPTION

[0032] The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings. In the description of the following embodiments, specific details such as specific system structures, technologies, etc. are presented for the purpose of illustration, not for the purpose of limitation, so as to thoroughly understand the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details that hinder the description of the present application.

[0033] As Figure 1 shown, the present application discloses a self-adaptive text classification method in the field of scientific information, comprising.

[0034] Embodiment one

[0035] The present embodiment discloses a self-adaptive text classification method in the field of scientific information, the steps comprising:

[0036] S01: training the classifier to establish the original clustering forest;

[0037] S02: self-adaptive selection of appropriate classifier to establish the optimal clustering forest, wherein the classifier is a clustering tree algorithm;

[0038] S03: test the latest sample to get the integrated final result.

[0039] Establishing the original dynamic clustering forest needs to be based on a time assumption that too old data blocks do not play any role in the current concept. In view of this, it is necessary to define a time window Mmax, and it is believed that only the classifiers in Mmax time period can adapt to the current concept. At the t time, the latest Mmax classifiers form an original clustering forest. If t≤Mmax, all historical classifiers will be added to the original clustering forest; if t>Mmax, only the classifiers (ft-Mmax+1, ft-Mmax+2,..., ft) are added to the original clustering forest.

[0040] Self-adaptive selection of sub-clustering forest. In order to construct the optimal dynamic clustering forest, the accuracy weight of each classifier needs to be calculated. Only the classifiers with larger weight can be added to the optimal dynamic clustering forest.

[0041] For each test sample x, first calculate the confidence weight of each classifier, and then obtain the final prediction result according to the voting strategy.

[0042] Embodiment Two

[0043] The DCF algorithm contains two important strategies: adaptive selection strategy and voting strategy.

[0044] (1) The goal of adaptive selection strategy is to select the classifiers which have good performance and conform to the current concept. In the existing text stream ensemble classification algorithms (such as AWE algorithm, AUE algorithm, etc.), the window of the classifiers is fixed, that is, the number of classifiers participating in the final ensemble prediction is fixed each time. However, when classifying the text stream with concept drift, the number of classifiers participating in the final prediction is obviously inconsistent at different time periods. When the concept is stable, the historical data blocks have a greater impact on the current concept, and the classification algorithm should make as many historical classifiers conforming to the current concept as possible to participate in the ensemble classification; when the concept drifts, the probability distribution of the historical data blocks is different from the new concept, and the classification algorithm should eliminate the influence of the historical classifiers in the ensemble model as much as possible. In order to conform to the actual concept drift environment, the adaptive selection strategy of the DCF algorithm dynamically changes the size of the classifiers participating in the classification according to the occurrence of concept drift.

[0045] (2) The voting strategy of the DCF algorithm is constructed based on the precision weight and the confidence weight. When the concept drifts, the historical data cannot adapt to the new concept, and obviously the classifiers established by the historical data cannot be applied to the new test samples. In this case, it is crucial to obtain as much information of the test samples as possible. Based on this, the voting strategy takes into account not only the overall performance of the classifier (i.e., the precision weight), but also the reliability of the classifier to the specific test sample (defined as the confidence weight). The confidence weight is defined according to the similarity between the test sample and the cluster center to which it belongs. The higher the similarity, the closer the distance between the test sample and the cluster center, and the greater the confidence of the test sample belonging to the cluster.

[0046] Embodiment Three

[0047] The confidence weight is an index to measure whether the classifier is reliable to the new test sample. Two classes (class 1 and class 2) are set at (i-3) time, (i-2) time and (i-1) time. Three classifiers are established according to the corresponding training data blocks, which are classifier 1, classifier 2 and classifier 3, respectively. At i time, the concept does not drift, and the three classifiers classify the test sample. Suppose a test sample belonging to class 2 arrives (indicated by a circle), and the three classifiers classify it. Only classifier 1 gets the correct result (class 2), and the results of the other classifiers are wrong (class 1). If the traditional majority voting method is used to integrate the classifiers, since the majority of the classifiers (two classifiers) get the same result, the final integrated result is class 1, which is undoubtedly a wrong classification result.

[0048] After introducing the confidence weight, the DCF algorithm needs to calculate the distance between the test sample and the cluster center of the classifier to which the test sample belongs, i.e. the similarity, denoted as di, d2 and d3. It can be concluded that di < d2, di < d3, which indicates that the confidence of classifier 1 is higher than that of classifier 2 and classifier 3. Based on this, we can obtain the correct class label of the test sample. The importance of the confidence weight occurs when the concept drift occurs. At time (j-2) and (j-1), two classifiers (classifier 1 and classifier 2) are constructed. When the concept drift occurs at time j, the true feature space has changed, but the feature space of classifier 1 and classifier 2 has not changed based on the old concept data block. Therefore, when the new test sample (annotated with a triangle, belonging to class 1) is classified by classifier 1 and 2, classifier 1 gets the wrong result (class 2), while the classification result of the test sample by classifier 2 is correct (class 1). If the ensemble model adopts the majority voting strategy to predict the class label, the correct classification result cannot be obtained because the voting results of the two classifiers are opposite. If the ensemble model adopts the precision weighted ensemble strategy to predict the final result, it can be concluded that the precision of classifier 1 is higher than that of classifier 2, so the ensemble result of the test sample is class 2, which is also an incorrect result.

[0049] However, in the DCF algorithm, the confidence of the two classifiers is calculated (here, the distance is simply calculated), and di > d2 is obtained. This indicates that classifier 2 is more reliable than classifier 1. Based on this, the test sample can obtain the correct classification result. It can be seen that the DCF algorithm defines two important weight indicators: precision weight and confidence weight. Among them, the precision weight uses historical data as much as possible to form an optimal ensemble model in the case of smooth text stream or concept drift. Because the concept changes unpredictably in time and direction, historical data is not reliable when the concept drifts, and only the latest test sample (without class label sample) can represent the distribution of the current concept. In order to mine more useful information of the test sample (current concept), the DCF algorithm introduces the confidence weight. If the test sample and the cluster center to which it belongs are more similar, the test sample is more likely to be classified into the category of the cluster to which it belongs, i.e. the classification of the test sample by the classifier is more reliable, and the confidence weight is defined as higher; otherwise, the confidence weight is defined as lower. It can be seen that the definition of the confidence weight depends on the distance (similarity) between the test sample and the cluster center, so we first explain the effect of similarity (distance) on the reliability of the test sample classification in the case of two-class classification problem.

[0050] Example Four

[0051] The dynamic and concept drift of text stream need to build classifiers constantly, but not all classifiers participate in the final classification, how to select the appropriate classifier to classify the test sample is an important problem for ensemble strategy. The adaptive selection strategy in DCF model dynamically divides the classifiers into two categories: good performance and poor performance, and selects the good classifiers to join the optimal clustering forest. In order to obtain the accuracy weight of each moment, DCF algorithm needs to calculate the accuracy of each classifier according to the validation set. According to the change of accuracy weight, it is judged whether the concept has drifted: if it is detected that the concept has not drifted, the optimal clustering forest is expected to contain as many historical classifiers of the same concept as possible; if the concept has drifted, only the classifiers that meet the accuracy weight requirement participate in the final ensemble classification. Voting strategy is used to integrate the prediction results of each classifier. Unlike existing algorithms whose voting weight depends on the accuracy of the classifier, in order to make full use of the information of new and old data as much as possible, the calculation of voting weight in DCF algorithm depends not only on the accuracy, but also on the confidence weight. For the ith classifier, the confidence weight and accuracy weight of the jth test sample xj are calculated, and the two are combined to obtain the final voting weight:

[0052] v i (x j )=φ i λ i，j

[0053] Where φi is the accuracy weight of classifier fi, λi,j is the confidence weight of classifier fi for test sample xj.

[0054] Let the output function of the ith classifier be fi(xj,l), then:

[0055]

[0056] Where Li,j is the output class label of test sample xj in the ith classifier, and l represents all possible output results of xj.

[0057] The ensemble output function f E (xj,l) of DCF model is calculated by the following formula:

[0058]

[0059] Where vi(xj) is the voting weight, and fi(xj,l) is the output function of classifier i

[0060] When f E (xj,l) takes the maximum value, the predicted class label Lj is calculated by the following formula:

[0061]

[0062] The integrated prediction result of the DCF is as follows:

[0063]

[0064] In the above embodiments, the description of each embodiment is focused on, and the part not described or recorded in a certain embodiment can be referred to the relevant description of other embodiments.

[0065] In each embodiment, the hardware implementation of the technology can directly use existing intelligent devices, including but not limited to industrial computers, PC computers, smart phones, handheld computers, floor-standing computers, etc. The input device is preferably a screen keyboard, the data storage and calculation module uses existing memory, calculator and controller, the internal communication module uses existing communication port and protocol, and the remote communication uses existing gprs network, world wide web, etc.

[0066] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is taken as an example, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit and module is only for convenient distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0067] The various function units in the embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit. When the integrated module / unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier wave signal, telecommunication signal and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electric carrier wave signal and telecommunication signal.

[0068] It should be noted that the terms used herein are only intended to describe specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form, unless the context clearly indicates otherwise, and it should also be understood that, when the terms "comprise" and / or "include" are used in the specification, there is a presence of the features, steps, operations, devices, components and / or combinations thereof.

[0069] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0070] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. An adaptive text classification method in the field of scientific and technological information, characterized in that, It includes the following steps: Train a classifier to establish an original clustering forest; Adaptively select a suitable classifier to establish an optimal clustering forest, where the classifier is a clustering tree algorithm; Test the latest samples to obtain the integrated final result; The training of the classifiers and the construction of the original cluster forest include: defining a time window Mmax, where only classifiers within the Mmax time window are likely to adapt to the current concept; at time t, the most recent Mmax classifiers form an original cluster forest; if t ≤ Mmax, all historical classifiers will be added to the original cluster forest; if t > Mmax, only classifier (f... t-Mmax+1 ,f t-Mmax+2 ,...,f t Add it to the original cluster forest; The clustering tree algorithm has an adaptive selection strategy and a voting strategy. The adaptive selection strategy is used to select classifiers with good performance and in line with the current concept, and the voting strategy is used to conduct integrated voting on the prediction results; When classifying a text stream with concept drift, the number of classifiers participating in the final prediction is obviously inconsistent in different time periods. When the concept is stable, the historical data blocks have a greater impact on the current concept, and the classification algorithm enables the historical classifiers in line with the current concept to participate in the integrated classification; when the concept drifts, the probability distribution of the historical data blocks is different from the new concept, and the classification algorithm eliminates the influence of the historical classifiers in the integrated model. To conform to the actual concept drift environment, the adaptive selection strategy of the DCF algorithm dynamically changes the scale of the classifiers participating in the classification according to the occurrence of concept drift.

2. The adaptive text classification method for the field of scientific and technological information according to claim 1, characterized in that: When the concept drifts, the historical data cannot adapt to the new concept, and the classifiers established by the historical data are also not applicable to the new test samples. It is crucial to obtain the information of the test samples. The voting strategy takes into account the accuracy weight and confidence weight of the classifiers.

3. The adaptive text classification method for the field of scientific and technological information according to claim 2, characterized in that: The confidence weight is that three classifiers are respectively established based on the corresponding training data blocks: classifier 1, classifier 2, and classifier 3. At time i, the concept has not drifted, and the three classifiers classify the test samples. Calculate the distance (i.e., similarity) between the center of the cluster node to which the test sample belongs in each classifier and the test sample, denoted as d1, d2, and d3. It can be obtained that d1 < d2 and d1 < d3, that is, the confidence of classifier 1 is higher than that of classifier 2 and classifier 3; when the concept drifts at times (j - 2) and (j - 1), two classifiers are constructed: classifier 1 and classifier 2. When the concept drifts at time j, the real feature space changes, and the feature spaces of classifier 1 and classifier 2 do not change. That is, when classifier 1 and 2 classify the new test samples, classifier 1 gets an incorrect result, while the classification result of classifier 2 for this sample is correct.

4. The adaptive text classification method for the field of scientific and technological information according to claim 3, characterized in that: The accuracy weight is that the adaptive selection strategy in the DCF algorithm dynamically divides the classifiers into two categories: those with good performance and those with poor performance, and selects the good classifiers to be added to the optimal clustering forest. To obtain the accuracy weight at each moment, the DCF algorithm calculates the accuracy of each classifier based on the validation set and judges whether the concept has drifted according to the change of the accuracy weight: if it is detected that the concept has not drifted, it is hoped that the optimal clustering forest contains as many historical classifiers of the same concept as possible; if the concept drifts, only the classifiers that meet the accuracy weight requirements participate in the final integrated classification.

5. The adaptive text classification method for the field of scientific and technological information according to claim 4, characterized in that: The voting strategy is used to integrate the prediction results of each classifier. For the i-th classifier, it is necessary to calculate the confidence weight and accuracy weight of the j-th test sample xj, and combine the two to obtain the final voting weight: v i (x j )=φ i l i,j Where φi is the accuracy weight of classifier fi, and λi,j is the confidence weight of classifier fi for test sample xj.

6. The adaptive text classification method for the field of scientific and technological information according to claim 5, characterized in that: Let the output function of the i-th classifier be fi(xj,l), then: Where Li,j is the output class label of the test sample xj in the i-th classifier, and l represents all possible output results of xj.

7. The adaptive text classification method for the field of scientific and technological information according to claim 6, characterized in that: The integrated output function f of the DCF model E (xj,l) is calculated using the following formula: Where vi(xj) is the voting weight, and fi(xj,l) is the output function of classifier i. When f E When (xj,l) reaches its maximum value, the predicted class label Lj is calculated using the following formula: The integrated prediction results of DCF are as follows:

Images