A classification model construction method and terminal device based on FLearnNSE-Pruned-Age

Abstract

The present invention provides a classification model construction method and terminal device based on FLearnNSE-Pruned-Age, belonging to the field of data analysis technology. The present invention first performs parameter preset, and when the training data set arrives and the time t≠1, calculates the integrated classifier H composed of the current base classifier set t‑1 In the training data set d t The error rate E t And normalized to get B t , by B t Update and normalize the instance weight #imgabs0# to calculate the penalty weight #imgabs1# in the training dataset d t The weighted error rate #imgabs3# of all base classifier sets based on the penalty weight #imgabs2# is calculated above, and the base classifiers in all base classifier sets are sorted from small to large according to the value of #imgabs4#, and the base classifier with the optimal weighted error rate is selected from the beginning among all base classifiers and integrated with the current base classifier set. The present invention can realize the rapid and accurate construction of the classification model for the training data set, and improve the construction efficiency and classification accuracy of the classification model.

Description

Technical Field

[0001] The present invention relates to the field of data analysis, and in particular to a classification model construction method and terminal equipment based on FLearnNSE-Pruned-Age (multi-classifier temporal integration of positive supplementation mechanism). Background Art

[0002] Due to the large number of typical application scenarios, such as weather forecasting, spam classification, user-differentiated product recommendations, etc., how to perform incremental classification mining on the gradually accumulated big data in non-stationary environments has received widespread attention. In this type of environment, the distribution probability of data generation is constantly changing, which may cause concept drift or even concept change, collectively referred to as non-stationary environments NSE (Non-Stationary Environments).

[0003] LearnNSE is an effective method for classification mining of accumulated big data in unstable environments. LearnNSE builds base classifiers for batches of data sets, considers the performance of each base classifier in all experienced environments, and then performs weighted integration and voting to construct the final classification model. The LearnNSE processing method has achieved remarkable results on a large number of generated data sets and real data sets, greatly improving the accuracy of classification predictions.

[0004] In order to be applicable to fields with high real-time requirements for big data classification mining, LearnNSE also provides a pruned version of the processing method LearnNSE-Pruned-Age. However, research has found that the pruning strategy used by the current pruned version of the LearnNSE-Pruned-Age method will lose the learned classification knowledge, lose the core LearnNSE ability to reuse the learned classification knowledge, and reduce the accuracy of the classification results. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a classification model construction method and terminal device based on FLearnNSE-Pruned-Age, aiming to improve the construction efficiency and classification accuracy of the classification model.

[0006] A classification model construction method based on FLearnNSE-Pruned-Age, comprising the steps of:

[0007] S1, parameter preset

[0008] A base classifier that makes up the ensemble classifier; parameters a and b are used to set the shape of the sigmoid weighting function; preset the base classifier capacity parameter ensembleSize; set the base classifier set current_experts that keeps track of changes in data, all base classifier sets all_experts, the selected base classifier set all_experts_chosen, and the upper limit of the capacity of the selected base classifier set chosenUpper;

[0009] S2, waiting for the tth training data set d to be processed t

[0010] Training dataset d t For {x t (i)∈X; y t (i)∈Y={1,...,c}}, i=1,...,m t ; where x t (i) represents the data point arriving at time t, X represents the different attributes of the data point, and y t (i) represents the category of the corresponding data point, Y represents different category labels, and m t represents the number of data points, c is the category of the data set, and i is the serial number of a data point in the training data set;

[0011] S3, determine whether the training data set has arrived. If the training data set has not arrived, return to S2; otherwise, determine whether t is equal to 1: When t = 1, the instance weight Penalty Weight are set to equal weight, execute S6; if t≠1, execute S4;

[0012] S4, calculate the ensemble classifier H composed of the base classifier set current_experts t-1 In the training data set d t The error rate E t , and the error rate E t Normalized, get B t ;

[0013] S5, using B t Update and normalize instance weights Calculate penalty weight

[0014] S6, using the training data set d t Train the base classifier and obtain the trained base classifier X→Y;

[0015] S7, if the ensemble classifier H t-1If the number of base classifiers retained in is greater than ensembleSize, then remove the integrated classifier H t-1 The base classifier with the smallest k in the set, adding the latest base classifier X→Y, construct H t ;

[0016] S8, in the training data set d t Calculate the base classifiers in all_experts based on the penalty weights The weighted error rate The error rate Normalized, we get

[0017] S9, for all base classifiers in the base classifier set all_experts, Sort the values ​​of from small to large, select the base classifier with the best weighted error rate from the beginning in all_experts to form all_experts_chosen for integration, and the number of base classifiers in all_experts_chosen is less than or equal to chosenUpper;

[0018] S10, calculate the voting weights of the selected base classifiers all_experts_chosen and the base classifiers in current_experts

[0019] S11, obtain the final integrated classification model;

[0020] S12, t=t+1, jump to S2.

[0021] The error rate E t for: Said And 0≤B t ≤1.

[0022] The instance weight for: The penalty weight

[0023] The weighted error rate if Then resample to generate the base classifier if but Said and

[0024] The optimal weighted error rate is:

[0025] The voting weight for: Where k′ is the subscript of the base classifier in current_experts and all_experts_chosen, indicating the generation time of the base classifier.

[0026] The integrated classification model is: For all base classifiers in current_experts and the selected base classifier all_experts_chosen.

[0027] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of the above-mentioned classification model building method.

[0028] The beneficial effects of the present invention are:

[0029] The present invention calculates the integrated classifier H composed of the current base classifier set current_experts t-1 In the training data set d t The error rate E t And normalized to get B t , by B t Update and normalize instance weights The penalty weight is calculated In the training data set d t Calculate the penalty-based weights of all base classifiers set all_experts The weighted error rate For all base classifiers in the base classifier set all_experts, The values ​​of are sorted from small to large, and the base classifier all_experts_chosen with the best weighted error rate is selected from the beginning in all_experts for integration. The present invention only calculates the integrated classifier H composed of the current base classifier set current_experts t-1 In the training data set d t The error rate E t It is not necessary to integrate all base classifiers to calculate the error rate, and the base classifier construction set all_experts_chosen with the optimal weighted error rate is selected for joint integration, which can realize the rapid and accurate construction of the classification model for the training data set and improve the construction efficiency and classification accuracy of the classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1This is a schematic diagram of the structure of the equipment for running the classification model building method based on FLearnNSE-Pruned-Age of the present invention;

[0031] Figure 2 This is a flow chart of the classification model construction method based on FLearnNSE-Pruned-Age according to the present invention;

[0032] FIG3(a) is a schematic diagram of a rotating chessboard (α=0°) in Example 1 of the classification model construction method of the present invention, and FIG3(b) is a schematic diagram of a rotating chessboard (α=45°) in Example 1 of the classification model construction method of the present invention;

[0033] Figure 4 This is a comparison chart of error rates of different pruning methods in Example 1 of the classification model construction method of the present invention;

[0034] Figure 5 This is a comparative analysis diagram of the error rates of Example 1 using the classification model construction method of the present invention and the LearnNSE method;

[0035] Figure 6 This is a comparative analysis chart of the running time of Example 1 using the classification model building method of the present invention and the LearnNSE method;

[0036] Figure 7 This is a comparison chart of error rates of different pruning methods in Example 2 of the classification model construction method of the present invention;

[0037] Figure 8 This is a comparative analysis diagram of the error rates of Example 2 using the classification model construction method of the present invention and the LearnNSE method;

[0038] Fig. 9 This is a comparative analysis chart of the running time of Example 2 using the classification model building method of the present invention and the LearnNSE method;

[0039] In the figure: 1001 - processor, 1002 - memory, 1003 - network module, 1004 - operating system, 1005 - parallel computing engine, 1006 - user interface, 1007 - communication line. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] Reference Figure 1 , Figure 1The device structure diagram of the hardware operating environment involved in the classification model construction method based on FLearnNSE-Pruned-Age of the present invention. The device involved in the embodiment of the present invention can be a terminal device with display and processing functions such as a PC (Personal Computer), a notebook computer, and a server.

[0042] like Figure 1 As shown, the terminal device may include: a processor 1001; a memory 1002; a network module 1003; an operating system 1004; a parallel computing engine 1005; a user interface 1006; and a communication line 1007. The processor 1001 may be a central processing unit (CPU), a graphics processing unit (GPU), etc. The memory 1002 may be a high-speed random access memory (RAM) or a stable memory (Non-Volatile Memory), such as a disk storage, a distributed file system (DFS), etc. The memory 1002 may optionally be a storage device independent of the aforementioned processor 1001. The network module 1003 may be a variety of network communication protocols, such as IPv4, IPv6, etc., and may optionally include a standard wired interface, a wireless interface (such as a wireless fidelity Wireless-Fidelity, i.e., a Wi-Fi interface). The operating system 1004 may be a variety of computer programs for managing computer hardware and software resources, such as windows, Linux, Unix, etc. The parallel computing engine 1005 can be a variety of environments that provide concurrent execution of task clusters, such as Spark engine, MapReduce engine, etc. The user interface 1006 can include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1006 can also include a standard wired interface, a wireless interface. Those skilled in the art can understand that Figure 1 The terminal shown in the figure does not constitute a limitation on the device for running the classification model building method, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0043] Continue to refer to Figure 1 , Figure 1 The memory 1002 as a computer storage medium may include a network module 1003, an operating system 1004, and a concurrent computing engine 1005. Figure 1In the embodiment, the network module 1003 can be used to connect servers and for cluster servers to communicate with each other; and the processor 1001 can be used to call the classification model building method stored in the memory 1002 and execute the process of the classification model building method provided in the embodiment of the present invention.

[0044] like Figure 2 As shown, a classification model construction method based on FLearnNSE-Pruned-Age specifically includes the following steps:

[0045] Step 1), parameter setting

[0046] A base classifier that makes up the ensemble classifier; set the parameters a and b of the sigmoid weighting function; the base classifier capacity ensembleSize; retain the current base classifier set current_experts that tracks changes in data, all base classifier sets all_experts, the selected base classifier set all_experts_chosen, and the selected base classifier set capacity upper limit chosenUpper.

[0047] Step 2), wait for the training data set d to be processed for the tth time t (Figure 3)

[0048] Training dataset d t Specifically, {x t (i)∈X; y t (i)∈Y={1,...,c}}, i=1,...,m t ; where x t (i) represents the data point arriving at time t, X represents the different attributes of the data point; y t (i) represents the category of the corresponding data point, and Y represents different category labels; m t Represents the number of data points; c is the category of the data set; i is the serial number of a data in the training data set.

[0049] Step 3), determine whether the training data set has arrived. If the training data set has not arrived, return to step 2), otherwise determine whether t is equal to 1. When t = 1, the instance weight Penalty Weight are set to equal weight, that is, for any i, Then execute step 6); if t≠1, execute step 4).

[0050] Step 4), calculate the integrated classifier H composed of the current base classifier set current_experts t-1 In the training data set d t The error rate on The error rate E t Normalized, we get

[0051] Step 5), update and normalize instance weights Calculate the penalty weight

[0052] Step 6), use the training data set d t Train the base classifier and obtain the trained base classifier X→Y.

[0053] Step 7), if the integrated classifier H t-1 If the number of base classifiers retained in is greater than ensembleSize, then remove the integrated classifier H t-1 The base classifier with the smallest k in the set, adding the latest base classifier X→Y, construct H t .

[0054] Step 8), in the training data set d t Calculate the base classifiers in all_experts based on the penalty weights The weighted error rate if Then resample to generate the base classifier if but The error rate Normalization,

[0055] Step 9), all base classifiers in the base classifier set all_experts are classified according to Sort the values ​​of from small to large, and select the optimal weighted error rate from the beginning in all_experts, that is The base classifiers form all_experts_chosen for integration, and the number of base classifiers in all_experts_chosen is less than or equal to chosen Upper.

[0056] Step 10), calculate the voting weights of the selected base classifiers all_experts_chosen and the base classifiers in current_experts The method is: k′ is the subscript of the base classifier in current_experts and all_experts_chosen, indicating the generation time of the base classifier; the base classifier that is not selected only records the classification error rate, and does not calculate the weighted error rate and weight, in order to improve execution efficiency.

[0057] Step 11), get the final integrated classification model For all base classifiers in current_experts and the base classifiers in all_experts_chosen.

[0058] Step 12), t=t+1, jump to step 2).

[0059] In the embodiment of the classification model construction method, different processing methods all follow the following provisions: (1) The accumulated big data is generated by an unstable environment, and the distribution and generation rules of the data are unknown in advance to each classification learning method. (2) The classification model construction methods all use the same base classifier (i.e., CART decision tree), and the parameters of CART are mergeleaves=on, minleaf=1, prune=on, surrogate=off; the parameters of the Sigmoid function are a=0.5, b=10; the number of retained base classifiers is ensemble Size=10, and the upper limit of the capacity of the selected base classifier all_experts_chosen is chosenUpper=10. (3) The processing methods all rely on the Spark engine to parallelize the parallelizable parts. (4) Each processing method is performed 10 times, and the classification error rate and processing time are recorded each time. (5) In order to reflect the effectiveness of the classification model construction method, the processing method newest, which only uses the latest data for training and prediction, is also included in the comparative analysis as one of the pruning methods.

[0060] Example 1

[0061] A non-Gaussian data set is extracted from a typical XOR problem. The data set represents a rotated checkerboard. The parameter α represents the rotation angle of the lower left corner of the rotated checkerboard in the sampling window. FIG3(a) is a rotated checkerboard with α=0°, and FIG3(b) is a rotated checkerboard with α=45°. The data set is used as the data set used in Example 1.

[0062] The classification problem of this data set is a typical XOR two-class classification problem, which was originally relatively simple. However, due to the addition of the rotation of the chessboard in this embodiment, a challenging classification problem has been formed. This classification problem is a classification problem in a non-stable periodic environment, and the classification scene repeats every π rotation angle. In order to increase the difference and complexity of each classification and get closer to the real environment, 10% noise data is added to the entire training data set. Each batch of data consists of 1024 instances extracted from the sampling window, and each test data set consists of 1024 data points with real class labels extracted from the same batch window at a resolution of 32*32. A total of 200 batches of data sets were extracted.

[0063] The 200 batches of data are stored in the memory, and the processor calls the classification model building method to read the 200 batches of data for processing to obtain the following processing results.

[0064] Table 1 Error rates of different processing methods on the Checkerboard dataset

[0065]

[0066] from Figure 4 As can be seen from the processing results shown, the classification model construction method of the present invention has achieved three pruning processing methods: the classification model construction method of only training the latest data set (newest), the LearnNSE pruned version processing method (LearnNSE-Pruned-Age) and the classification model construction method of the present invention (represented by FLearnNSE-Pruned-Age in the figure, Figure 5-9 It can be found that in the 101 to 200 batches of data generated by the chessboard rotation [π~2π], the classification model construction method of the present invention achieved a lower classification accuracy than the 1 to 100 batches of data generated by the rotation [0~π], while the newest and LearnNSE-Pruned-Age processing methods did not show such characteristics. This shows that the classification model construction method of the present invention has a core ability similar to the LearnNSE method in periodic classification scenarios, which is very important for reusing learned classification knowledge.

[0067] It is particularly noteworthy that Figure 5 As shown in the results in Table 1, it can be seen that on the Checkerboard dataset, the classification model construction method of the present invention achieves a lower classification error rate than the LearnNSE processing method, and has excellent classification performance.

[0068] Depend on Figure 6As shown in the figure, for different batches of data sets, the processing time of the classification learning of FLearnNSE-Pruned-Age and LearnNSE shows that when the batches of data sets increase, the processing time of the classification model construction method of the present invention is gradually less than that of the LearnNSE method, and with the increase of the batches of data sets, it shows an expanding trend, which reflects the advantages of the FLearnNSE-Pruned-Age processing method, which is an efficient and effective classification model construction method.

[0069] Example 2

[0070] The Weather dataset used in Example 2 is a long-term accumulated weather dataset, which is collected using sensors in hundreds of regions around the world. It contains weather characteristic attribute values ​​such as wind speed, temperature, humidity, and air pressure in these regions since 1930, as well as a classification label indicating whether it is raining. This dataset is a real-world dataset used to test the classification performance of different classification mining processing methods, such as accuracy, learning efficiency, etc.

[0071] A subset of the Weather dataset was selected for classification mining. This subset is a long-term, periodic, and accumulated large dataset. Attributes with a missing rate of more than 15% in the dataset were not selected, and the remaining 8 attributes were used as feature values ​​for the training set. The dataset contains a total of 18,159 records, of which 31% of the data has a category label of rain = yes and 69% of the data has a category label of rain = no.

[0072] The data set is divided into annual data sets, with 365 records forming one annual weather data set. Each time the data is processed, the first batch of data is used for training, followed by the second batch of data for testing, and so on, for a total of 47 batches of data sets analyzed.

[0073] The 47 batches of data are stored in the memory, and the processor calls the classification model building method to read the 47 batches of data for processing to obtain the following processing results.

[0074] from Figure 7The processing results shown show that for this real-world dataset, the FLearnNSE-Pruned-Age processing method achieved the lowest classification error rate among the three pruning processing methods. This shows that the pruning processing method is effective for both artificially generated datasets and real-world datasets. In addition, the classification error rate of the LearnNSE-Pruned-Age processing method is lower than that of the newest processing method, which reflects the advantages of the integrated method. The FLearnNSE-Pruned-Age processing method has a lower classification error rate than the LearnNSE-Pruned-Age method, which proves that the FLearnNSE-Pruned-Age processing method can overcome the deficiency of LearnNSE-Pruned-Age that it cannot reuse the learned classification knowledge, and is suitable for fast classification learning for long-term accumulated big data, and can further improve the accuracy of the classification model.

[0075] Table 2 Error rates of different processing methods on the Weather dataset

[0076]

[0077]

[0078] Depend on Figure 8 , Fig. 9 As shown in Table 2, in this embodiment, when the classification accuracy is very close to that of the LearnNSE processing method, the classification learning time of the FLearnNSE-Pruned-Age processing method is equivalent to 1 / 2 of that of the LearnNSE processing method, which reflects the advantages of this processing method. This result shows that the FLearnNSE-Pruned-Age processing method can be used as an efficient and effective pruning method of LearnNSE in application scenarios with high requirements for real-time classification learning, expanding the application scenarios of the time series classification mining processing method.

[0079] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, including a number of instructions for enabling a device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) and a cluster consisting of multiple devices to execute the methods described in each embodiment of the present invention.

[0080] It should be noted that, in the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0081] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A classification model construction method based on FLearnNSE-Pruned-Age, characterized in that: The device involved in the classification model construction method is a terminal device with display and processing functions, the terminal device comprises a processor (1001) and a memory (1002), the memory (1002) comprises a network module (1003), an operating system (1004) and a concurrent computing engine (1005), the processor (1001) calls and executes the classification model construction method stored in the memory (1002), and the classification model construction method comprises the steps of: S1, parameter preset A base classifier that makes up the ensemble classifier; parameters a and b are used to set the shape of the sigmoid weighting function; Preset the base classifier capacity parameter ensembleSize; set the base classifier set current_experts that keeps track of changes in data, all base classifier sets all_experts, the selected base classifier set all_experts_chosen, and the upper limit of the capacity of the selected base classifier set chosenUpper; S2, waiting for the tth training data set d to be processed t Training dataset d t For {x t (i)∈X; y t (i)∈Y={1,...,c}}, i=1,...,m t ; where x t (i) represents the data point arriving at time t, X represents the different attributes of the data point, and y t (i) represents the category of the corresponding data point, Y represents different category labels, and m t represents the number of data points, c is the category of the data set, and i is the serial number of a data point in the training data set; The training data set d t It is the Weather dataset; S3, determine whether the training data set has arrived. If the training data set has not arrived, return to S2; otherwise, determine whether t is equal to 1: When t = 1, the instance weight Penalty Weight are set to equal weight, execute S6; if t≠1, execute S4; S4, calculate the ensemble classifier H composed of the base classifier set current_experts t-1 In the training data set d t The error rate E t , and the error rate E t Normalized, get B t ; S5, using B t Update and normalize instance weights Calculate penalty weight S6, using the training data set d t Train the base classifier and obtain the trained base classifier S7, if the ensemble classifier H t-1 If the number of base classifiers retained in is greater than ensembleSize, then remove the integrated classifier H t-1 The base classifier with the smallest k in the set, adding the latest base classifier X→Y, construct H t ; S8, in the training data set d t Calculate the base classifiers in all_experts based on the penalty weight D i t The weighted error rate The error rate Normalized, we get S9, for all base classifiers in the base classifier set all_experts, Sort the values ​​of from small to large, select the base classifier with the best weighted error rate from the beginning in all_experts to form all_experts_chosen for integration, and the number of base classifiers in all_experts_chosen is less than or equal to chosenUpper; S10, calculate the voting weights of the selected base classifiers all_experts_chosen and the base classifiers in current_experts S11, obtain the final integrated classification model; S12, t=t+1, jump to S2.

2. The method for constructing a classification model of FLearnNSE-Pruned-Age according to claim 1, characterized in that: The error rate E t for: Said And 0≤B t ≤1.

3. The method for constructing a classification model of FLearnNSE-Pruned-Age according to claim 2, characterized in that: The instance weight for: The penalty weight 4. The method for constructing a classification model of FLearnNSE-Pruned-Age according to claim 1, characterized in that: The weighted error rate if Then resample to generate the base classifier if but Said and 5. The method for constructing a classification model of FLearnNSE-Pruned-Age according to claim 4, characterized in that: The optimal weighted error rate is:

6. The method for constructing a classification model of FLearnNSE-Pruned-Age according to claim 1, characterized in that: The voting weight for: where k ′ It is the subscript of the base classifier in current_experts and all_experts_chosen, indicating the generation time of the base classifier.

7. The method for constructing a classification model of FLearnNSE-Pruned-Age according to claim 6, characterized in that: The integrated classification model is: For all base classifiers in current_experts and the selected base classifier all_experts_chosen.

8. A terminal device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of the classification model building method according to any one of claims 1 to 7.

Images