A decision-level multi-model dynamic fusion classification method

Abstract

The application discloses a decision-level multi-model dynamic fusion classification method, and steps are as follows: establishing each base model and obtaining a data set containing samples and real classification categories, obtaining a predicted classification category through base model prediction, calculating classification accuracy in combination with the real categories and constructing a matrix; converting the predicted classification category into a one-hot encoding form to obtain a voting matrix; after filtering the constructed matrix, the contribution weight of each base model is calculated by using the TOPSIS method; the voting matrix and the weight are multiplied by a scalar to obtain an effective category voting weight matrix, and after element-wise Hadamard product, a two-dimensional matrix of the voting fusion classification category of all samples is generated through matrix addition, and finally the two-dimensional matrix is processed to obtain the fusion classification category. The application aims at the problem that the performance of the existing fusion classification method depends on the sample data quality, dynamically selects the advantage base model and the category which is good at prediction, and dynamically assigns the weight, so that more accurate fusion classification is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-model fusion, and particularly relates to a decision-level multi-model dynamic fusion classification method. BACKGROUND

[0002] Different models have different recognition accuracies for different class data of the same data set, that is, the recognition ability of the model for some class data is stronger, but the recognition ability of the model for other class data is weaker. For this problem, the general method is to construct a more complex model, optimize the parameters or optimize the algorithm, such as support vector machine, decision tree, evolutionary algorithm, swarm intelligence algorithm, etc. After 2010, with the popularization of high-performance computers and the improvement of hardware device performance, deep learning provides a new way to solve the problem, such as convolutional neural network, generative adversarial network, reinforcement learning and other artificial intelligence technologies, which have obtained good classification recognition effect. However, there are still some problems in a single classifier, such as the network structure design has not enough theoretical support, and only relies on the experience of the designer, which brings the sharp rise of the computing power demand and the model training time. And the performance of the single classifier is excessively dependent on the data size of the training sample, while in actual production, large-scale high-quality samples are often difficult to obtain, and the high-dimensional characteristics of the data lead to serious overfitting of the model.

[0003] To solve the above problems, ensemble learning represented by Random Forest, Adaboost, GBDT, LightGBM and their various improved algorithms is an effective fusion learning idea, that is, to combine multiple general base models in a certain way to produce a fusion prediction result to form a final decision. According to whether the fusion process changes with the characteristics of the sample to be classified, it can be divided into static fusion and dynamic fusion. Static fusion treats all samples equally. Too many base models not only increase the cost of storage space, running time and resources, but also lead to the dominance of base models with negative effects, resulting in incorrect fusion results. Compared with single base model and static fusion, dynamic fusion only needs to obtain several general base models to achieve high performance and effectively reduce training costs. Moreover, compared with static fusion, dynamic fusion can select part of the base models that are beneficial to fusion according to the characteristics of the sample to be classified, rather than directly fusing all base models, that is, dynamic selection. In addition, the classification accuracy of a single base model for different samples is different, and dynamic fusion can dynamically weight different samples according to the advantages of each base model. Although the current dynamic fusion technology can adjust the fusion structure according to different samples, it does not consider the influence of some low-confidence samples on the final decision when weighting the vote. For example, when the base model has a large weight, even if the confidence value of some samples is very low, the weighted final fusion result will still be dominated, which will greatly affect the decision and even lead to incorrect decision. How to avoid the influence of low-confidence samples on the final decision is also a difficulty that needs to be solved in current dynamic fusion technology. Based on "dynamic fusion", the present application studies a decision-level multi-model dynamic fusion classification method according to the characteristics of the base models to be fused. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application provides a decision-level multi-model dynamic fusion classification method, which aims to solve the problems in the background art.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a decision-level multi-model dynamic fusion classification method, comprising:

[0006] Step S1: Establishing each base model, obtaining different data sets of each base model, the different data sets containing samples and real classification categories, predicting the samples by each base model to obtain predicted classification categories, obtaining classification accuracy based on the predicted classification categories and the real classification categories, and constructing a classification accuracy matrix based on the classification accuracy;

[0007] Step S2: converting the predicted classification categories into one-hot encoding form, and calculating the voting matrix according to the classification accuracy matrix obtained in step S1;

[0008] Step S3: filtering the classification precision matrix obtained in step S1 to obtain a filtered precision matrix, and using the TOPSIS method to calculate the filtered precision matrix to obtain the contribution weight corresponding to each base model;

[0009] Step S4: performing scalar multiplication on the voting matrix in step S2 and the contribution weight corresponding to each base model in step S3 to obtain an effective class voting weight matrix of each base model;

[0010] Step S5: performing element-wise Hadamard product processing on the voting matrix and the effective class voting weight matrix to obtain an element-wise Hadamard product result, and performing matrix addition calculation on the element-wise Hadamard product result to obtain a voting fusion class two-dimensional matrix of all samples in the data set;

[0011] Step S6: processing the voting fusion class two-dimensional matrix to obtain the final fusion classification class of all samples in the data set.

[0012] Further, the specific process of step S1 is as follows:

[0013] m base models are established using machine learning or neural network algorithms; ;

[0014] m different data sets are selected for the base models; the different data sets contain k samples, and the different data sets contain n real classification classes, ; the different data sets are text data sets, image data sets, structured digital data sets, and unstructured data sets;

[0015] The samples in the different data sets are predicted by the m base models to obtain predicted classification classes, and the classification precision under different classification classes is obtained by combining the predicted classification classes with the real classification classes. ; is the classification precision of the mth base model for the nth real classification class;

[0016] Based on a classification precision matrix P is constructed, which is represented as:

[0017] ;

[0018] In the formula, is the classification precision of the first base model for the nth real classification class; is the classification precision of the second base model for the nth real classification class; is the classification precision of the mth base model for the first real classification class.

[0019] Further, the specific process of step S2 is as follows:

[0020] The predicted classification categories are converted into one-hot encoded forms, and the voting matrix P of the same dimension is calculated based on the classification accuracy matrix P obtained in step S1. ballots The specific process is as follows:

[0021] In the classification accuracy matrix P The corresponding position occupies 1 vote by default, totaling Ticket; when Below average accuracy hour, The corresponding position is recorded as 0 votes, when Higher than average accuracy hour, The corresponding position receives the vote, indicating:

[0022] ;

[0023] In the formula, This represents the total number of categories in the classification task.

[0024] Furthermore, the specific process of step S3 is as follows:

[0025] The classification accuracy matrix P obtained in step S1 is filtered using a standardized positive dimensionless evaluation index system to obtain the filtered accuracy matrix. For the filtered precision matrix The contribution weights of each base model are calculated using the approximation of the ideal solution sorting method.

[0026] The filtered precision matrix P' represents:

[0027] ;

[0028] In the formula, The filtered accuracy of the first base model on the first true classification category; The filtered accuracy of the first base model on the nth true classification category; The accuracy of the second base model after filtering on the first true classification category; The accuracy of the second base model after filtering on the nth true classification category; Let m be the filtered accuracy of the m-th base model on the first true classification category; Let m be the filtered accuracy of the m-th base model on the n-th true classification category;

[0029] Based on the filtered accuracy matrix Determine the optimal and worst solutions: Optimal solution It is by The worst-case scenario is composed of the maximum values ​​in each column. Depend on The minimum value in each column constitutes the structure; The optimal solution for the filtered accuracy of the m-th base model on the n-th true classification category; This represents the worst-performing solution for the m-th base model on the n-th true classification category after filtering.

[0030] Furthermore, the specific process for obtaining the contribution weights corresponding to each base model is as follows:

[0031] Calculation using the approximation of ideal solution sorting method and Euclidean distance and ,express:

[0032] ;

[0033] ;

[0034] In the formula, Let be the Euclidean distance from the m-th basis model to the optimal solution; In the filtered precision matrix, the first... The maximum value of each real classification category column; Let be the Euclidean distance from the m-th basis model to the worst solution; In the filtered precision matrix, the first... The minimum value of each true classification category;

[0035] Calculate the relative similarity between the m-th basis model and the optimal solution, expressed as:

[0036] ;

[0037] In the formula, The degree of similarity between the m-th base model and the optimal solution;

[0038] The contribution weights corresponding to each base model are obtained, and are represented as follows: ;

[0039] In the formula, This represents the contribution weight corresponding to the Mth base model.

[0040] Furthermore, the specific process of step S4 is as follows:

[0041] For the voting matrix P in step S2 ballots Perform scalar multiplication according to the dimension direction of the rows and the contribution weights corresponding to each base model in step 3 to obtain the effective class voting weight matrix of each base model, represented as follows:

[0042] S_ballotsm [ ×P ballots ];

[0043] wherein S_ballots m is the effective class voting weight matrix of the mth base model.

[0044] Further, the specific process of step S6 is: according to the voting fusion class two-dimensional matrix resultall in step S5, searching the prediction classification class corresponding to the maximum value of the coded element in the voting fusion class two-dimensional matrix resultall in the direction of dimension 1, obtaining the final fusion classification class labelall of all samples of the data set, which is expressed as:

[0045] ;

[0046] wherein, is a function of taking the index corresponding to the maximum value of the voting fusion class two-dimensional matrix; is the column dimension of the voting fusion class two-dimensional matrix.

[0047] Compared with the prior art, the present application has the following beneficial effects:

[0048] (1) The present application can dynamically select the classes that each base model is good at predicting for fusion, and dynamically assign weights to obtain the final fusion classification class, thereby improving the classification performance and robustness of each base model without increasing the data volume and the complexity of each base model, aiming at the problem that the performance of the current fusion classification method depends on the quality of the sample data.

[0049] (2) The method of the present application also has certain practical value for multi-source / multi-modal data fusion. The decision-level dynamic selection and dynamic weighting fusion evaluation system established can effectively solve the problem that the base models established are difficult to fuse and have low fusion precision due to the "inconsistent data scale" between data, and obtain the optimal fusion result. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is the flow chart of the method of the present application.

[0051] Figure 2 is the prediction accuracy graph of the lightweight deep learning model MobilenetV2 of the present application.

[0052] Figure 3 is the prediction accuracy graph of the high-efficiency convolutional neural network model EfficientnetB0 of the present application.

[0053] Figure 4 is the prediction accuracy graph of the lightweight convolutional neural network model ShuffleNetv2 of the present application.

[0054] Figure 5 The prediction accuracy graph for the lightweight neural network model MNASNet0.75 of the application.

[0055] Figure 6 The prediction accuracy graph for the MobileNetV3_Small model of the application.

[0056] Figure 7 The overall accuracy graph of the multi-model dynamic fusion of the application.

[0057] Figure 8 The schematic diagram of sample 1_265.bmp of the application. DETAILED DESCRIPTION

[0058] Case 1:

[0059] As shown in the Figure 1 application provides a technical solution: a decision-level multi-model dynamic fusion classification method, comprising:

[0060] Step S1: Establish each base model, obtain different data sets of each base model, different data sets contain samples and real classification categories, predict the samples through each base model to obtain predicted classification categories, obtain classification accuracy based on the predicted classification categories combined with the real classification categories, and construct a classification accuracy matrix based on the classification accuracy;

[0061] Step S2: Convert the predicted classification categories into one-hot encoding form, and calculate according to the classification accuracy matrix obtained in step S1 to obtain a voting matrix;

[0062] Step S3: Filter the classification accuracy matrix obtained in step S1 to obtain a filtered accuracy matrix, and calculate the filtered accuracy matrix using the TOPSIS method to obtain the contribution weight corresponding to each base model;

[0063] Step S4: Perform scalar multiplication on the voting matrix in step S2 and the contribution weight corresponding to each base model in step S3 to obtain an effective class voting weight matrix of each base model;

[0064] Step S5: Perform element-wise Hadamard product processing on the voting matrix and the effective class voting weight matrix to obtain the result of element-wise Hadamard product, and perform matrix addition calculation on the result of element-wise Hadamard product to obtain a voting fusion class two-dimensional matrix of all samples in the data set;

[0065] Step S6: Process the voting fusion class two-dimensional matrix to obtain the final fusion classification category of all samples in the data set.

[0066] The specific process of step S1 is:

[0067] m base models are respectively established using machine learning or neural network algorithms;

[0068] The m base models are single models participating in fusion and independently trained respectively; they can be machine learning algorithm models of different types (logistic regression, K nearest neighbor, support vector machine, random forest, etc.) or neural networks of different structures / parameters (multilayer perception MLP, convolutional neural network, recurrent neural network, Transformer, etc.);

[0069] The application adopts a lightweight deep learning model MobilenetV2, an efficient convolutional neural network model EfficientnetB0, a lightweight convolutional neural network model ShuffleNetv2, a lightweight neural network model MNASNet0.75 and a MobileNetV3_Small model as the base model; ;

[0070] Different data sets of the m base models are selected; the different data sets contain k samples, and the different data sets contain n real classification categories, The different data sets are text data sets (natural language classification scenarios), image data sets (visual classification scenarios), structured digital data sets (table classification scenarios) and unstructured data sets (special classification scenarios, audio data and time series data);

[0071] The samples in the different data sets are predicted by the m base models to obtain predicted classification categories, and the classification accuracy under different classification categories is obtained by combining the predicted classification categories with the real classification categories ; The classification accuracy of the mth base model for the nth real classification category;

[0072] Based on A classification accuracy matrix P is constructed, which is represented by:

[0073] ;

[0074] In the formula, The classification accuracy of the first base model for the nth real classification category; The classification accuracy of the second base model for the nth real classification category; The classification accuracy of the mth base model for the first real classification category.

[0075] The specific process of step S2 is:

[0076] The predicted classification categories are converted into one-hot encoding form, and a voting matrix P of the same dimension is obtained according to the classification precision matrix P obtained in step S1 ballots , and the specific process is as follows:

[0077] The classification precision matrix P is The corresponding position is by default 1 vote, and the total is When the average precision is low , the corresponding position is recorded as 0 votes, and when the average precision is high , the corresponding position gets votes, which means:

[0078] ;

[0079] In the formula, is the total number of categories in the classification task.

[0080] The specific process of step S3 is as follows:

[0081] The classification precision matrix P obtained in step S1 is subjected to standardization and dimensionless evaluation index system filtering to obtain a filtered precision matrix The filtered precision matrix is calculated using the Technique for Order Preference by Similarity to Ideal Solution (Topsis method) to obtain the contribution weight corresponding to each base model.

[0082] The filtered precision matrix P' is represented as:

[0083] ;

[0084] In the formula, is the filtered precision of the first base model on the first real classification category; is the filtered precision of the first base model on the nth real classification category; is the filtered precision of the second base model on the first real classification category; is the filtered precision of the second base model on the nth real classification category; is the filtered precision of the mth base model on the first real classification category; is the filtered precision of the mth base model on the nth real classification category;

[0085] Based on the filtered precision matrix , the optimal scheme and the worst scheme are determined: the optimal scheme is composed of the maximum value of each column in ; and the worst scheme​​​ is the minimum value in each column of the filtered precision rate matrix P ; is the optimal scheme of the mth base model on the nth real classification category after filtering; is the worst scheme of the mth base model on the nth real classification category after filtering.

[0086] The specific process of obtaining the contribution weight corresponding to each base model is as follows:

[0087] The Euclidean distance of and is calculated by the TOPSIS method and , indicating:

[0088] ;

[0089] ;

[0090] In the formula, is the Euclidean distance of the mth base model to the optimal scheme; is the maximum value of the nth real classification category column in the filtered precision rate matrix P ; is the Euclidean distance of the mth base model to the worst scheme; is the minimum value of the nth real classification category column in the filtered precision rate matrix P ;

[0091] The relative closeness of the mth base model to the optimal scheme is calculated, indicating:

[0092] ;

[0093] In the formula, is the relative closeness of the mth base model to the optimal scheme;

[0094] The contribution weight corresponding to each base model is obtained, indicating: ;

[0095] In the formula, is the contribution weight corresponding to the Mth base model.

[0096] The specific process of step S4 is as follows:

[0097] The voting matrix P ballots in step S2 is respectively multiplied by the contribution weight corresponding to each base model in the direction of the row dimension (i.e. the dimension 0 direction) to obtain the effective category voting weight matrix of each base model, indicating:

[0098] ​​S_ballots m =[ ×P ballots ];

[0099] In the formula, S_ballots m is the effective class voting weight matrix of the mth base model;

[0100] Wherein, the shape of the effective class voting weight matrix of the mth base model is an m*n matrix.

[0101] Wherein, the specific process of step S6 is:

[0102] According to the voting fusion class two-dimensional matrix resultall in step S5, the prediction classification class corresponding to the maximum value of the coded element in the voting fusion class two-dimensional matrix resultall is found in the direction of the dimension of 1, and the final fusion classification class labelall of the data set all samples is obtained, which is represented as:

[0103] ;

[0104] In the formula, is a function of taking the index corresponding to the maximum value of the voting fusion class two-dimensional matrix; is the column dimension of the voting fusion class two-dimensional matrix.

[0105] Implementation case 2:

[0106] Effect verification experiment of the decision-level multi-model dynamic fusion classification technology of the application

[0107] Wherein, the m base models include a lightweight deep learning model MobilenetV2, an efficient convolutional neural network model EfficientnetB0, a lightweight convolutional neural network model ShuffleNetv2, a lightweight neural network model MNASNet0.75, and a MobileNetV3_Small model;

[0108] Wherein, the n real classification categories include label0, label1, label2, label3, label4, and label5; label0 is a real classification category 0, label1 is a real classification category 1, label2 is a real classification category 2, label3 is a real classification category 3, label4 is a real classification category 4, and label5 is a real classification category 5;

[0109] After the lightweight deep learning model MobilenetV2 completes prediction on samples in different data sets, the classification accuracy under different classification categories is obtained by combining the predicted classification categories with the real classification categories, and the results are 98.3%, 91.7%, 100.0%, 98.3%, 100.0%, and 80.0% in turn; the overall accuracy of the lightweight deep learning model MobilenetV2 is 94.72%, as shown in Figure 2 .

[0110] After the efficient convolutional neural network model EfficientnetB0 completes prediction on samples in different data sets, the classification accuracy under different classification categories is obtained by combining the predicted classification categories with the real classification categories, and the results are 95.0%, 93.3%, 98.3%, 100.0%, 100.0%, and 88.3% in turn; the overall accuracy of the efficient convolutional neural network model EfficientnetB0 is 95.83%, as shown in Figure 3 .

[0111] After the lightweight convolutional neural network model ShuffleNetv2 completes prediction on samples in different data sets, the classification accuracy under different classification categories is obtained by combining the predicted classification categories with the real classification categories, and the results are 100.0%, 93.3%, 100.0%, 96.7%, 100.0%, and 96.7% in turn; the overall accuracy of the lightweight convolutional neural network model ShuffleNetv2 is 97.78%, as shown in Figure 4 .

[0112] After the lightweight neural network model MNASNet0.75 completes prediction on samples in different data sets, the classification accuracy under different classification categories is obtained by combining the predicted classification categories with the real classification categories, and the results are 100.0%, 98.3%, 100.0%, 100.0%, 100.0%, and 83.3% in turn; the overall accuracy of the lightweight neural network model MNASNet0.75 is 96.94%, as shown in Figure 5 .

[0113] After the MobileNetV3_Small model completes prediction on samples in different data sets, the classification accuracy under different classification categories is obtained by combining the predicted classification categories with the real classification categories, and the results are 98.3%, 90.0%, 100.0%, 100.0%, 100.0%, and 100.0% in turn; the overall accuracy of the MobileNetV3_Small model is 98.06%, as shown in Figure 6 .

[0114] wherein the percentage classification accuracy predicted by the lightweight deep learning model MobilenetV2 (98.3%, 91.7%, 100.0%, 98.3%, 100.0%, 80.0% as displayed in decimal form rounded off) is converted to the classification accuracy in the original decimal form: 0.9833, 0.9167, 1.0000, 0.9833, 1.0000, 0.8000.

[0115] The percentage classification accuracy predicted by the efficient convolutional neural network model EfficientnetB0 (95.0%, 93.3%, 98.3%, 100.0%, 100.0%, 88.3% as displayed in decimal form rounded off) is converted to the classification accuracy in the original decimal form: 0.9500, 0.9333, 0.9833, 1.0000, 1.0000, 0.8833.

[0116] The percentage classification accuracy predicted by the lightweight convolutional neural network model ShuffleNetv2 (100.0%, 93.3%, 100.0%, 96.7%, 100.0%, 96.7% as displayed in decimal form rounded off) is converted to the classification accuracy in the original decimal form: 1.0000, 0.9333, 1.0000, 0.9667, 1.0000, 0.9667.

[0117] The percentage classification accuracy predicted by the lightweight neural network model MNASNet0.75 (100.0%, 98.3%, 100.0%, 100.0%, 100.0%, 83.3% as displayed in decimal form rounded off) is converted to the classification accuracy in the original decimal form: 1.0000, 0.9833, 1.0000, 1.0000, 1.0000, 0.8333.

[0118] The percentage classification accuracy predicted by the MobileNetV3_Small model (98.3%, 90.0%, 100.0%, 100.0%, 100.0%, 100.0% as displayed in decimal form rounded off) is converted to the classification accuracy in the original decimal form: 0.9833, 0.9000, 1.0000, 1.0000, 1.0000, 1.0000.

[0119] Step S1: Construct the classification accuracy matrix P, which represents:

[0120] .

[0121] The predicted classification category in step S2 is converted to One-Hot form, which represents:

[0122] The prediction classification class 0 is converted to: [1, 0, 0, 0, 0, 0];

[0123] The prediction classification class 1 is converted to: [0, 1, 0, 0, 0, 0];

[0124] The prediction classification class 2 is converted to: [0, 0, 1, 0, 0, 0];

[0125] The prediction classification class 3 is converted to: [0, 0, 0, 1, 0, 0];

[0126] The prediction classification class 4 is converted to: [0, 0, 0, 0, 1, 0];

[0127] The prediction classification class 5 is converted to: [0, 0, 0, 0, 0, 1].

[0128] In step S2, the voting matrix is calculated, where m base models make 6 classifications of the prediction classification class, and each class has a default of 1 vote, i.e. the total number of votes is 6 votes;

[0129] The accuracy of the lightweight deep learning model MobilenetV2 on the test data of the 6 prediction classification classes is [0.9833, 0.9167, 1.0000, 0.9833, 1.0000, 0.8000], and the average classification accuracy of the lightweight deep learning model MobilenetV2 is: (0.9833+0.9167+1.0000+0.9833+1.0000+0.8000) / 6=0.9472,

[0130] The classification accuracy of the lightweight deep learning model MobilenetV2 on "label5" is 0.8000, which is lower than the average classification accuracy 0.9472, so the corresponding position "label5" is marked as 0 votes, and the remaining positions are evenly distributed with a total of 6 votes, i.e. 6 / 5 votes; Similarly, the voting matrix of the efficient convolutional neural network model EfficientnetB0, the lightweight convolutional neural network model ShuffleNetv2, the lightweight neural network model MNASNet0.75, and the MobileNetV3_Small model is obtained, i.e.

[0131] .

[0132] In step S3, the filtered accuracy matrix is calculated using the Technique for Order Preference by Similarity to Ideal Solution (Topsis method) to obtain the contribution weight corresponding to each base model;

[0133] The filtered accuracy matrix , which represents:

[0134] ;

[0135] The filtered precision matrix is ranked by the TOPSIS method The contribution weight corresponding to each base model is calculated as: [0.0671, 0.1713, 0.3035, 0.1600, 0.2981].

[0136] In step S4, the effective class voting weight matrix of each base model is calculated, specifically:

[0137] The effective class weight of the "lightweight deep learning model MobilenetV2" is: [6 / 5x0.0671, 6 / 5x0.0671, 6 / 5x0.0671, 6 / 5x0.06710, 6 / 5x0.0671, 0x0.0671] = [0.0806, 0.0806, 0.0806, 0.0806, 0.0806, 0,0000], and the effective class weight of the remaining base model can be similarly calculated.

[0138] The effective class weight of the "efficient convolutional neural network model EfficientnetB0" is: [6 / 5x0.1713, 6 / 5x0.1713, 6 / 5x0.1713, 6 / 5x0.1713, 6 / 5x0.1713, 0x0.1713] = [0.2055, 0.2055, 0.2055, 0.2055, 0.2055, 0.0000];

[0139] The effective class weight of the "lightweight convolutional neural network model ShuffleNetv2" is: [1x0.3035, 1x0.3035, 1x0.3035, 1x0.3035, 1x0.3035, 1x0.3035] = [0.3035, 0.3035, 0.3035, 0.3035, 0.3035, 0.3035];

[0140] The effective class weight of the "lightweight neural network model MNASNet0.75" is: [6 / 5x0.1600, 6 / 5x0.1600, 6 / 5x0.1600, 6 / 5x0.1600, 6 / 5x0.1600, 0x0.1600] = [0.1920, 0.1920, 0.1920, 0.1920, 0.1920, 0.0000];

[0141] The effective class weight of the "MobileNetV3_Small model" is: [1x0.2981, 1x0.2981, 1x0.2981, 1x0.2981, 1x0.2981, 1x0.2981] = [0.2981, 0.2981, 0.2981, 0.2981, 0.2981, 0.2981];

[0142] The effective class voting weight matrix of each base model is constructed, which is represented as:

[0143] .

[0144] In step S5, the voting fusion class two-dimensional matrix is calculated, specifically: (1) The actual class of sample 1_265.bmp (as shown in Figure 8 ) is "label1", that is, [0, 1, 0, 0, 0, 0];

[0145] The prediction of the lightweight deep learning model MobilenetV2 for sample 1_265.bmp is [0, 0, 0, 1, 0, 0], that is, "label3";

[0146] The prediction of the high-efficiency convolutional neural network model EfficientnetB0 for sample 1_265.bmp is [0, 1, 0, 0, 0, 0], that is, "label1";

[0147] The prediction of the lightweight convolutional neural network model ShuffleNetv2 for sample 1_265.bmp is [0, 0, 0, 0, 0, 1], that is, "label5";

[0148] The prediction of the lightweight neural network model MNASNet0.75 for sample 1_265.bmp is [0, 1, 0, 0, 0, 0], that is, "label1";

[0149] The prediction of the MobileNetV3_Small model for the sample is [0, 1, 0, 0, 0, 0], that is, "label1";

[0150] (2) The voting fusion class two-dimensional matrix is calculated

[0151] The voting matrix of the lightweight deep learning model MobilenetV2 is: [0.0806x0, 0.0806x0, 0.0806x0, 0.0806x1, 0.0806x0, 0x0] = [0.0000, 0.0000, 0.0000, 0.0806, 0.0000, 0.0000];

[0152] Similarly, it can be known that:

[0153] The voting matrix of the Efficient Convolutional Neural Network model EfficientnetB0 is: [0.2055x0, 0.2055x1, 0.2055x0, 0.2055x0, 0.2055x0, 0x0] = [0.0000, 0.2055, 0.0000, 0.0000, 0.0000, 0.0000].

[0154] The voting matrix of the lightweight Convolutional Neural Network model ShuffleNetv2 is: [0.3035x0, 0.3035x0, 0.3035x0, 0.3035x0, 0.3035x0, 0.3035x1] = [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3035].

[0155] The voting matrix of the lightweight neural network model MNASNet0.75 is: [0.1920x0, 0.1920x1, 0.1920x0, 0.1920x0.1920x0, 0x0] = [0.0000, 0.1920, 0.0000, 0.0000, 0.0000, 0.0000].

[0156] The voting matrix of the MobileNetV3_Small model is: [0.2981x0, 0.2981x1, 0.2981x0, 0.2981x0.2981x0, 0.2981x0] = [0.0000, 0.2981, 0.0000, 0.0000, 0.0000, 0.0000].

[0157] The voting matrix of the lightweight deep learning model MobilenetV2, the voting matrix of the Efficient Convolutional Neural Network model EfficientnetB0, the voting matrix of the lightweight Convolutional Neural Network model ShuffleNetv2, the voting matrix of the lightweight neural network model MNASNet0.75, and the voting matrix of the MobileNetV3_Small model are added to obtain the voting fusion class two-dimensional matrix: [[0.0000, 0.6956, 0.0000, 0.0806, 0.0000, 0.3035]].

[0158] In step S6, according to the voting fusion class two-dimensional matrix in step S5, the maximum value of the coding element in the voting fusion class two-dimensional matrix is found according to the direction with a dimension of 1, which represents:

[0159] The corresponding classification category

[0160] The voting fusion category two-dimensional matrix [[0.0000, 0.6956, 0.0000, 0.0806, 0.0000, 0.3035]] is encoded according to the direction with a dimension of 1, and the maximum value 0.6956 of the encoded element corresponds to the classification category of "label1"; therefore, the final prediction category of the sample 1_265.bmp of the multi-model dynamic fusion classification technology is "label1".

[0161] Similarly, the fusion prediction of the samples of the same data set can be performed to obtain the final fusion classification category of all samples of the data set; the multi-model dynamic fusion classification method is used for the test data corresponding to the six real classification categories, and the classification accuracy under each real classification category is 100.0%, 95.0%, 100.0%, 100.0%, 100.0% and 96.7% respectively; the overall accuracy of the multi-model dynamic fusion is 98.61%, as shown in the following table. Figure 7

[0162] Embodiment Case 3:

[0163] The method of the application also has certain practical value for multi-source / multi-modal data fusion. In order to verify the effectiveness of the multi-model dynamic fusion method, the MobileNetV3_Small model is used as a basic model in this embodiment case to classify the provided Milling Data Set tool milling multi-modal data set. The Milling Data Set tool milling multi-modal data set includes a data set of six modalities;

[0164] The data set of the six modalities includes AE-spindle spindle acoustic emission, AE-table platform acoustic emission, SmcAC alternating current spindle motor current, SmcDC direct current spindle motor current, Vib-spindle spindle vibration and Vib-table platform vibration.

[0165] In this embodiment case 3, the data set of the six modalities is used for experimental verification, and the VB value of tool wear (the VB value is the average width of the tool relief wear zone) is used as the 3-classification standard, in which 0~0.3mm normal wear is "label1", 0.3~0.4mm sharp wear is "label2", and 0.4mm blade collapse at any time is "label3". is normal wear; is sharp wear; is blade collapse at any time;

[0166] ​​​​The VB values of tool wear deletion are data missing NaN (run2, run3, run5, run16 with no measured VB value) and outliers (run8, run9, run10), totaling 13 pieces of 6 modal data; run2 is the 2nd experiment; run3 is the 3rd experiment; run5 is the 5th experiment; run16 is the 6th experiment; run8 is the 8th experiment; run9 is the 9th experiment; run10 is the 10th experiment;

[0167] The present embodiment uses the Gramian Angular Field (GAF) algorithm to convert the obtained 13 pieces of 6 modal data into 6 modal image data;

[0168] For AE-spindle acoustic emission, the precision of the MobileNetV3_Small model for 3 classification categories ( , , ) test data is respectively: 0.9000, 0.1500, 0.7750, and the overall accuracy of AE-spindle-MobileNetV3_Small is 0.7000; AE-spindle-MobileNetV3_Small is the experimental identification of the MobileNetV3_Small model for AE-spindle acoustic emission test;

[0169] For AE-table platform acoustic emission, the precision of the MobileNetV3_Small model for 3 classification categories ( , , ) test data is respectively: 0.9250, 0.8500, 0.4250; the overall accuracy of AE-table-MobileNetV3_Small is 0.7100;

[0170] AE-table-MobileNetV3_Small is the experimental identification of the MobileNetV3_Small model for AE-table platform acoustic emission;

[0171] For SmcAC AC spindle motor current, the precision of the MobileNetV3_Small model for 3 classification categories ( , , ) test data is respectively: 0.6000, 0.5500, 0.4250, and the overall accuracy of SmcAC-MobileNetV3_Small is 0.5200;

[0172] SmcAC-MobileNetV3_Small is the experimental identifier for the MobileNetV3_Small model of SmcAC AC spindle motor current.

[0173] For the SmcDC DC spindle motor current, the MobileNetV3_Small model classifies three categories ( , , The accuracy rates of the test data are 1.0000, 0.3000, and 0.7250, respectively, and the overall accuracy rate of SmcDC-MobileNetV3_Small is 0.7500.

[0174] SmcDC-MobileNetV3_Small is the experimental identifier for the MobileNetV3_Small model of SmcDC DC spindle motor current.

[0175] For Vib-spindle spindle vibration, the MobileNetV3_Small model classifies three categories ( , , The accuracy rates of the test data are 0.7750, 0.3000, and 0.5750, respectively, and the overall accuracy rate of Vib-spindle-MobileNetV3_Small is 0.6000.

[0176] Vib-spindle-MobileNetV3_Small is the experimental identifier for the MobileNetV3_Small model for vibration of the Vib-spindle spindle.

[0177] For Vib-table platform vibration, the MobileNetV3_Small model classifies three categories ( , , The accuracy rates of the test data were 0.0750, 0.4500, and 0.8500, respectively, and the overall accuracy rate of Vib-table-MobileNetV3_Small was 0.4600.

[0178] Vib-table-MobileNetV3_Small is an experimental identifier for the MobileNetV3_Small model of vibration on the Vib-table platform.

[0179] The AE-spindle-MobileNetV3_Small, AE-table-MobileNetV3_Small, SmcAC-MobileNetV3_Small, SmcDC-MobileNetV3_Small, Vib-spindle-MobileNetV3_Small, and Vib-table-MobileNetV3_Small are included as V-based models, and the six models are included as V-based models.

[0180] Step S1: Obtain the classification accuracy matrix of the V-based model under the classification task, denoted as , which represents

[0181] ;

[0182] In the formula, is the classification accuracy matrix of the V-based model.

[0183] In step S2, the , , is calculated, and the voting matrix is calculated. The specific process is as follows:

[0184] (1) Convert , , to One-Hot encoding, that is,

[0185] is converted to: [1, 0, 0];

[0186] is converted to: [0, 1, 0];

[0187] is converted to: [0, 0, 1];

[0188] (2) Calculate the voting matrix, wherein the three classifications of , , are classified by the V-based model, and each category has one vote by default, that is, the total number of votes is 3. When the classification accuracy of the V-based model is lower than the average accuracy, it is recorded as 0 votes, and the remaining positions are evenly distributed to the total number of votes (3), which represents

[0189] ;

[0190] In the formula, is the voting matrix of the V-based model.

[0191] Step S3: Filter to obtain the filtered The contribution weight corresponding to the V-base model is calculated by using the TOPSIS method.

[0192] The filtered , represents:

[0193] ;

[0194] The filtered is calculated by using the TOPSIS method, and the contribution weight corresponding to the V-base model is [0.2024, 0.3021, 0.0000, 0.2040, 0.1250, 0.1664].

[0195] Step S4: Calculate the effective category voting weight matrix of the V-base model, specifically:

[0196] The effective category weight of "AE-spindle-MobileNetV3_Small" is [3 / 2x0.2024, 0x0.2024, 3 / 2x0.2024] = [0.3036, 0.0000, 0.3036];

[0197] Similarly, it can be known that

[0198] The effective category weight of "AE-table-MobileNetV3_Small" is [3 / 2x0.3021, 3 / 2x0.3021, 0x0.3021] = [0.4532, 0.4532, 0.0000];

[0199] The effective category weight of "SmcAC-MobileNetV3_Small" is [3 / 2x0.0000, 3 / 2x0.0000, 0x0.0000] = [0.0000, 0.0000, 0.0000];

[0200] The effective category weight of "SmcDC-MobileNetV3_Small" is [3 / 2x0.2040, 0x0.2040, 3 / 2x0.2040] = [0.3060, 0.0000, 0.3060];

[0201] The effective category weight of "Vib-spindle-MobileNetV3_Small" is [3 / 2x0.1250, 0x0.1250, 3 / 2x0.1250] = [0.1875, 0.0000, 0.1875];

[0202] The effective class weights for “Vib-table-MobileNetV3_Small” are: [0×0.1664, 0×0.1664, 3×0.1664] = [0.0000, 0.0000, 0.4993];

[0203] Among them, the effective class voting weight matrix of the V-basis model ,express:

[0204] .

[0205] Step S5: (1) Calculate the two-dimensional matrix of voting fusion categories, specifically:

[0206] AE-spindle-MobileNetV3_Small pair , , The predicted voting matrix is: [0.3036, 0.0000, 0.0000], [0.3036, 0.0000, 0.0000], [0.3036, 0.0000, 0.0000];

[0207] AE-table-MobileNetV3_Small , , The predicted voting matrix is: [0.4532, 0.0000, 0.0000], [0.4532, 0.0000, 0.0000], [0.4532, 0.0000, 0.0000];

[0208] SmcAC-MobileNetV3_Small , , The predicted voting matrix is: [0.0000, 0.0000, 0.0000], [0.0000, 0.0000, 0.0000], [0.0000, 0.0000, 0.0000];

[0209] SmcDC-MobileNetV3_Small , , The predicted voting matrix is: [0.3060, 0.0000, 0.0000], [0.3060, 0.0000, 0.0000], [0.3060, 0.0000, 0.0000];

[0210] Vib-spindle-MobileNetV3_Small pair , , The predicted voting matrix is: [0.1875, 0.0000, 0.0000], [0.0000, 0.0000, 0.1875], [0.1875, 0.0000, 0.0000];

[0211] Vib-table-MobileNetV3_Small , , The predicted voting matrix is: [0.0000, 0.0000, 0.4993], [0.0000, 0.0000, 0.4993], [0.0000, 0.0000, 0.4993].

[0212] (2) Calculate the voting fusion class two-dimensional matrix

[0213] The predicted voting matrix of AE-spindle-MobileNetV3_Small, the predicted voting matrix of AE-table-MobileNetV3_Small, the predicted voting matrix of SmcAC-MobileNetV3_Small, the predicted voting matrix of SmcDC-MobileNetV3_Small, the predicted voting matrix of Vib-spindle-MobileNetV3_Small, and the predicted voting matrix of Vib-table-MobileNetV3_Small are fused to obtain the predicted voting matrix of the V-based model: [[1.2504, 0.0000, 0.4993], [1.0629, 0.0000, 0.6867], [1.2504, 0.0000, 0.4993]].

[0214] Wherein, step S6: according to the predicted voting matrix of the V-based model in step S5, the predicted classification category corresponding to the maximum value of the coding element in the predicted voting matrix of the V-based model is found in the direction with a dimension of 1;

[0215] The predicted voting matrix of the V-based model [[1.2504, 0.0000, 0.4993], [1.0629, 0.0000, 0.6867], [1.2504, 0.0000, 0.4993]] in the direction with a dimension of 1, the classification categories corresponding to the maximum values of the coding elements 1.2504, 1.0629, and 1.2504 are respectively “ ”, ”, ”; Therefore, the final prediction categories of the multi-model dynamic fusion classification technology are “ ”, ”. with the actual category of the sample.

[0216] Similarly, AE-spindle-MobileNetV3_Small 0.7000, AE-table-MobileNetV3_Small 0.7100, SmcAC-MobileNetV3_Small 0.5200, SmcDC-MobileNetV3_Small 0.7500, Vib-spindle-MobileNetV3_Small 0.6000, Vib-table-MobileNetV3_Small 0.4600 are fused to obtain the final fusion classification category of 6 modal image data The overall accuracy of the multi-model dynamic fusion is 0.7700, which is higher than that of the other six base models: AE-spindle-MobileNetV3_Small 0.7000, AE-table-MobileNetV3_Small 0.7100, SmcAC-MobileNetV3_Small 0.5200, SmcDC-MobileNetV3_Small 0.7500, Vib-spindle-MobileNetV3_Small 0.6000, Vib-table-MobileNetV3_Small 0.4600.

[0217] It can be seen that the multi-model dynamic fusion of the present application is not simply to directly delete the V base model with low overall accuracy, but to dynamically select and dynamically weight the advantages of the V base model in different categories of data. For example, the precision of “Vib-table-MobileNetV3_Small” on the , , The precision of the test data of the category is: 0.0750, 0.4500, 0.8500, and the overall accuracy of Vib-table-MobileNetV3_Small is only 0.4600, which is much lower than that of the remaining five base models. However, since Vib-table-MobileNetV3_Small has a higher accuracy (0.8500) in predicting “ ”, it is given a contribution weight of 0.1664 and an effective category weight of: [0.0000, 0.0000, 0.4993] to predict the category of “ ”.

[0218] Although the overall accuracy of the base model "SmcAC-MobileNetV3_Small" is 0.5200, which is much higher than the accuracy 0.4600 of "Vib-table-MobileNetV3_Small", the contribution weight and effective class weight of 0.0000 assigned to it are: [0.0000, 0.0000, 0.0000], and it is eliminated from the fusion system.

[0219] Therefore, the decision-level multi-model dynamic fusion classification method of the present application can effectively solve the problem that the base model established is difficult to fuse and the fusion precision is low due to the "data scale inconsistency" between data, and obtain the optimal fusion result.

[0220] Although embodiments of the present application have been shown and described, it will be understood by those having ordinary skill in the art that various changes, modifications, substitutions and alterations can be made hereto without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A decision-level multi-model dynamic fusion classification method, characterized in that, Comprise: Different data sets include AE-spindle spindle acoustic emission, AE-table platform acoustic emission, SmcAC AC spindle motor current, SmcDC DC spindle motor current, Vib-spindle spindle vibration, Vib-table platform vibration; Step S1: establish each base model, obtain different data sets of each base model, different data sets contain samples and real classification categories, predict the samples by each base model to obtain predicted classification categories, obtain classification accuracy based on the predicted classification categories combined with the real classification categories, and construct a classification accuracy matrix based on the classification accuracy; Step S2: convert the predicted classification categories into one-hot encoding form, and calculate according to the classification accuracy matrix obtained in step S1 to obtain a voting matrix; Step S3: filter the classification accuracy matrix obtained in step S1 to obtain a filtered accuracy matrix, and calculate the filtered accuracy matrix using the TOPSIS method to obtain the contribution weight corresponding to each base model; Step S4: perform scalar multiplication on the voting matrix in step S2 and the contribution weight corresponding to each base model in step S3 to obtain an effective class voting weight matrix of each base model; Step S5: perform element-wise Hadamard product processing on the voting matrix and the effective class voting weight matrix to obtain the result of element-wise Hadamard product, and perform matrix addition calculation on the result of element-wise Hadamard product to obtain a voting fusion class two-dimensional matrix of all samples in the data set; Step S6: process the voting fusion class two-dimensional matrix to obtain the final fusion classification category of all samples in the data set; The specific process of step S1 is: The m base models are established by using a lightweight deep learning model MobilenetV2, an efficient convolutional neural network model EfficientnetB0, a lightweight convolutional neural network model ShuffleNetv2, a lightweight neural network model MNASNet0.75, and a MobileNetV3_Small model. ; selecting a different data set of m base models; the different data set contains k samples, the different data set contains n real classification categories; ; The samples in different data sets are predicted by m base models to obtain predicted classification categories, and the classification accuracy under different classification categories is obtained by combining the predicted classification categories with the real classification categories ; is the classification accuracy of the mth base model for the nth real classification category Based on A classification precision matrix P is constructed, denoted by: ； In the formula, is the classification accuracy of the first base model for the n-th real classification category; is the classification accuracy of the second base model for the n-th real classification category; is the classification accuracy of the m-th base model for the first real classification category; The specific process of step S2 is: The predicted classification category is converted into a one-hot encoding form, and a voting matrix P of the same dimension is obtained according to the classification precision matrix P obtained in step S1 ballots The specific process is as follows: In the classification accuracy matrix P The corresponding position occupies 1 vote by default, totaling Ticket; when Below average accuracy hour, The corresponding position is recorded as 0 votes, when Higher than average accuracy hour, The corresponding position receives the vote, indicating: ； In the formula, is the total number of classes in the classification task.

2. The method of claim 1, wherein the method is a decision-level multi-model dynamic fusion classification method. The specific process of step S3 is: The classification precision rate matrix P obtained in step S1 is subjected to a normalized positive dimensionless evaluation index system filtering process to obtain a filtered precision rate matrix The filtered precision rate matrix is subjected to a standardization process to obtain a standardization precision rate matrix The contribution weight corresponding to each base model is calculated using the approximation ideal solution ranking method. The filtered accuracy matrix P' is represented as: ； wherein, is the filtered precision of the 1st base model on the 1st real class category; is the filtered precision of the 1st base model on the n-th real class category; is the filtered precision of the 2nd base model on the 1st real class category; is the filtered precision of the 2nd base model on the n-th real class category; is the filtered precision of the m-th base model on the 1st real class category; is the filtered precision of the m-th base model on the n-th real class category; based on the filtered precision matrix determining the optimal and worst-case scenarios: optimal scenario is formed by the maximum value in each column of ; worst-case scenario is formed by the minimum value in each column of ; is the filtered precision optimal scenario of the mth base model on the n th real classification class; is the filtered precision worst-case scenario of the mth base model on the n th real classification class.

3. The method of claim 2, wherein the method is a decision-level multi-model dynamic fusion classification method. The specific process of obtaining the contribution weight corresponding to each base model is: The Euclidean distance of and by the technique of Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) and is represented by: ； ； In the formula, is the Euclidean distance from the mth base model to the optimal scheme; is the maximum value of the mth real classification category column in the filtered precision matrix; is the minimum value of the mth real classification category column in the filtered precision matrix; is the Euclidean distance from the mth base model to the worst scheme; is the maximum value of the mth real classification category column in the filtered precision matrix; is the minimum value of the mth real classification category column in the filtered precision matrix; Calculate the relative closeness of the mth base model to the optimal scheme, represented as: ； In the formula, is the relative closeness degree of the mth base model to the optimal scheme. The contribution weight corresponding to each base model is obtained, and is represented as: ; In the formula, is the contribution weight corresponding to the Mth base model.

4. The method of claim 3, wherein the method is a decision-level multi-model dynamic fusion classification method. The specific process of step S4 is: The voting matrix P in step S2 ballots The effective class voting weight matrix of each base model is obtained by performing scalar multiplication on the dimension direction of the row and the contribution weight corresponding to each base model in step 3, and is expressed as: S_ballots m = [ ×P ballots ] S_ballots = S_ballots + S_ballots m S_ballots is the effective class voting weight matrix of the mth base model.

5. The method of claim 4, wherein the step of dynamically fusing the classification results of the multiple models comprises: determining a weight of each model according to a performance of each model; and fusing the classification results of the multiple models according to the weights of the multiple models. The specific process of step S6 is: According to the voting fusion class two-dimensional matrix resultall in step S5, find the maximum value of the coding element in the voting fusion class two-dimensional matrix resultall in the direction of dimension 1, and obtain the final fusion classification category labelall of all samples in the data set, represented as: ； In the formula, is a function of taking the index corresponding to the maximum value of the two-dimensional matrix of the voting fusion category; is the column dimension of the two-dimensional matrix of the voting fusion category.

Images