Disease detection system based on width ensemble learning

Abstract

The application discloses a disease detection system based on width integrated learning, comprising: a data acquisition module, which pre-processes medical data to obtain input samples; a primary classification module, which sequentially weights the input samples according to classification errors, improves integrated diversity, uses a lightweight width learning network as a primary classifier, trains the weighted samples, uses an incremental learning mode of the width learning network, selectively integrates excellent primary classifiers for prediction, and improves integrated accuracy; a secondary classification module, which inputs the prediction result of the primary classifier into a secondary classifier as enhanced features, adaptively adds input samples obtained from the data acquisition module according to the average prediction accuracy of the primary classifier, trains the secondary classifier, and obtains a final disease detection result to assist doctors in diagnosis and treatment. The application can early prevent and detect diseases through a machine learning method, and reduce the burden on the medical system and society.

Description

Technical Field

[0001] The present invention relates to the technical field of disease detection, and in particular to a disease detection system based on wide ensemble learning. Background Art

[0002] Research in disease detection systems using machine learning is very active. Machine learning methods can automatically analyze patient data and assist doctors in providing treatment, playing a significant role in improving the accuracy, speed, and efficiency of early diagnosis of various diseases. Ensemble learning integrates multiple machine learning methods through a strategy, improving decision accuracy through group decision-making. Ensemble learning, based on traditional machine learning and deep learning methods, is widely used in disease detection. However, traditional machine learning methods rely on prior knowledge and feature engineering, resulting in limited feature extraction capabilities and limited model generalization. Most traditional machine learning methods are based on linear assumptions, making them difficult to handle nonlinear problems and resulting in weak model fitting capabilities. Deep learning methods, on the other hand, extract rich and abstract features for disease detection by increasing the number of neural network layers. However, increasingly deeper neural network structures increase the computational resource burden and computational time overhead, and the gradient-based weight update method can lead to vanishing and exploding gradient problems. Furthermore, deep learning requires massive amounts of data for training, which can lead to overfitting with smaller datasets and poor generalization.

[0003] Significant progress has been made in the field of machine learning for disease detection systems, but existing methods struggle to achieve good detection results with minimal training cost. Wide learning, a single-hidden-layer neural network, boasts a lightweight network structure that doesn't rely on excessive computational resources. Its use of pseudo-inverse to directly calculate weights enables fast and efficient training. Its linear mapping of feature nodes and nonlinear mapping of augmented nodes provide the network with powerful fitting capabilities. To address this, a wide ensemble learning method was designed, using a lightweight wide learning network as the primary classifier. Ensemble learning methods were then used to improve the wide learning network's disease detection capabilities and generalization performance. The higher the accuracy and the greater the diversity, the better the ensemble effect. Summary of the Invention

[0004] The present invention aims to overcome the shortcomings and deficiencies of existing technologies by proposing a disease detection system based on wide ensemble learning. This system uses a wide learning network as a primary classifier to extract linear and nonlinear features from samples, providing high precision for ensemble learning. By leveraging the incremental model of the wide learning network, selective ensemble learning techniques are used to select excellent primary classifiers, further improving the accuracy of ensemble learning. Samples are sequentially weighted according to their classification difficulty to increase the diversity of ensemble learning. The predictions of the primary classifiers are used as enhanced features to input into secondary classifiers, allowing for multi-level ensemble training to improve the generalization of the ensemble model. Furthermore, the lightweight wide learning network reduces training difficulty and improves disease detection efficiency.

[0005] To achieve the above objectives, the present invention provides a technical solution: a disease detection system based on wide ensemble learning, comprising:

[0006] A data acquisition module is used to acquire a medical data set containing multiple medical samples and normalize the samples to obtain input samples of uniform size;

[0007] The primary classification module is used to optimize the samples of ensemble learning, average the weights of the initialized samples, weight the input samples, and use n primary classifiers to train the weighted samples. The primary classifier is a width learning network. When the accuracy of the primary classifier is insufficient, the incremental mode of the width learning network is used to improve the accuracy of the primary classifier by increasing the number of hidden layer nodes. The primary classifier with excellent performance is selected to predict the weighted samples to improve the accuracy of the ensemble learning. In the prediction process, for the prediction results of the previous primary classifier, the samples are weighted according to the error of each sample, the weight of the sample with incorrect prediction is increased, and the weight of the sample with correct prediction is reduced, so that the next primary classifier pays more attention to the difficult-to-classify samples. The input samples of different primary classifiers are weighted differently to improve the diversity of ensemble learning. High accuracy and rich diversity can bring performance improvements to the width ensemble network. The n primary classifiers are trained iteratively in a loop to finally obtain the prediction results and accuracy of the n primary classifiers.

[0008] The secondary classification module uses a classifier with learning ability to merge the prediction results of each primary classifier, optimizes the features of the integrated learning, and generates the final prediction results; wherein, the classifier with learning ability is called a secondary classifier, the prediction results of n primary classifiers are merged and input as enhanced features into the secondary classifier for further training, and the average accuracy of the n primary classifiers is calculated. When the average accuracy is less than the set threshold, the samples obtained in the data acquisition module are input into the secondary classifier together as the original samples. Otherwise, only the prediction results of the merged primary classifiers are input into the secondary classifier, and the computational complexity of the secondary classifier is adaptively adjusted to improve the training efficiency. The integrated output obtained after the secondary classifier training is used as the final disease detection result.

[0009] Furthermore, the data acquisition module constrains the sample features of each dimension to the same order of magnitude, thereby balancing the contribution of each feature.

[0010] Furthermore, the specific operation steps of the primary classification module are as follows:

[0011] 1) Initialize the sample weight w. Assume that there are m samples in the medical dataset and the weight of each sample is initialized to Taking the width learning network as the primary classifier, the sample set weight w1 of the first primary classifier G1 is:

[0012]

[0013] Where w 1,i , is the i-th sample;

[0014] 2) At the kth ensemble learning iteration, according to the kth primary classifier G k The sample weight w k,i Weight the original sample X, and k In the sample distribution X k Classify in:

[0015] X k =w k,i X; i=1,2,...,m; k=1,2,...,n

[0016] 3) X k Enter G k Train and get the prediction results

[0017]

[0018] 4) Calculate G k The classification error rate c k :

[0019]

[0020] where Y is the sample label, I is the indicator function,

[0021] 5) Set the precision threshold c, when c k < c, start the incremental mode of the width learning network, improve the precision of the primary classifier by increasing the number of hidden layer nodes, without retraining the primary classifier, update the prediction results of the kth primary classifier after increment Improve the efficiency of selective integration; repeat step 4), calculate the G k classification error rate c k of the primary classifier after increment until c k ≥ c, select the primary classifier that reaches the precision threshold for subsequent integration;

[0022] 6) For the ith sample, calculate the G classification error rate corresponding to its prediction result k

[0023]

[0024] where Y i is the label of the ith sample;

[0025] 7) Assuming the number of sample categories is R, calculate the base classifier coefficient G of the ith sample according to the classification error rate k

[0026]

[0027] 8) According to for the subsequent k = 2, 3, …, n primary classifiers, give the ith sample a weight w k,i in the kth iteration, increase the weight of the classification error sample, reduce the weight of the correct sample, so that the next primary classifier can correct some classification error samples in the previous primary classifier according to the error degree of the sample prediction, so that different primary classifiers complement each other:

[0028]

[0029] where w k-1,i is the weight given to the ith sample in the k-1th iteration, αk-1 is the k-1th base classifier coefficient, is its prediction result, e is the exponential function, Z k-1 is the normalization factor, c k-1 ​​​​is the classification error rate in the k-1th iteration;

[0030] 9) In the classification error rate c k When it reaches 0 or the number of primary classifiers reaches the upper limit n, stop building the primary classifier; repeat iterative steps 2) to 8), train k=2, 3, ..., n primary classifiers, and obtain the prediction results of a total of n primary classifiers

[0031] Furthermore, the specific operation steps of the secondary classification module are as follows:

[0032] 1) Combine the prediction results of n primary classifiers obtained in the primary classification module into the result Calculate the average precision of n primary classifiers

[0033]

[0034] 2) Adaptively adjust the input of the secondary classifier F according to the average accuracy of n primary classifiers, set the accuracy threshold ACC, if Then the original sample X is combined with splicing, input the secondary classifier F, train and get the integrated output Y′ as the final disease detection result; otherwise, only Input the secondary classifier F and train it to obtain the integrated output Y′ as the final disease detection result;

[0035]

[0036] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0037] 1. This paper designs a selective integration method that can achieve efficient disease detection at a low computational cost.

[0038] 2. Common sequence weighting methods update the weight of each sample with the same magnitude. This paper designs a sample sequence weighting method. Based on the prediction results of the previous primary classifier, the weight of the samples with incorrect predictions is dynamically increased and the weight of the samples with correct predictions is reduced according to the degree of prediction error of different samples. This allows the subsequent primary classifier to pay more attention to difficult-to-classify samples, improves the diversity of ensemble learning, and thus improves the performance of ensemble learning.

[0039] 3. Common selective ensemble methods require screening multiple trained classifiers, removing poorly performing classifiers, and retraining higher-performing classifiers. This paper designs a selective ensemble method that, through an incremental model of a width learning network, improves the accuracy of the primary classifier and reduces classifier loss without requiring model retraining, ultimately increasing the accuracy of ensemble learning with less time.

[0040] 4. This invention designs a multi-layer ensemble method. The prediction results of the primary classifier are used as enhanced features to input into the secondary classifier, further enhancing the generalization capability of the ensemble learning. Based on the average accuracy of the primary classifier, the original input samples are adaptively added to the secondary classifier, adaptively adjusting the computational complexity of the secondary classifier to improve training efficiency.

[0041] In summary, the present invention can improve the diversity of ensemble learning through the sequence weighting method, efficiently improve the accuracy of ensemble learning through the selective ensemble method, and further enhance the generalization ability of ensemble learning by adding enhanced features to train secondary classifiers, thereby improving the efficiency and accuracy of disease detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a framework diagram of the system of the present invention. DETAILED DESCRIPTION

[0043] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0044] like Figure 1 As shown, this embodiment discloses a disease diagnosis system based on width ensemble learning, including the following functional modules:

[0045] The data acquisition module is used to obtain a medical data set containing multiple medical samples, normalize the samples to obtain input samples of uniform size, and constrain the sample features of each dimension to the same order of magnitude, thereby balancing the contribution of each feature.

[0046] The primary classification module is used to optimize the samples of ensemble learning. The average initialized sample weights are used to weight the input samples. The weighted samples are trained using n primary classifiers, and the primary classifier is a broad learning network (BLS). When the accuracy of the primary classifier is insufficient, the incremental mode of the broad learning network is used to improve the accuracy of the primary classifier by increasing the number of hidden layer nodes. Primary classifiers with excellent performance are selected to predict the weighted samples, thereby improving the accuracy of ensemble learning. During the prediction process, for the prediction results of the previous primary classifier, the samples are weighted according to the error of each sample, the weight of the sample with incorrect prediction is increased, and the weight of the sample with correct prediction is reduced, so that the next primary classifier pays more attention to the samples that are difficult to classify. The input samples of different primary classifiers are weighted differently to improve the diversity of ensemble learning. High accuracy and rich diversity can bring performance improvements to the wide ensemble network. The n primary classifiers are trained iteratively in a loop, and the prediction results and accuracy of n primary classifiers are finally obtained.

[0047] The secondary classification module uses a classifier with learning capabilities to merge the prediction results of each primary classifier, optimizes the features of the integrated learning, and generates the final prediction results. Among them, the classifier with learning capabilities is called a secondary classifier. The prediction results of n primary classifiers are merged and input as enhanced features into the secondary classifier for further training. The average accuracy of the n primary classifiers is calculated. When the average accuracy is less than the set threshold, the samples obtained in the data acquisition module are input into the secondary classifier together as the original samples; otherwise, only the prediction results of the merged primary classifiers are input into the secondary classifier. The computational complexity of the secondary classifier is adaptively adjusted to improve training efficiency. The integrated output obtained after training the secondary classifier is used as the final disease detection result.

[0048] Specifically, the specific operating steps of the primary classification module are as follows:

[0049] 1) Initialize the sample weight w. Assume that there are m samples in the medical dataset and the weight of each sample is initialized to Taking the width learning network as the primary classifier, the sample set weight w1 of the first primary classifier G1 is:

[0050]

[0051] Among them, w 1,i , is the i-th sample;

[0052] 2) At the kth ensemble learning iteration, according to the kth primary classifier G k The sample weight w k,i Weight the sample X. kClassify in sample distribution X k :

[0053] X k = w k,i X; i = 1, 2,..., m; k = 1, 2,..., n

[0054] where the number of primary classifiers n is set to 6;

[0055] 3) input X k into G k to obtain the prediction result

[0056]

[0057] 4) calculate the classification error rate c k of G k :

[0058]

[0059] where Y is the sample label, and I is an indicator function,

[0060] 5) set the accuracy threshold c, when c k <c, start the incremental mode of BLS, increase the number of feature nodes and enhanced nodes to improve the accuracy of the primary classifier, without repeating the training of the primary classifier, the prediction result of the kth primary classifier after increment can be updated improve the efficiency of selective integration, such as G1, G2 in Figure 1 Repeat step 4), calculate the classification error rate c k of G k after increment, until c k ≥ c, select the primary classifier that reaches the accuracy threshold for subsequent integration. If c k ≥ c without increment, the primary classifier has reached the accuracy threshold, and there is no need to increment the BLS, and the subsequent steps are directly performed, saving certain calculation cost, as shown in G Figure 1 n ;

[0061] 6) for the ith sample, calculate the corresponding classification error rate c k of G i k according to its prediction result :

[0062]

[0063] where Y i is the label of the ith sample;​

[0064] 7) Assuming that the number of sample categories is R, according to the classification error rate Calculate G k The base classifier coefficients for:

[0065]

[0066] 8) According to For the subsequent k=2,3,…,n primary classifiers, the i-th sample is given weight w in the k-th iteration k,i , increase the weight of misclassified samples and reduce the weight of correct samples, so that the next primary classifier can correct some misclassified samples in the previous primary classifier according to the error degree of sample prediction, so that different primary classifiers complement each other:

[0067]

[0068] Where w k-1,i is the weight given to the i-th sample in the k-1th iteration, αk-1 is the k-1th base classifier coefficient, For its prediction result, e is the index, Z k-1 is the normalization factor, c k-1 is the classification error rate in the k-1th iteration.

[0069] 9) In the classification error rate c k When it reaches 0 or the number of primary classifiers reaches the upper limit n, the construction of the primary classifier is stopped. Repeat iterative steps 2) to 8) to train k=2, 3, ..., n primary classifiers to obtain the prediction results of a total of n primary classifiers.

[0070] Specifically, the specific operation steps of the secondary classification module are as follows:

[0071] 1) Combine the n primary classifier prediction results obtained in the primary classification module into the result Calculate the average precision of n primary classifiers:

[0072]

[0073] 2) Adaptively adjust the input of the secondary classifier based on the average accuracy of n primary classifiers. Use the logistic regression algorithm as the secondary classifier. Set the accuracy threshold ACC. If Then the original sample X is combined with the enhanced feature Splicing, input secondary classifier, use cross-validation method to prevent overfitting, training to obtain integrated output Y' as the final disease detection result; otherwise, only Input the secondary classifier F. After training, the integrated output Y′ is obtained as the final disease detection result.

[0074]

[0075] The experiment selected different types of disease datasets to verify the effectiveness of the system of the present invention. The breast cancer dataset contains 569 samples, each containing 30-dimensional features, which is used to detect breast cancer; the thyroid dataset contains 7,200 samples, each containing 21-dimensional features, which is used to detect thyroid function, and the labels are divided into three categories, including normal, hyperthyroidism, or hypothyroidism; the bupa dataset contains 345 samples, each containing 6-dimensional features, which is used to detect liver disease; and the dermatology dataset contains 366 samples from 6 skin diseases, each containing 34-dimensional features. For datasets with missing values, such as dermatology, the missing feature values ​​are replaced with the mode of the feature. For binary classification datasets, such as breast cancer and bupa, the data labels are set to {+1,-1} to simplify the subsequent sample weight update and the calculation of the primary classifier weight. The dataset is randomly divided into two parts, with the training set:test set = 4:1.

[0076] The detailed information of the dataset is shown in Table 1.

[0077] Table 1 Dataset details

[0078] Dataset name Sample size Number of features Number of categories Disease type breast cancer 569 30 2 Breast cancer thyroid 7200 21 3 Thyroid function bupa 345 6 2 Liver disease dermatology 366 34 6 skin disease

[0079] This experiment uses accuracy as the evaluation criterion. The accuracy is calculated as follows:

[0080]

[0081] In the formula, m is the number of samples, is the predicted value of the method for the i-th sample, Y is the sample label of the i-th sample, I is the indicator function,

[0082] To verify the effectiveness of the system, the precision is used as the evaluation standard, and comparative experiments are carried out on four commonly used disease data sets breast cancer, thyroid, bupa and dermatology. The comparison methods include three Adaboost ensemble algorithms based on different traditional machine learning methods, including support vector machine (SVM), K nearest neighbor (KNN), Gaussian naive Bayes (GNB), and GCForest and XGBoost based on decision tree as the base learner, and Stacking method, and the precision of the test set is shown in Table 2.

[0083] Table 2 Experimental result analysis table

[0084]

[0085]

[0086] Among them, the optimal result is indicated by bold, and the experimental results can fully prove the effectiveness of the system. By performing sequence weighting on the sample, the diversity of the ensemble is improved; by the selective ensemble method, the accuracy of the ensemble is improved. In addition, the secondary classifier is introduced, and the prediction result of the primary classifier is further trained, so that the generalization performance is more excellent.

[0087] Experimental conclusion: in view of the shortcomings of the existing algorithm, the disease detection system based on width ensemble learning is proposed. The experimental results on the four public disease data sets breast cancer, thyroid, bupa and dermatology show that the use of the system has certain improvement on the disease detection precision, which is better than the existing method. In future research, the improvement of the feature extraction ability of the width learning network and the application problem of the multi-index selective ensemble method in the disease detection task will be explored, which has good application prospect and is worth popularizing.

[0088] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited by the above embodiments, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the present application should be equivalent replacement methods, which are all included in the protection scope of the present application.

Claims

1. A disease detection system based on wide ensemble learning, characterized by: include: A data acquisition module is used to acquire a medical data set containing multiple medical samples and normalize the samples to obtain input samples of uniform size; The primary classification module is used to optimize the samples of ensemble learning, average the weights of the initialized samples, weight the input samples, and use n primary classifiers to train the weighted samples. The primary classifier is a width learning network. When the accuracy of the primary classifier is insufficient, the incremental mode of the width learning network is used to improve the accuracy of the primary classifier by increasing the number of hidden layer nodes. The primary classifier with excellent performance is selected to predict the weighted samples to improve the accuracy of the ensemble learning. In the prediction process, for the prediction results of the previous primary classifier, the samples are weighted according to the error of each sample, the weight of the sample with incorrect prediction is increased, and the weight of the sample with correct prediction is reduced, so that the next primary classifier pays more attention to the difficult-to-classify samples. The input samples of different primary classifiers are weighted differently to improve the diversity of ensemble learning. High accuracy and rich diversity can bring performance improvements to the width ensemble network. The n primary classifiers are trained iteratively in a loop to finally obtain the prediction results and accuracy of the n primary classifiers. The secondary classification module uses a classifier with learning ability to merge the prediction results of each primary classifier, optimizes the features of the integrated learning, and generates the final prediction results; wherein, the classifier with learning ability is called a secondary classifier, the prediction results of n primary classifiers are merged and input as enhanced features into the secondary classifier for further training, and the average accuracy of the n primary classifiers is calculated. When the average accuracy is less than the set threshold, the samples obtained in the data acquisition module are input into the secondary classifier together as the original samples. Otherwise, only the prediction results of the merged primary classifiers are input into the secondary classifier, and the computational complexity of the secondary classifier is adaptively adjusted to improve the training efficiency. The integrated output obtained after the secondary classifier training is used as the final disease detection result.

2. The disease detection system based on wide ensemble learning according to claim 1, characterized in that: The data acquisition module constrains the sample features of each dimension to the same order of magnitude, thereby balancing the contribution of each feature.

3. The disease detection system based on wide ensemble learning according to claim 2, characterized in that: The specific operation steps of the primary classification module are as follows: 1) Initialize the sample weight w. Assume that there are m samples in the medical dataset and the weight of each sample is initialized to Taking the width learning network as the primary classifier, the sample set weight w1 of the first primary classifier G1 is: Where w 1,i , is the i-th sample; 2) At the kth ensemble learning iteration, according to the kth primary classifier G k The sample weight w k,i Weight the original sample X, and k In the sample distribution X k Classify in: X k =w k,i X;i=1,2,...,m;k=1,2,...,n 3) X k Enter G k Train and get the prediction results 4) Calculate G k The classification error rate c k : In the formula, Y is the sample label, I is the indicator function, 5) Set the accuracy threshold c, at c k When <c, the incremental mode of the width learning network is started, and the accuracy of the primary classifier is improved by increasing the number of hidden layer nodes. The prediction results of the kth primary classifier after the increment can be updated without repeating the training of the primary classifier. Improve the efficiency of selective integration; repeat step 4) and calculate the incremental G k The classification error rate c k , until c k ≥c, the primary classifiers that reach the accuracy threshold are selected for subsequent integration; 6) For the i-th sample, according to its prediction result Calculate G k The corresponding classification error rate for: Where Y i is the label of the i-th sample; 7) Assuming that the number of sample categories is R, according to the classification error rate Calculate G k The base classifier coefficients for: 8) According to For the subsequent k=2,3,…,n primary classifiers, the i-th sample is given weight w in the k-th iteration k,i , increase the weight of misclassified samples and reduce the weight of correct samples, so that the next primary classifier can correct some misclassified samples in the previous primary classifier according to the error degree of sample prediction, so that different primary classifiers complement each other: Where w k-1,i is the weight given to the i-th sample in the k-1th iteration, α k-1 is the k-1th base classifier coefficient, is the prediction result, e is the index, Z k-1 is the normalization factor, c k-1 is the classification error rate in the k-1th iteration; 9) In the classification error rate c k When it reaches 0 or the number of primary classifiers reaches the upper limit n, stop building the primary classifier; repeat iterative steps 2) to 8), train k=2, 3, ..., n primary classifiers, and obtain the prediction results of a total of n primary classifiers 4. The disease detection system based on wide ensemble learning according to claim 3, characterized in that: The specific operation steps of the secondary classification module are as follows: 1) Combine the prediction results of n primary classifiers obtained in the primary classification module into the result Calculate the average precision of n primary classifiers 2) Adaptively adjust the input of the secondary classifier F according to the average accuracy of n primary classifiers, set the accuracy threshold ACC, if Then the original sample X is combined with splicing, input the secondary classifier F, train and get the integrated output Y′ as the final disease detection result; otherwise, only Input the secondary classifier F and train it to obtain the integrated output Y′ as the final disease detection result;

Images