Image recognition method and system based on TOP2 hybrid decision model integration algorithm

Abstract

The invention discloses an image recognition method and system based on a TOP2 hybrid decision model integration algorithm, and belongs to the technical field of image data processing. The objective of the invention is to reduce the over-fitting risk of the model and significantly improve the calculation efficiency and test precision of the model on various data sets. The technical key points are as follows: initializing random seeds of base models, training a plurality of base models based on a data set, initializing a decision matrix, calculating and outputting forward inference of the base models, updating the decision matrix, judging whether the forward inference of all the base models and the updating of the decision matrix are completed or not, giving out judgment conditions, rearranging the decision matrix, and finally obtaining the decision matrix. And giving a rearranged category index set, exporting an optimal sorting set of category indexes according to an optimal sorting index sequence, calculating a decision matrix statistic, and judging that the statistic conforms to statistical significance. The method is suitable for multi-model decision-making scenes such as medical image analysis and monitoring video processing.

Description

Technical Field

[0001] The present invention belongs to the field of image data processing technology, and more specifically, to machine learning technology in computer science and technology, especially machine learning for algorithm design and implementation of integrated models in intelligent decision-making systems. Background Art

[0002] Ensemble learning is a common method in machine learning that aims to improve prediction accuracy by combining multiple models. Key ensemble learning methods include stacking, boosting, and bagging. In addition, average probability and majority voting are also commonly used as basic decision ensemble strategies.

[0003] Average Probability Method: This method makes the final decision by calculating the average of the predicted probabilities of all models. Although this method can reduce the variance of the model, it may not be sensitive enough to imbalanced datasets or extreme values.

[0004] Majority Voting: In this strategy, the final prediction result is based on the consensus of the majority of models. This method is simple and easy to implement, but the reliability and stability of its decision-making results may be affected when there is a large difference in model performance.

[0005] Stacking: This method involves taking the outputs of multiple different models as the input of a new model. Although stacking can effectively leverage the strengths of different models, its main drawback is that its model training process is complex and prone to overfitting.

[0006] Boosting: This method iteratively adjusts the weights of the data to emphasize data points that were misclassified by the previous model. Although boosting can improve the accuracy of the model, it is very sensitive to noise and outliers and can easily lead to overfitting.

[0007] Bagging: Multiple models are trained by sampling with replacement from the original dataset and then averaging or voting on the outputs of these models. Although bagging can significantly reduce the variance of the model, its effect is not obvious when the dataset is small or the models are not very different.

[0008] References:

[0009] 1. Kumar, Suyash et al. "A Comprehensive Survey on Ensemble Methods." 2022IEEE 7th International conference for Convergence in Technology (I2CT) (2022): 1-7.

[0010] 2. Mienye, Ibomoiye Domor and Yanxia Sun. "A Survey of EnsembleLearning: Concepts, Algorithms, Applications, and Prospects." IEEE Access 10(2022):99129-99149.

[0011] From the above, we can see that the existing ensemble learning methods still have problems such as complex model training process, risk of model overfitting, and the computational efficiency and test accuracy need to be improved. Summary of the Invention

[0012] The technical problems to be solved by the present invention are:

[0013] In order to reduce the risk of overfitting of the model and significantly improve the computational efficiency and test accuracy of the model on various data sets, the present invention provides an image recognition method and system based on the TOP2 hybrid decision model integration algorithm.

[0014] The technical solution adopted by the present invention to solve the above technical problems is:

[0015] An image recognition method based on a TOP2 hybrid decision model integration algorithm, wherein the implementation process of the method is as follows:

[0016] Initialize base model random seeds: Use the same training framework and model f, but use different random seeds to initialize m base models;

[0017] Training multiple base models based on the data set: For a specific machine learning task, the parameters of m base models are gradient updated to minimize the loss function and obtain all the required trained base models.

[0018] Decision matrix initialization: Initialize the decision matrix M so that its elements are all 0; at the same time, prepare the input data x, which is one-dimensional or multi-dimensional data. The input data x represents image data. The trained base model Discarding factor d, significance factor α, set of category indices S = {c1, c2, ..., cN};

[0019] The forward inference calculation output of the base model: For the i-th base model Calculate the eigenvector obtained by the input data x through the basis model

[0020] Update the decision matrix: Update each element of the decision matrix M according to the following formula:

[0021] c a ←argmax i∈S y i

[0022]

[0023] Where: y i The feature vector representing the output;

[0024] c a 、c b Respectively represent the indexes corresponding to the two features with the largest and second largest values ​​in the feature vector;

[0025] Represents the elements on the diagonal of the decision matrix M; Represents the off-diagonal elements of the decision matrix M;

[0026] Whether all base models have been calculated: This part determines whether the forward inference of all base models and the update of the decision matrix have been completed. The judgment condition is: if i = m;

[0027] Rearrange the decision matrix: According to the answer-ranking algorithm, the decision matrix is ​​rearranged to obtain the rearranged matrix M ranked The best sort index order π * ;

[0028] Rearrange the category index set: according to the best sort index order π * Derive the best sorted set of category indexes

[0029] S * ←π * (S)

[0030] Calculate the decision matrix statistics: Let M ranked The row index corresponding to the largest element on the diagonal is k; calculate the significance test statistic of the Z test:

[0031]

[0032] The statistic is statistically significant: the rearranged matrix M obtained by sorting the decision matrixranked Is there an impact caused by sampling randomness? According to the preconditions of Z test and central limit theorem, the impact of M ranked Key elements of sorting And statistic Z to judge, if it satisfies Z ≥ z α , then rearrange the matrix M rked If statistical significance is satisfied, S * The first element S * (1) is listed as the answer obtained by the final integration strategy, otherwise S * The kth element S * (k) is output as the final answer.

[0033] The method is used in medical image analysis and surveillance video processing multi-model decision-making scenarios.

[0034] In the medical image analysis scenario, x represents the image data of the medical image, the output of the base model is the output of the medical image classification model, and then this method is used to make a decision output;

[0035] In the multi-model decision-making scenario of surveillance video processing, x represents a single frame of image data of the video.

[0036] An image recognition method system based on a TOP2 hybrid decision model integrated algorithm has a program module corresponding to the above steps, and executes the steps in the above image recognition method based on the TOP2 hybrid decision model integrated algorithm during operation.

[0037] A computer-readable storage medium stores a computer program, wherein the computer program is configured to implement the steps of the image recognition method based on the TOP2 hybrid decision model integration algorithm when called by a processor.

[0038] The present invention has the following beneficial technical effects:

[0039] This algorithm significantly reduces the risk of overfitting of the model by performing hierarchical modeling on the base model, and can significantly improve the computational efficiency and test accuracy of the model on various data sets.

[0040] Tables 1-8 show a comparison of the performance of the TOP2 hybrid decision algorithm on various image classification datasets. As can be seen, on classification datasets commonly used in ensemble models, the TOP2 hybrid decision algorithm outperforms majority voting, average probability, and stacking. Furthermore, this performance improvement is achieved with virtually no additional computing power.

[0041] For the Office-31 (DSLR) dataset, the effectiveness of the algorithm of the present invention is visualized, as shown in Figure 3 As shown in the figure, after the TOP2 hybrid decision-making algorithm calculates, the model's output changes from an error (the result of majority voting) to a correct one. Furthermore, the corrected incorrect answer is highly similar to the correct answer, demonstrating the advanced nature of the TOP2 hybrid decision-making algorithm. The ensemble strategy has remained unchanged for 30 years since the introduction of stacking. The TOP2 hybrid decision-making algorithm, for the first time, comprehensively surpasses stacking, representing a revolutionary innovation.

[0042] The present invention is applicable to multi-model decision-making scenarios such as medical image analysis and surveillance video processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is the calculation flow chart of the TOP2 hybrid decision algorithm;

[0044] Figure 2 This is the system structure diagram of the TOP2 hybrid decision algorithm;

[0045] Figure 3 Screenshot of the visualization of the algorithm's effectiveness based on the Office-31 (DSLR) dataset. DETAILED DESCRIPTION

[0046] like Figures 1 to 3 As shown, for the implementation of the image recognition method based on the TOP2 hybrid decision model integration algorithm described in the present invention, the operation steps of each process are introduced in detail below:

[0047] Initialize base model random seeds: Use the same training framework and model f, but use different random seeds to initialize m base models.

[0048] Training multiple base models based on the data set: For a specific machine learning task, the parameters of m base models are gradient updated to minimize the loss function and obtain all the required trained base models.

[0049] Decision matrix initialization: Initialize the decision matrix M so that its elements are all 0. At the same time, prepare the input data x (the input data can be one-dimensional or multi-dimensional data), and the trained base model Discarding factor d, significance factor α, set of category indices S = {c1, c2, ..., c N}.

[0050] The forward inference calculation output of the base model: For the i-th base model Calculate the eigenvector obtained by the input data x through the basis model

[0051] Update the decision matrix: Update each element of the decision matrix M according to the following formula:

[0052] c a ←argmax i∈S y i

[0053]

[0054]

[0055] Whether all base models have been calculated: This part determines whether the forward inference of all base models and the update of the decision matrix have been completed. The judgment condition is: if i=m.

[0056] Rearrange the decision matrix: According to the answer ranking algorithm (Answer-Ranking algorithm) publicly available in the paper Kong, Yuqing et al. "Eliciting Thinking Hierarchy without a Prior." Neural Information Processing Systems (2021), the decision matrix is ​​rearranged to obtain the rearranged matrix M ranked The best sort index order π * .

[0057] Rearrange the category index set: according to the best sort index order π * Derive the best sorted set of category indexes

[0058] S * ←π * (S)

[0059] Calculate the decision matrix statistics: Let M ranked The row index corresponding to the largest element on the diagonal is k. Calculate the significance test statistic of the Z test:

[0060]

[0061] The statistic is statistically significant: the rearranged matrix M obtained by sorting the decision matrix ranked Is there an impact caused by sampling randomness? According to the preconditions of Z test and central limit theorem, the impact of M ranked Key elements of sorting And statistic Z to judge, if it satisfies Z ≥ z α , then rearrange the matrix M rankedIf statistical significance is satisfied, S * The first element S * (1) is listed as the answer obtained by the final integration strategy, otherwise S * The kth element S * (k) is output as the final answer.

[0062] like Figure 2 As shown, Figure 2 This is the system structure diagram of the entire TOP2 hybrid decision algorithm. During the training phase, the base model with the same structure is trained on the same data set using different random seeds to obtain the trained base model f θ During the prediction phase, all models will receive the top 1 and top 2 categories, which are the two most probable indices in the output feature vector. The M matrix is ​​then updated using the four formulas for updating the decision matrix. Each time a base model result is input, two elements in the M matrix are updated until all base models have been input. Next, the M matrix is ​​reordered by row and column permutations using the Answer-Ranking algorithm. The index categories corresponding to the elements in the first row are selected for output, resulting in the output of the integrated decision.

[0063] The technical solution of the present invention is verified as follows:

[0064] The effectiveness of this decision method was verified in the field of image classification. In the verification of 6 image classification datasets in various fields, the TOP2 HD decision methods surpassed the previous methods.

[0065] TABLE VIII

[0066] COMPARATIVE EXPERIMENTAL RESULTS ON PLANTDOC

[0067]

[0068] Table 1 Performance comparison of TOP2 hybrid decision algorithms on the PlantDoC dataset

[0069] TABLE VII

[0070] COMPARATIVE EXPERIMENTAL RESULTS ON OFFICE-31

[0071] AMAZON→WEBCAM

[0072]

[0073] Table 2 Performance comparison of TOP2 hybrid decision algorithm on Office-31 (WEBCAM) dataset

[0074] TABLE VI

[0075] COMPARATIVE EXPERIMENTAL RESULTS oN oFFICE-31

[0076] AMAZON → DSLR

[0077]

[0078] Table 3 Performance comparison of TOP2 hybrid decision algorithm on Office-31 (DSLR) dataset

[0079] TABLE V

[0080] COMPARATIVE EXPERIMENTAL RESULTS ON CIFAR-100

[0081]

[0082] Table 4 Performance comparison of TOP2 hybrid decision algorithms on the CIFAR-100 dataset

[0083] TABLE IV

[0084] COMPARATIVE EXPERIMENTAL RESULTS ON CIFAR-1 0

[0085]

[0086] Table 5 Performance comparison of TOP2 hybrid decision algorithms on the CIFAR-10 dataset

[0087] TABLE III

[0088] COMPARATIVE EXPERIMENTAL RESULTS ON TINY-IMAGENET

[0089]

[0090] Table 6 Performance comparison of TOP2 hybrid decision algorithms on the Tiny-InageNet dataset

[0091] TABLE IX

[0092] COMPARATIVE EXPERIMENTAL RESULTS ONANIMAL-10N

[0093]

[0094] Table 7 Performance comparison of the TOP2 hybrid decision-making algorithms on the Anima1-10N dataset

[0095] The performance comparison of the TOP2 hybrid decision algorithm on various image classification datasets is shown in Tables 1 to 7. It can be seen that in the classification datasets commonly used by ensemble models, the TOP2 hybrid decision algorithm is more advanced than the majority voting method, the average probability method, and the stacking method. At the same time, the performance improvement can be achieved with almost no additional computing power. In order to explore different base models To determine whether this would affect the performance of the algorithm, additional experiments were conducted as shown in Table 8. As can be seen from Table 8, the performance indicators of the method of the present invention are relatively superior.

[0096] TABLE XI

[0097] COMPARISON OF DIFFERENT STRUCTURES ON PLNATDOC

[0098]

[0099] Table 8 Performance comparison of TOP2 hybrid decision algorithms using different base models on the PlantDoc dataset For the Office-31 (DSLR) dataset, we visualize the effectiveness of the algorithm, such as Figure 3 As shown in the figure, after calculation by the TOP2 hybrid decision-making algorithm, the output result of the model changes from wrong (the result of the majority voting method) to correct. It can be seen that there is a high similarity between the corrected wrong answer and the correct answer, which reflects the advanced nature of the TOP2 hybrid decision-making algorithm.

[0100] The integrated strategy has remained unchanged for 30 years since Stacking was proposed. The TOP2 hybrid decision algorithm has completely surpassed Stacking for the first time, which is a revolutionary invention and innovation.

[0101] Experiments on the algorithm of the present invention:

[0102] The experimental environment of the algorithm of the present invention is as follows:

[0103] Hardware: The experiments were conducted on a computer equipped with an Intel i7 processor and 16GB of RAM.

[0104] Software environment: Use Python 3.9, relying on NumPy and PyTorch libraries.

[0105] Dataset: Use the image classification dataset disclosed in Part 4 of this patent disclosure.

[0106] Experimental process:

[0107] Initialization: Use the image as the input data x of the algorithm, all base models Use a pre-trained convolutional neural network, remove the last fully connected layer, and adapt it to different datasets. Initialize the TOP2 hybrid decision matrix M to a zero matrix.

[0108] Algorithm Execution: All base models are trained. After training, the base models are forward-inferred to calculate outputs and update the decision matrix. A new order of category indices is obtained using the answer ranking algorithm. For different datasets, different dropout factors d and significance levels α are set, and a grid search is performed until the optimal ensemble accuracy is achieved.

[0109] Application scenarios: This algorithm is suitable for multi-model decision-making scenarios such as medical image analysis and surveillance video processing.

[0110] The method of the present invention has been verified through simulation experiments and actual applications to achieve the technical effects claimed by the present invention. The algorithm (method) proposed in the present invention is the underlying technical core of the present invention, and various products can be derived based on the algorithm. Based on the algorithm (method) proposed in the present invention, an image recognition system based on the TOP2 hybrid decision model integrated algorithm is developed using a programming language. The system has program modules corresponding to the steps of the above-mentioned technical solution, and executes the steps in the above-mentioned image recognition method based on the TOP2 hybrid decision model integrated algorithm during operation.

[0111] The developed system (software) computer program is stored on a computer-readable storage medium. The computer program is configured to implement the steps of the above-mentioned image recognition method based on the TOP2 hybrid decision model integration algorithm when called by a processor. This materializes the present invention on a carrier, becoming a computer program product.

[0112] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] The computer programs (also referred to as programs, software, software applications, or code) of the present invention include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0114] It should be understood that the various forms of the above-described processes may be used, and steps may be reordered, added, or deleted. For example, the steps described in this application may be performed in parallel, sequentially, or in a different order. As long as the desired results of the technical solutions disclosed in this application can be achieved, they are all within the scope of protection of the present invention.

Claims

1. An image recognition method based on a TOP2 hybrid decision model integration algorithm, characterized in that: The implementation process of the method is: Initialize base model random seeds: Use the same training framework and model f, but use different random seeds to initialize m base models; Train multiple base models based on the data set: For a specific machine learning task, perform gradient updates on the parameters of m base models to minimize their loss function and obtain all the required trained base models. Decision matrix initialization: Initialize the decision matrix M so that its elements are all 0; prepare input data x at the same time. The input data is one-dimensional or multi-dimensional data. The input data x represents image data. The trained base model The discard factor d, the significance factor α, the set of category indices S = {c1, c2, ..., c N }; The forward inference calculation output of the base model is: Calculate the eigenvector obtained by the input data x through the base model Update the decision matrix: Update each element of the decision matrix M according to the following formula: c a ←argmax i∈S y i Where: y i The feature vector representing the output; c a 、c b Respectively represent the indexes corresponding to the two features with the largest value and the second largest value in the feature vector; Represents the elements on the diagonal of the decision matrix M; Represents the off-diagonal elements of the decision matrix M; Whether all base models have been calculated: This part determines whether the forward inference of all base models and the update of the decision matrix have been completed. The judgment condition is: ifi = m; Rearrange the decision matrix: According to the answer-ranking algorithm, the decision matrix is ​​rearranged to obtain the rearranged matrix M ranked Best sort index order π * ; Rearrange the category index collection: according to the best sort index order π * Derive the best sorted set of category indexes S * ←π * (S) Calculate the decision matrix statistics: Let M ranked The row index corresponding to the largest element on the diagonal is k; calculate the significance test statistic of the Z test: The statistic is statistically significant: the rearrangement matrix M obtained by sorting the decision matrix ranked Is there an impact caused by sampling randomness? According to the preconditions of Z test and central limit theorem, the impact of M ranked Key elements of sorting And the statistic Z is used to judge, if it satisfies Z ≥ z α , then rearrange the matrix M ranked If statistical significance is met, S * The first element S * (1) is listed as the answer obtained by the final integrated strategy, otherwise S * The kth element S * (k) is output as the final answer.

2. The image recognition method based on the TOP2 hybrid decision model integration algorithm according to claim 1 is characterized in that: The method is used in medical image analysis and surveillance video processing multi-model decision-making scenarios.

3. The image recognition method based on the TOP2 hybrid decision model integration algorithm according to claim 2 is characterized in that: In the medical image analysis scenario, x represents the image data of the medical image, the output of the base model is the output of the medical image classification model, and then this method is used to make a decision output; In the multi-model decision-making scenario of surveillance video processing, x represents a single frame of image data of the video.

4. An image recognition method system based on TOP2 hybrid decision model integration algorithm, characterized by: The system has a program module corresponding to the steps of any one of claims 1 to 3, and executes the steps in the above-mentioned image recognition method based on the TOP2 hybrid decision model integration algorithm when running.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the image recognition method based on the TOP2 hybrid decision model integration algorithm described in any one of claims 1 to 3 when called by a processor.

Images