OCT retinal lesion image recognition method and system
By modifying and fusing InceptionV3, InceptionResNet and Xception network models in OCT retinopathy image recognition, adding attention mechanisms and output layers, and performing transfer learning and fine-tuning, the insufficient adaptability of multi-network fusion strategies and data sets in the existing technology is solved. The problem of this achieves efficient image recognition effect.
Patent Information
- Application Number
- CN202111246210.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-26
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2041-10-26
AI Technical Summary
The existing OCT retinopathy image recognition technology has the limitations of multi-network fusion strategy and the insufficient adaptability of the model to the data set, resulting in poor recognition results.
By modifying the InceptionV3, InceptionResNet and Xception network models, adding attention mechanisms and output layers, and fusing them, combining transfer learning and fine-tuning parameters, adjusting weight parameters to improve classification capabilities.
It significantly improves the recognition and classification ability of OCT retinopathy images, maintains high prediction accuracy when the data volume decreases, adapts to different data sets, and improves the accuracy and recall rate.
Smart Images
Figure CN114037660B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an OCT retinal lesion image recognition method and system. Background Art
[0002] OCT (optical coherence tomography) is a new type of tomographic imaging technology that is developing rapidly and has the most promising development prospects. OCT images of the retina have a good role in assisting the diagnosis of the disease. However, due to certain differences among doctors in different regions, there are also certain differences in the ability to recognize retinal OCT images. If retinal diseases are not discovered in time, it is likely to cause vision loss or even blindness. Therefore, the automatic identification of retinal OCT images to assist doctors came into being.
[0003] Identification of OCT retinal lesion images from OCT retinal images can be roughly divided into two categories. The first category is the retinal OCT lesion image algorithm detection based on machine learning, which uses commonly used image processing algorithms (oriented gradient histogram, scale-invariant feature transform, SURF (Speeded Up Robust Features) features, ORB (Oriented FAST and Rotated BRIEF) features, LBP (Local binary pattern) features, etc.) to extract image features, and then input the extracted image features into commonly used machine learning algorithms (support vector machine, random forest, etc.), and the machine learning algorithm determines the category to which the image belongs. However, commonly used image feature extraction algorithms often lose a lot of image information and cannot fully represent this feature. In addition, there are certain similarities in retinal OCT lesion images, resulting in poor results of commonly used image processing algorithms. The second category is OCT retinal lesion image detection for deep learning, which uses a convolutional neural network structure to solve image feature extraction and achieve classification effects. However, the current effect based on the commonly used convolutional neural network structure is not good and has certain limitations.
[0004] In summary, the current OCT retinopathy image recognition has at least the following problems:
[0005] First, there are certain limitations based on the multi-network fusion strategy, which cannot bring into play the advantages of each network model.
[0006] Second, the network model based on retinal OCT image detection has certain limitations and cannot adapt well to existing datasets. Summary of the invention
[0007] In view of this, it is necessary to provide an OCT retinal lesion image recognition method and system.
[0008] The present invention provides an OCT retinal lesion image recognition method, which comprises the following steps: a. acquiring retinal OCT images to form a training set and a test set respectively; b. preprocessing the images of the training set and the test set; c. modifying three network models of InceptionV3, InceptionResNet and Xception, adding an attention mechanism and an output layer, and constructing a network model; d. using the preprocessed training set and test set to train and test the constructed network model to identify OCT retinal lesion images.
[0009] Specifically, the step b specifically includes:
[0010] Scale the width and height of the training set and test set;
[0011] The pixel values of the training set and the test set are normalized.
[0012] Specifically, the step c comprises the following steps:
[0013] c1, modify the three network models InceptionV3, InceptionResNet and Xception one by one, and add attention mechanism to each network model;
[0014] c2, add an output layer after the attention mechanism of the three network models to obtain three base classifiers: M, N, P, and the predicted probability values of each base classifier for different categories on each sample: M_predict, N_predict, P_predict;
[0015] c3, fusing the output layer results of the three base classifiers to obtain the final output value of the network model;
[0016] c4, transfer learning of the network parameters of the three network models: fix the network parameters of the three network models, and on this basis, train the network parameters of the attention mechanism CBAM and the output part of the network model;
[0017] c5, unfreeze the network parameters of the three network models and fine-tune all parameters in the network models.
[0018] Specifically, the step c3 comprises the following steps:
[0019] First, three variables X, Y, and Z are defined in the network model. The three variables correspond to three network models respectively, and the variables change according to the training of the network model.
[0020] The weight values for models M, N, and P are: X / (X+Y+Z), Y / (X+Y+Z), and Z / (X+Y+Z), respectively. Therefore, the predicted probability value of the network model for each sample in each category is:
[0021] y_predict=(X / (X+Y+Z))*M_predict+(Y / (X+Y+Z))*N_predict+(Z / (X+Y+Z))*P_predict.
[0022] The present invention provides an OCT retinal lesion image recognition system, which includes an acquisition module, a preprocessing module, a construction module and a prediction module, wherein:
[0023] The acquisition module is used to acquire retinal OCT images to form a training set and a test set respectively;
[0024] The preprocessing module is used to preprocess the images of the training set and the test set;
[0025] The construction module is used to modify the three network models of InceptionV3, InceptionResNet and Xception, add attention mechanism and output layer, and construct the network model;
[0026] The prediction module is used to train and test the constructed network model using the preprocessed training set and test set to identify OCT retinal lesion images.
[0027] Specifically, the preprocessing module is specifically used for:
[0028] Scale the width and height of the training set and test set;
[0029] The pixel values of the training set and the test set are normalized.
[0030] Specifically, the building blocks are used to:
[0031] c1, modify the three network models InceptionV3, InceptionResNet and Xception one by one, and add attention mechanism to each network model;
[0032] c2, add an output layer after the attention mechanism of the three network models to obtain three base classifiers: M, N, P, and the predicted probability values of each base classifier for different categories on each sample: M_predict, N_predict, P_predict;
[0033] c3, fusing the output layer results of the three base classifiers to obtain the final output value of the network model;
[0034] c4, transfer learning of the network parameters of the three network models: fix the network parameters of the three network models, and on this basis, train the network parameters of the attention mechanism CBAM and the output part of the network model;
[0035] c5, unfreeze the network parameters of the three network models and fine-tune all parameters in the network models.
[0036] Specifically, the output layer results of the three base classifiers are fused to obtain the final output value of the network model, which is specifically used for:
[0037] First, three variables X, Y, and Z are defined in the network model. The three variables correspond to three network models respectively, and the variables change according to the training of the network model.
[0038] The weight values for models M, N, and P are: X / (X+Y+Z), Y / (X+Y+Z), and Z / (X+Y+Z), respectively. Therefore, the predicted probability value of the network model for each sample in each category is:
[0039] y_predict=(X / (X+Y+Z))*M_predict+(Y / (X+Y+Z))*N_predict+(Z / (X+Y+Z))*P_predict.
[0040] This application first cleans the dataset images downloaded from the Internet to obtain an input image format suitable for the network model; secondly, this application uses the three network models 'InceptionV3', 'InceptionResNet' and 'Xception' that have achieved excellent results in the 'IMAGEEGT' competition as the three branches of the network of this application; finally, the network of this application fuses the output results of these three network models to obtain extremely high image recognition and classification capabilities.
[0041] Therefore, compared with the prior art, the advantages of this application include:
[0042] (1) The integration of the three network models that achieved excellent results in 'IMAGENET' and the attention mechanism CBAM can further improve the classification capabilities of the three network models;
[0043] (2) The strategy of multi-network model fusion can adjust the weight parameters of the final base classifier according to different data sets, thereby obtaining a higher classification ability and identifying OCT retinal lesion images. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 is a flow chart of the OCT retinopathy image recognition method of the present invention;
[0045] Figure 2 A detailed flow chart of step S3 provided in an embodiment of the present invention;
[0046] Figure 3 FIG. 4 is a hardware architecture diagram of the OCT retinal lesion image recognition system of the present invention. DETAILED DESCRIPTION
[0047] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0048] See also Figure 1 , which is a flowchart of a preferred embodiment of the OCT retinal lesion image recognition method of the present invention.
[0049] Step S1, acquiring retinal OCT images to form a training set and a test set respectively.
[0050] Specifically:
[0051] Step S11, download relevant retinal OCT actual hospital images from the Internet as training set 1 and test set respectively, and divide the images of the training set 1 into four categories: choroidal neovascularization image (CNV), diabetic macular edema image (DME), drusen image (DRUSEN) and normal image.
[0052] In this embodiment, a total of 108,312 retinal OCT images were downloaded as training set 1, and 1,000 retinal OCT images were downloaded as test set. The number of choroidal neovascularization images (CNV), diabetic macular edema images (DME), drusen images (DRUSEN), and normal images in the training set 1 are 37,206, 11,349, 8,617, and 51,140, respectively; the number of images in the four categories in the test set is 250.
[0053] Step S12, extracting images of each category from training set 1 as training set 2.
[0054] In this embodiment, 1000 images are randomly selected from each category of images in training set 1 as training set 2. The test set remains unchanged.
[0055] The training set is used to train the network model, and the test set is used to evaluate the quality of the network model.
[0056] Step S2, preprocessing the images of the training set and the test set. Specifically:
[0057] Step S21: scaling the width and height of the training set and the test set.
[0058] In this embodiment, the images of training set 1, training set 2, and test set are scaled to 299*299 px in width and height.
[0059] Step S22: normalize the pixel values of the training set and the test set.
[0060] In this embodiment, the image pixel values of training set 1, training set 2, and test set are normalized to between 0 and 1 to facilitate the subsequent training of the network model.
[0061] Step S3, modify the three network models of InceptionV3, InceptionResNet and Xception, add attention mechanism and output layer, and build the network model. Please refer to Figure 2 Specifically:
[0062] Step S31, modify the three network models of InceptionV3, InceptionResNet and Xception one by one, and add an attention mechanism to each network model.
[0063] In order to use the weight parameters in "IMAGENET", delete the top layer of the network model and add the attention mechanism CBAM (Convolutional Block Attention Module) on this basis.
[0064] The beneficial effect here is that the network model that achieved good results in the "IMAGENET" competition can further improve the network effect by adding an attention mechanism.
[0065] Step S32, add an output layer after the attention mechanism of the three network models to obtain three base classifiers: M, N, P, and the predicted probability values of each base classifier for different categories on each sample are: M_predict, N_predict, P_predict respectively.
[0066] Step S33, fusing the output layer results of the three base classifiers to obtain the final output value of the network model. Specifically including:
[0067] Step S331: First, three variables X, Y, and Z are defined in the network model. The three variables correspond to three network models respectively, and the variables change according to the training of the network model.
[0068] Step S332: The weight values of models M, N, and P are respectively: X / (X+Y+Z), Y / (X+Y+Z), and Z / (X+Y+Z). Therefore, the predicted probability value of the network model for each sample in each category is:
[0069] y_predict=(X / (X+Y+Z))*M_predict+(Y / (X+Y+Z))*N_predict+(Z / (X+Y+Z))*P_predict
[0070] Step S333: The parameter update method for X, Y, and Z is:
[0071] A. Assume that the output probability of a certain sample in the final network model is: [a1, a2, a3, a4], where a1+a2+a3+a4=1.
[0072] B. Assume that the output probability of a certain sample in the network model M is: [m1,m2,m3,m4], where m1+m2+m3+m4=1.
[0073] C. Assume that the output probability of a certain sample in the network model N is: [n1,n2,n3,n4], where n1+n2+n3+n4=1.
[0074] D. Assume that the output probability of a certain sample in the network model P is: [p1, p2, p3, p4], where p1+p2+p3+p4=1.
[0075] Assuming that the true label of the sample is [1,0,0,0], this application uses 'categorical_crossentropy', so the loss value of the model is:
[0076] loss=-(1*ln(a1)+0*ln(a2)+0*ln(a3)+0*ln(a4))
[0077] in
[0078] Right now
[0079] therefore:
[0080] The derivative of loss with respect to X is: The update for X is:
[0081] The derivative of loss with respect to Y is: The update for Y is:
[0082] The derivative of loss with respect to Z is: The update for Z is:
[0083] The advantages here are:
[0084] First, the sum of the weight values of each base classifier is 1, which can ensure that the sum of the output probability values of each sample in different categories is 1.
[0085] Second: As the network model progresses, its weight values will be updated to ensure the rationality of the base classifier weight values.
[0086] Step S34, firstly, transfer learning is performed on the network parameters of the three network models: that is, the network parameters of the three network models are fixed, and on this basis, the network parameters of the attention mechanism CBAM and the output part of the network model are trained. Specifically:
[0087] In this implementation, the transfer learning includes: first removing the top layer of each network model, adding the attention mechanism CBAM and the network parameters of the output part of the network model to the top layer. At this time, the network model only adjusts the network parameters of the attention mechanism CBAM and the output part of the network model.
[0088] The advantage here is that it avoids the situation in which the parameters of the three network models based on 'IMAGENET' are drastically changed due to the instability of the network in the early stage, which fails to achieve the purpose of transfer learning.
[0089] Step S35, unfreeze the network parameters of the three network models, and fine-tune all parameters in the network models. Specifically:
[0090] This embodiment adjusts the parameters of three network models, the attention mechanism module, and the output layer module.
[0091] The advantage here is that the three network models target different data sets and need to be re-optimized here to adapt to different data sets and finally improve the predictive ability of the network models.
[0092] Step S4, using the preprocessed training set and test set to train and test the constructed network model to identify OCT retinal lesion images.
[0093] For training set 1, training set 2, and test set, step S3 is performed, and the results are: 97.1% and 97.8% respectively. It can be seen that the network model of the present application can improve the prediction ability of the test set when the amount of data decreases sharply, and the reasons are as follows:
[0094] There is a certain imbalance between the categories of the original data set, which will cause a certain deviation in the model's prediction.
[0095] See also Figure 3 FIG. 1 is a hardware architecture diagram of an OCT retinal lesion image recognition system 10 of the present invention. The system comprises: an acquisition module 101 , a preprocessing module 102 , a construction module 103 and a prediction module 104 .
[0096] The acquisition module 101 is used to acquire retinal OCT images to form a training set and a test set. Specifically:
[0097] The acquisition module 101 downloads the actual hospital images of retinal OCT from the Internet as training set 1 and test set, and divides the images of training set 1 into four categories: choroidal neovascularization image (CNV), diabetic macular edema image (DME), drusen image (DRUSEN) and normal image.
[0098] In this embodiment, the acquisition module 101 downloads a total of 108,312 retinal OCT images as training set 1, and downloads 1,000 retinal OCT images as a test set. The number of choroidal neovascularization images (CNV), diabetic macular edema images (DME), drusen images (DRUSEN), and normal images in the training set 1 are 37,206, 11,349, 8,617, and 51,140, respectively; the number of images in the four categories in the test set is 250.
[0099] The acquisition module 101 extracts images of various categories from the training set 1 as the training set 2.
[0100] In this embodiment, the acquisition module 101 randomly selects 1000 images from each category of images in the training set 1 as the training set 2. The test set remains unchanged.
[0101] The training set is used to train the network model, and the test set is used to evaluate the quality of the network model.
[0102] The preprocessing module 102 is used to preprocess the images of the training set and the test set. Specifically:
[0103] The preprocessing module 102 scales the width and height of the training set and the test set.
[0104] In this embodiment, the preprocessing module 102 scales the width and height of the images of the training set 1, the training set 2, and the test set to 299*299 px.
[0105] The preprocessing module 102 normalizes the pixel values of the training set and the test set.
[0106] In this embodiment, the preprocessing module 102 normalizes the image pixel values of the training set 1, the training set 2, and the test set to between 0 and 1 to facilitate the subsequent training of the network model.
[0107] The construction module 103 is used to modify the three network models of InceptionV3, InceptionResNet and Xception, add the attention mechanism and the output layer, and construct the network model. Specifically:
[0108] The construction module 103 modifies the three network models, InceptionV3, InceptionResNet and Xception, one by one, and adds an attention mechanism to each network model.
[0109] In order to use the weight parameters in "IMAGENET", delete the top layer of the network model and add the attention mechanism CBAM (Convolutional Block Attention Module) on this basis.
[0110] The beneficial effect here is that the network model that achieved good results in the "IMAGENET" competition can further improve the network effect by adding an attention mechanism.
[0111] The construction module 103 adds an output layer after the attention mechanism of the three network models to obtain three base classifiers: M, N, P, and the predicted probability values of each base classifier for different categories on each sample are: M_predict, N_predict, P_predict respectively.
[0112] The construction module 103 fuses the output layer results of the three base classifiers to obtain the final output value of the network model. Specifically, it includes:
[0113] First, three variables X, Y, and Z are defined in the network model, wherein the three variables correspond to three network models respectively, and the variables change according to the training of the network model.
[0114] Then, the weight values for models M, N, and P are: X / (X+Y+Z), Y / (X+Y+Z), and Z / (X+Y+Z). Therefore, the predicted probability value of the network model for each sample in each category is:
[0115] y_predict=(X / (X+Y+Z))*M_predict+(Y / (X+Y+Z))*N_predict+(Z / (X+Y+Z))*P_predict
[0116] Finally, the parameter update method for X, Y, and Z is:
[0117] A. Assume that the output probability of a certain sample in the final network model is: [a1, a2, a3, a4], where a1+a2+a3+a4=1.
[0118] B. Assume that the output probability of a certain sample in the network model M is: [m1,m2,m3,m4], where m1+m2+m3+m4=1.
[0119] C. Assume that the output probability of a certain sample in the network model N is: [n1,n2,n3,n4], where n1+n2+n3+n4=1.
[0120] D. Assume that the output probability of a certain sample in the network model P is: [p1, p2, p3, p4], where p1+p2+p3+p4=1.
[0121] Assuming that the true label of the sample is [1,0,0,0], this application uses 'categorical_crossentropy', so the loss value of the model is:
[0122] loss=-(1*ln(a1)+0*ln(a2)+0*ln(a3)+0*ln(a4))
[0123] in
[0124] Right now
[0125] therefore:
[0126] The derivative of loss with respect to X is: The update for X is:
[0127] The derivative of loss with respect to Y is: The update for Y is:
[0128] The derivative of loss with respect to Z is: The update for Z is:
[0129] The advantages here are:
[0130] First, the sum of the weight values of each base classifier is 1, which can ensure that the sum of the output probability values of each sample in different categories is 1.
[0131] Second: As the network model progresses, its weight values will be updated to ensure the rationality of the base classifier weight values.
[0132] The construction module 103 performs transfer learning on the network parameters of the three network models: that is, the network parameters of the three network models are fixed, and on this basis, the network parameters of the attention mechanism CBAM and the output part of the network model are trained.
[0133] The advantage here is: avoid the early stage of network instability, which will cause the parameters of the three network models based on 'IMAGENET' to be changed drastically, and fail to achieve the purpose of transfer learning. Specifically:
[0134] In this implementation, the transfer learning includes: first removing the top layer of each network model, adding the attention mechanism CBAM and the network parameters of the output part of the network model to the top layer. At this time, the network model only adjusts the network parameters of the attention mechanism CBAM and the output part of the network model.
[0135] The construction module 103 unfreezes the network parameters of the three network models and fine-tunes all parameters in the network models. Specifically:
[0136] This embodiment adjusts the parameters of three network models, the attention mechanism module, and the output layer module.
[0137] The advantage here is that the three network models target different data sets and need to be re-optimized here to adapt to different data sets and finally improve the predictive ability of the network models.
[0138] The prediction module 104 is used to train and test the constructed network model using the preprocessed training set and test set to identify OCT retinal lesion images.
[0139] For training set 1, training set 2, and test set, step S3 is performed, and the results are: 97.1% and 97.8% respectively. It can be seen that the network model of the present application can improve the prediction ability of the test set when the amount of data decreases sharply, and the reasons are as follows:
[0140] There is a certain imbalance between the categories of the original data set, which will cause a certain deviation in the model's prediction.
[0141] This application can greatly improve the advantages of each classification network model, and finally obtain a very high accuracy and recall rate: first, the image downloaded from the network is cleaned to obtain an input image format suitable for the network model; secondly, this application uses the three major network models 'InceptionV3', 'InceptionResNet' and 'Xception' that have achieved excellent results in the 'IMAGEEGT' competition as the three branches of the network of this application; finally, the network of this application fuses the output results of these three network models to obtain extremely high classification capabilities, thereby identifying OCT retinal lesion images.
[0142] Although the present invention has been described with reference to the current preferred embodiments, those skilled in the art should understand that the above-mentioned preferred embodiments are only used to illustrate the present invention and are not used to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.
Claims
1. An OCT retinopathy image recognition method, characterized in that: The method comprises the following steps: a. Obtain retinal OCT images to form training sets and test sets respectively; b. Preprocessing the images of the training set and the test set; c. Modify the three network models of InceptionV3, InceptionResNet and Xception, add attention mechanism and output layer, and build the network model; d. Use the preprocessed training set and test set to train and test the constructed network model to identify OCT retinopathy images; The step c specifically comprises the following steps: c1, modify the three network models InceptionV3, InceptionResNet and Xception one by one, and add attention mechanism to each network model; c2, add an output layer after the attention mechanism of the three network models to obtain three base classifiers: M, N, P, and the predicted probability values of each base classifier for different categories on each sample: M_predict, N_predict, P_predict; c3, fusing the output layer results of the three base classifiers to obtain the final output value of the network model; The step c3 specifically includes the following steps: First, three variables X, Y, and Z are defined in the network model. The three variables correspond to three network models respectively, and the variables change according to the training of the network model. The weight values for models M, N, and P are: X / (X+Y+Z), Y / (X+Y+Z), and Z / (X+Y+Z), respectively. Therefore, the predicted probability value of the network model for each sample in each category is: y_predict=(X / (X+Y+Z))*M_predict+(Y / (X+Y+Z))*N_predict+(Z / (X+Y+Z))*P_predict; The parameter update method for X, Y, and Z is: A. Assume that the output probability of a certain sample in the final network model is: [a1, a2, a3, a4], where a1+a2+a3+a4=1; B. Assume that the output probability of a certain sample in the network model M is: [m1,m2,m3,m4], where m1+m2+m3+m4=1; C. Assume that the output probability of a certain sample in the network model N is: [n1,n2,n3,n4], where n1+n2+n3+n4=1; D. Assume that the output probability of a certain sample in the network model P is: [p1, p2, p3, p4], where p1+p2+p3+p4=1; Assuming that the true label of the sample is [1,0,0,0], this application uses 'categorical_crossentropy', so the loss value of the model is: loss=-(1*ln(a1)+0*ln(a2)+0*ln(a3)+0*ln(a4)) in Right now therefore: The derivative of loss with respect to X is: The update for X is: The derivative of loss with respect to Y is: The update for Y is: The derivative of loss with respect to Z is: The update for Z is:
2. The method according to claim 1, characterized in that The step b specifically comprises: Scale the width and height of the training set and test set; The pixel values of the training set and the test set are normalized.
3. The method according to claim 2, characterized in that The step c also includes the following steps: c4, transfer learning of the network parameters of the three network models: fix the network parameters of the three network models, and on this basis, train the network parameters of the attention mechanism CBAM and the output part of the network model; c5, unfreeze the network parameters of the three network models and fine-tune all parameters in the network models.
4. An OCT retinopathy image recognition system, characterized in that: The system includes an acquisition module, a preprocessing module, a construction module and a prediction module, wherein: The acquisition module is used to acquire retinal OCT images to form a training set and a test set respectively; The preprocessing module is used to preprocess the images of the training set and the test set; The construction module is used to modify the three network models of InceptionV3, InceptionResNet and Xception, add attention mechanism and output layer, and construct the network model; The prediction module is used to train and test the constructed network model using the preprocessed training set and test set to identify OCT retinal lesion images; The building blocks are specifically used for: c1, modify the three network models InceptionV3, InceptionResNet and Xception one by one, and add attention mechanism to each network model; c2, add an output layer after the attention mechanism of the three network models to obtain three base classifiers: M, N, P, and the predicted probability values of each base classifier for different categories on each sample: M_predict, N_predict, P_predict; c3, fusing the output layer results of the three base classifiers to obtain the final output value of the network model; The output layer results of the three base classifiers are fused to obtain the final output value of the network model, which is specifically used for: First, three variables X, Y, and Z are defined in the network model. The three variables correspond to three network models respectively, and the variables change according to the training of the network model. The weight values for models M, N, and P are: X / (X+Y+Z), Y / (X+Y+Z), and Z / (X+Y+Z), respectively. Therefore, the predicted probability value of the network model for each sample in each category is: y_predict=(X / (X+Y+Z))*M_predict+(Y / (X+Y+Z))*N_predict+(Z / (X+Y+Z))*P_predict; The parameter update method for X, Y, and Z is: A. Assume that the output probability of a certain sample in the final network model is: [a1, a2, a3, a4], where a1+a2+a3+a4=1; B. Assume that the output probability of a certain sample in the network model M is: [m1,m2,m3,m4], where m1+m2+m3+m4=1; C. Assume that the output probability of a certain sample in the network model N is: [n1,n2,n3,n4], where n1+n2+n3+n4=1; D. Assume that the output probability of a certain sample in the network model P is: [p1, p2, p3, p4], where p1+p2+p3+p4=1; Assuming that the true label of the sample is [1,0,0,0], this application uses 'categorical_crossentropy', so the loss value of the model is: loss=-(1*ln(a1)+0*ln(a2)+0*ln(a3)+0*ln(a4)) in Right now therefore: The derivative of loss with respect to X is: The update for X is: The derivative of loss with respect to Y is: The update for Y is: The derivative of loss with respect to Z is: The update for Z is:
5. The system according to claim 4, characterized in that The preprocessing module is specifically used for: Scale the width and height of the training set and test set; The pixel values of the training set and the test set are normalized.
6. The system according to claim 5, characterized in that The building blocks are also used to: c4, transfer learning of the network parameters of the three network models: fix the network parameters of the three network models, and on this basis, train the network parameters of the attention mechanism CBAM and the output part of the network model; c5, unfreeze the network parameters of the three network models and fine-tune all parameters in the network models.