Image recognition method, device and equipment and application in alzheimer's disease prediction

Abstract

The application provides an image recognition method, device and equipment and application in Alzheimer's disease prediction, the image recognition method comprises: obtaining an image to be recognized, and pre-processing the image to be recognized to obtain a first feature image, a second feature image and a third feature image; inputting the first feature image, the second feature image and the third feature image into a CNN classification model to obtain a first prediction result; inputting the first feature image, the second feature image and the third feature image into an ML classification model to obtain a second prediction result; and obtaining the probability that the image to be recognized belongs to a predetermined classification category by using the first prediction result and the second prediction result, which can not only extract effective features of the image to be recognized, but also effectively fuse the effective features of the image to be recognized, thereby improving the recognition accuracy of the image to be recognized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to an image recognition method, device and equipment and application in Alzheimer's disease prediction. BACKGROUND

[0002] Alzheimer's disease (AD) presents a stage of progression, and once it enters the moderate to severe stage, the disease will immediately accelerate and be irreversible. Existing research shows that the best treatment period for AD is in the early stage, so early prediction of AD is very important for the treatment of AD.

[0003] Currently, AD is mainly predicted by a prediction method based on a neurological examination, a prediction method based on positron emission tomography combined with nuclear magnetic resonance, a prediction method based on a cerebrospinal fluid biomarker, a prediction method based on exosome and peripheral blood mass spectrometry, and a prediction method based on a machine learning algorithm. Among them: (1) the prediction method based on the neurological examination has the defects of missed diagnosis or misdiagnosis; (2) the prediction method based on positron emission tomography combined with nuclear magnetic resonance has the defects of large heterogeneity of prediction results and low resolution; (3) the prediction method based on the cerebrospinal fluid biomarker has the defects of limited sensitivity and specificity, strong invasiveness, and great difficulty in clinical development; (4) the prediction method based on exosome and peripheral blood mass spectrometry has the defects of invasive detection and long time consumption; and (5) the prediction method based on the machine learning algorithm has the defect that it cannot well extract and fuse the features of the retinal layers with different image modes. SUMMARY

[0004] Therefore, the purpose of the present application is to provide an image recognition method, device and equipment and application in Alzheimer's disease prediction, which can not only extract effective features of an image to be recognized, but also effectively fuse the effective features of the image to be recognized, so as to improve the accuracy of image recognition.

[0005] The technical solution adopted by the present application is as follows:

[0006] An image recognition method comprises:

[0007] S1: obtaining an image to be recognized and pre-processing the image to be recognized to obtain a first feature image, a second feature image and a third feature image;

[0008] S2: inputting the first feature image, the second feature image and the third feature image into a CNN classification model to obtain a first prediction result;

[0009] S3: inputting the first feature image, the second feature image and the third feature image into an ML classification model to obtain a second prediction result;

[0010] S4: obtaining a probability that the to-be-identified image belongs to a predetermined classification category by using the first prediction result and the second prediction result.

[0011] Further, in step S1, the pre-processing of the to-be-identified image comprises:

[0012] S11: performing denoising processing on the to-be-identified image by using an anisotropic diffusion filtering algorithm to obtain a first pre-processed image;

[0013] S12: performing enhancement processing on the first pre-processed image by using an improved histogram equalization algorithm to obtain a second pre-processed image;

[0014] S13: performing contour fitting processing on the second pre-processed image by using a Canny algorithm to obtain a third pre-processed image;

[0015] S14: performing normalization processing on the third pre-processed image to obtain a fourth pre-processed image;

[0016] S15: extracting the first feature image, the second feature image and the third feature image from the fourth pre-processed image.

[0017] Further, step S12 comprises:

[0018] S121: performing gray scale transformation on the first pre-processed image by using a histogram equalization algorithm to obtain a first gray scale image, and the first gray scale image comprises L gray scale layers;

[0019] S122: performing gray scale mapping on the L gray scale layers of the first gray scale image according to a gray scale mapping formula respectively to obtain the second pre-processed image, and the gray scale mapping formula is: wherein T k is a gray scale value of the kth gray scale layer in the second pre-processed image; S max is a maximum gray scale value of the first pre-processed image; S min is a minimum gray scale value of the first pre-processed image; S k is a gray scale value of the kth gray scale layer in the first pre-processed image; a is a brightness compensation coefficient, and 0≤a≤1.

[0020] Further, step S13 comprises:

[0021] S131: gray-scale processing is performed on the second pre-processed image to obtain a second gray-scale image; let Gray(x, y) represent a value of the second gray-scale image, then Gray(x, y) = 0.299 + 0.587 + 0.114, wherein: R, G and B are respectively red channel values, blue channel values and green channel values corresponding to the second pre-processed image; x is a numerical value of the second gray-scale image in the x direction; y is a numerical value of the second gray-scale image in the y direction;

[0022] S132: Gaussian filtering is performed on the second gray-scale image to obtain a filtered image, and let K represent a value of the filtered image, then wherein: sigma is a parameter;

[0023] S133: the gradient amplitude G and the gradient direction theta of a pixel point of the filtered image are calculated by using a Sobel operator, and: G = sqrt(G x 2 + y 2 ), wherein: G x is a gradient of the pixel point of the filtered image in the x direction, and G y is a gradient of the pixel point of the filtered image in the y direction, and * is a convolution symbol;

[0024] S134: the edge of the filtered image is subjected to non-maximum suppression processing by using the gradient amplitude G of the pixel point of the filtered image to determine an edge point, to obtain a contour fitting image;

[0025] S135: double-threshold detection is performed on the contour fitting image, and the pixel point with a pixel value lower than a low threshold in the edge point of the contour fitting image is suppressed, to obtain the third pre-processed image.

[0026] Further, in step S2, the CNN classification model comprises a feature extraction module and a first fusion module, the feature extraction module comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer and a feature fusion layer;

[0027] Step S2 comprises:

[0028] S21: the first feature image is input into the first feature extraction layer and the feature fusion layer respectively, to obtain first feature F 1-1 and second feature F 2-1 of the first feature image respectively;

[0029] S22: input the second feature image into the second feature extraction layer and the feature fusion layer respectively, and obtain first feature F 1-2 and second feature F 2-2 of the second feature image respectively;

[0030] S23: input the third feature image into the third feature extraction layer and the feature fusion layer respectively, and obtain first feature F 1-3 and second feature F 2-3 of the third feature image respectively;

[0031] S24: input first feature F 1-1 and second feature F 2-1 of the first feature image into the first fusion module, and obtain first prediction probability p 1-1 of the first feature image; input first feature F 1-2 and second feature F 2-2 of the second feature image into the first fusion module, and obtain first prediction probability p 1-2 of the second feature image; input first feature F 1-3 and second feature F 2-3 of the third feature image into the first fusion module, and obtain first prediction probability p 1-3 of the third feature image; first prediction probability p 1-1 of the first feature image, first prediction probability p 1-2 of the second feature image and first prediction probability p 1-3 of the third feature image form the first prediction result.

[0032] Further, in step S3, the ML classification model includes an SVM classification model, a logistic regression classification model, a decision tree classification model and a second fusion module;

[0033] Step S3 includes:

[0034] S31: input the first feature image into the SVM classification model, the logistic regression classification model and the decision tree classification model respectively, and obtain first classification result C 1-1 , second classification result C 2-1 and third classification result C 3-1 of the first feature image respectively; input the second feature image into the SVM classification model, the logistic regression classification model and the decision tree classification model respectively, and obtain first classification result C 1-2 , second classification result C 2-2 and third classification result C 3-2; inputting the third feature image into the SVM classification model, the logistic regression classification model and the decision tree classification model respectively to obtain first classification result C 1-3 , second classification result C 2-3 and third classification result C 3-3 of the third feature image respectively.

[0035] S32: inputting the first classification result C 1-1 , second classification result C 2-1 and third classification result C 3-1 of the first feature image into the second fusion module to obtain second prediction probability p 2-1 of the first feature image; inputting the first classification result C 1-2 , second classification result C 2-2 and third classification result C 3-2 of the second feature image into the second fusion module to obtain second prediction probability p 2-2 of the second feature image; and inputting the first classification result C 1-3 , second classification result C 2-3 and third classification result C 3-3 of the third feature image into the second fusion module to obtain second prediction probability p 2-3 of the third feature image; and the second prediction probability p 2-1 of the first feature image, the second prediction probability p 2-2 of the second feature image and the second prediction probability p 2-3 of the third feature image form the second prediction result.

[0036] Further, step S4 comprises:

[0037] S41: performing weighted processing on the first prediction probability p 1-1 and second prediction probability p 2-1 of the first feature image, the first prediction probability p 1-2 and second prediction probability p 2-2 of the second feature image, and the first prediction probability p 1-3 and second prediction probability p 2-3 of the third feature image to obtain prediction result P0 of the to-be-identified image belonging to a predetermined classification category, and P0 = ω1 × p 1-1 + ω2 × p 2-1 + ω3 × p 1-2 + ω4 × p 2-2 + ω5 × p 1-3 + ω6 × p 2-3 , wherein ω1, ω2, ω3, ω4, ω5 and ω6 are respectively p 1-1, p 2-1 , p 1-2 , p 2-2 , p 1-3 and p 2-3 weight, and ω1+ω2+ω3+ω4+ω5+ω6=1;

[0038] S42: scaling the prediction result P0 of the to-be-identified image belonging to the predetermined classification category by using an activation function to obtain a probability P of the to-be-identified image belonging to the predetermined classification category, and P0=ω1P1+ω2P2+ω3P3+ω4P4+ω5P5+ω6P6;

[0039] The application further provides an image recognition device, comprising:

[0040] an acquisition module, configured to acquire a to-be-identified image, and perform preprocessing on the to-be-identified image to obtain a first feature image, a second feature image and a third feature image;

[0041] a first prediction module, configured to input the first feature image, the second feature image and the third feature image into a CNN classification model to obtain a first prediction result;

[0042] a second prediction module, configured to input the first feature image, the second feature image and the third feature image into an ML classification model to obtain a second prediction result;

[0043] a third prediction module, configured to obtain a probability P of the to-be-identified image belonging to a predetermined classification category by using the first prediction result and the second prediction result, and P0=ω1P1+ω2P2+ω3P3+ω4P4+ω5P5+ω6P6;

[0044] The application further provides an image recognition device, comprising:

[0045] a processor;

[0046] a memory, configured to store a computer program executed by the processor;

[0047] When the processor executes the computer program, the image recognition method is realized.

[0048] The application further provides an application of the image recognition method in Alzheimer's disease prediction.

[0049] Compared with the prior art, the application has the following beneficial effects:

[0050] The image recognition method disclosed by the application realizes the extraction of effective features of the first feature image, the second feature image and the third feature image by using the CNN classification model and the ML classification model, and predicts the to-be-identified image by using the prediction results of the CNN classification model and the ML classification model, thereby significantly improving the recognition accuracy of the to-be-identified image.

[0051] For better understanding and implementation, the present application is described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 Flow chart of the image recognition method in the embodiment one of the present application;

[0053] Figure 2 Schematic diagram of non-maximum suppression;

[0054] Figure 3 Structural schematic diagram of the CNN classification model;

[0055] Figure 4 Structural schematic diagram of the first feature extraction layer;

[0056] Figure 5 Structural schematic diagram of the ML classification model;

[0057] Figure 6 Structural block diagram of the image recognition device in the embodiment one of the present application;

[0058] Figure 7 Comparison chart of the prediction result of the Alzheimer's disease prediction by the image recognition method in the embodiment one of the present application and the prediction result of the Alzheimer's disease prediction by other machine learning algorithms;

[0059] In the drawings, the reference signs are as follows:

[0060] 1, acquisition module; 11, acquisition unit; 12, preprocessing unit; 121, image denoising unit; 122, image enhancement unit; 123, image contour fitting unit; 124, image normalization unit; 125, image extraction unit; 2, first prediction module; 21, first feature extraction unit; 22, second feature extraction unit; 23, third feature extraction unit; 24, first fusion unit; 3, second prediction module; 31, classification unit; 32, second fusion unit; 4, third prediction module; 41, prediction unit; 42, scaling unit. DETAILED DESCRIPTION

[0061] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0062] The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the description of the application and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It also will be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0063] The following description refers to the accompanying drawings. In the following description, same numbers used in different drawings represent the same or similar elements unless otherwise described. The implementations described in the following example embodiments are not meant to represent all implementations consistent with the application. On the contrary, they are meant to represent just a few of the many implementations consistent with the application as claimed herein. In the description of the application, it should be understood that the terms "first", "second", "third", etc., merely identify similar objects for ease of description and are not meant to indicate or imply relative importance or significance.

[0064] In addition, in the description of the application, "multiple" means two or more, unless otherwise stated. "And / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0065] It should be understood that the embodiments of the application are not limited to the precise structures described and shown in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the embodiments of the application is limited only by the appended claims.

[0066] Embodiment One

[0067] With reference to Figure 1 The embodiment provides an image recognition method, comprising:

[0068] S1: acquiring a to-be-recognized image, and performing preprocessing on the to-be-recognized image to obtain a first feature image, a second feature image and a third feature image.

[0069] In the embodiment, since the edge of the to-be-recognized image is relatively fuzzy, it will affect the accuracy of the first feature image, the second feature image and the third feature image extraction. Therefore, in the embodiment, the to-be-recognized image needs to be preprocessed to obtain an image with clearer edges, so as to improve the accuracy of the first feature image, the second feature image and the third feature image extraction.

[0070] Specifically in this embodiment, the pre-processing of the image to be recognized includes:

[0071] S11: performing denoising processing on the image to be recognized by using an anisotropic diffusion filtering algorithm to obtain a first pre-processed image.

[0072] In order to reduce the influence of noise on the edge detection of the image to be recognized, it is necessary to first perform denoising processing on the image to be recognized to prevent false detection caused by noise. In this embodiment, the anisotropic diffusion filtering algorithm is used to perform denoising processing on the image to be recognized, because the anisotropic diffusion filtering algorithm mainly performs smoothing processing on the image to be recognized, which can remove the noise of the image to be recognized while preserving or even enhancing the edges of the image to be recognized.

[0073] In this embodiment, when the anisotropic diffusion filtering algorithm is used to perform denoising processing on the image to be recognized, the image to be recognized is first regarded as a heat field, and each pixel point is regarded as a heat flow. Then, it is determined whether to diffuse to the surrounding according to the relationship between each pixel point and the surrounding pixel points. For example, when the difference between a certain pixel point and the surrounding pixel points is large, it means that the pixel point is likely to be a boundary, and at this time, the pixel point does not diffuse in this direction, so as to preserve the boundary of the image to be recognized, and obtain the first pre-processed image.

[0074] Specifically, when the anisotropic diffusion filtering algorithm is used to perform denoising processing on the image to be recognized, iterative processing is needed, let I t denote the image to be recognized after t iterations, I t+1 denote the image to be recognized after (t+1) iterations, then:

[0075]

[0076] wherein (1) are all divergences, which are the partial derivatives of the image to be recognized after t iterations in N (north), S (south), E (east) and W (west) directions respectively, and the calculation formula is: x, y respectively denote the values of the image to be recognized after t iterations in x direction and y direction.

[0077] (2) cN x,y , cS x,y , cE x,y , cW x,y are the heat conduction coefficients of N (north), S (south), E (east) and W (west) directions respectively, and the calculation formula is: Moreover, cNx,y x,y x,y x,y The values of S, E, W and are all small, because when the difference between a pixel point and its surrounding pixel points is large, the pixel point is likely to be a boundary, and the surrounding pixel points are not likely to diffuse in this direction, thereby preserving the boundary. In addition, k is a constant related to the thermal conductivity cN x,y x,y x,y x,y The larger the value, the smoother the image and the less likely to preserve the boundary.

[0078] (3) λ is a constant related to the thermal conductivity cN x,y x,y x,y x,y The larger the value, the smoother the image.

[0079] S12: enhancing the first pre-processed image using the improved histogram equalization algorithm to obtain a second pre-processed image.

[0080] Because directly using the histogram equalization algorithm to enhance the first pre-processed image will not effectively improve the gray dynamic range of the first pre-processed image, only part of the gray layers of the enhanced first pre-processed image contain effective information, which affects the enhancement effect of the first pre-processed image, therefore, the improved histogram equalization algorithm is used to enhance the first pre-processed image in the present application.

[0081] Furthermore, the steps of enhancing the first pre-processed image using the improved histogram equalization algorithm include the following steps:

[0082] S121: performing gray transformation on the first pre-processed image using the histogram equalization algorithm to obtain a first gray image, and the first gray image includes L gray layers, and the gray range of each gray layer is 0, 1, 2, 3, …, L-1, respectively;

[0083] S122: performing gray mapping on the L gray layers of the first gray image according to a gray mapping formula to obtain the second pre-processed image, and the gray mapping formula is: wherein T k is the gray value of the kth gray layer in the second pre-processed image; S max is the maximum gray value of the first pre-processed image; S min is the minimum gray value of the first pre-processed image; and S k ​​​​​​​​​is a gray value of a kth gray level in the first preprocessed image; a is a brightness compensation coefficient, and 0≤a≤1.

[0084] S13: performing contour fitting processing on the second preprocessed image by using a Canny algorithm to obtain a third preprocessed image.

[0085] In this embodiment, step S13 includes the following sub-steps:

[0086] S131: performing gray-scale processing on the second preprocessed image to obtain a second gray-scale image; let Gray(x, y) represent a value of the second gray-scale image, then Gray(x, y) = 0.299 + 0.587 + 0.114, where R, G and B are respectively a red channel value, a blue channel value and a green channel value corresponding to the second preprocessed image; x is a numerical value of the second gray-scale image in the x direction; and y is a numerical value of the second gray-scale image in the y direction.

[0087] S132: performing Gaussian filtering processing on the second gray-scale image to obtain a filtered image, let K represent a Gaussian kernel when the Gaussian filtering processing is performed on the second gray-scale image, then where σ is a standard deviation, and the Gaussian filtering processing on the second gray-scale image can reduce the influence of noise on the edge detection result of the second preprocessed image.

[0088] S133: calculating a gradient amplitude G and a gradient direction θ of a pixel point of the filtered image by using a Sobel operator, and where G x is a gradient of the pixel point of the filtered image in the x direction, and G y is a gradient of the pixel point of the filtered image in the y direction, and * is a convolution symbol; and A is a 3x3 window in the filtered image.

[0089] Since the edge of the second preprocessed image can point to various directions, four operators are used to detect horizontal edges, vertical edges and diagonal edges of the second preprocessed image in this embodiment, and the Sobel operator returns the gradient G x of the pixel point of the filtered image in the x direction and the gradient G y of the pixel point of the filtered image in the y direction.

[0090] S134: performing non-maximum suppression processing on the edge of the filtered image by using the gradient amplitude G of the pixel point of the filtered image to determine an edge point, to obtain a contour fitting image.

[0091] Non-maximum suppression is an edge thinning technique, which acts on thinning "edges". After gradient calculation on the filtered image, the edges of the filtered image are still very fuzzy based on gradient values alone. Non-maximum suppression can suppress all gradient values other than local maximum values to 0, and the method of non-maximum suppression for each pixel point in the image is: comparing the gradient of the current pixel point with the gradients of two pixel points along the positive and negative gradient directions. If the gradient of the current pixel point is greater than the gradients of the two pixel points along the positive and negative gradient directions, the pixel point is retained as an edge point, otherwise the pixel point is suppressed.

[0092] In particular in the present embodiment, with reference to Figure 2 , g1, g2, g3, g4 are all pixel points, the diagonal line in the figure represents the gradient direction of pixel point c, pixel point dTmp1 and pixel point dTmp2 represent two pixel points along the positive and negative gradient directions respectively, and the gradients of pixel point dTmp1 and pixel point dTmp2 are calculated in a linear difference manner, specifically: G(dTmp1) = w1*G(g2) + (1-w1)*G(g1), G(dTmp2) = w2*G(g3) + (1-w2)*G(g4), wherein: distance(dTmp1, g2) represents the distance between pixel point dTmp1 and pixel point g2, distance(g1, g2) represents the distance between pixel point g1 and pixel point g2, distance(dTmp1, g3) represents the distance between pixel point dTmp2 and pixel point g3, and distance(g3, g4) represents the distance between pixel point g3 and pixel point g4.

[0093] When G(c) > G(dTmp1) and G(c) > G(dTmp2), pixel point c is retained and is a boundary point of the contour-fitted image; when G(c) ≤ G(dTmp1) or G(c) ≤ G(dTmp2), pixel point c is suppressed.

[0094] S135: Double-threshold detection is performed on the contour-fitted image, and pixel points with pixel values lower than a low threshold among the edge points of the contour-fitted image are suppressed to obtain the third pre-processed image.

[0095] Since the image fitting image is an image subjected to maximum suppression processing, the boundary thereof can accurately represent the actual edge of the image, but there are still edge pixels caused by noise and color changes. In order to solve these false responses, it is necessary to filter the edge pixels using gradient values to retain edge pixels with high gradient values. Specifically, in this embodiment, a double threshold algorithm is used to perform double threshold detection on the contour fitting image, and the pixel points with pixel values lower than the low threshold among the edge points of the contour fitting image are suppressed to obtain the third preprocessed image.

[0096] When the double threshold algorithm is used to perform double threshold detection on the contour fitting image, a plurality of threshold values are first determined; then, for each threshold value, the occurrence probability of each segmentation characteristic value is obtained using a histogram, and the threshold value is used to divide the segmentation characteristic values into two categories, and the intra-class variance and inter-class variance g of each category are calculated; then, the threshold value with the maximum inter-class variance g is taken as the best high threshold value, and the threshold value with the minimum intra-class variance is taken as the best low threshold value; finally, the gradient of the edge pixel point is compared with the best high threshold value and the best low threshold value respectively, and when the gradient of the edge pixel point is less than the best high threshold value and greater than the best low threshold value, the edge pixel point is retained; when the gradient of the edge pixel point is not less than the best high threshold value or the gradient of the edge pixel point is not greater than the best low threshold value, the edge pixel point is suppressed.

[0097] S14: performing normalization processing on the third preprocessed image to obtain a fourth preprocessed image.

[0098] Since the positions of the first feature image, the second feature image and the third feature image in different to-be-identified images are often not in the same region, in order to improve the extraction accuracy of the first feature image, the second feature image and the third feature image, it is necessary to perform normalization processing on the third preprocessed image, so that the first feature image, the second feature image and the third feature image are located in the same region, thereby improving the extraction accuracy of the first feature image, the second feature image and the third feature image.

[0099] S15: extracting the first feature image, the second feature image and the third feature image from the fourth preprocessed image.

[0100] S2: inputting the first feature image, the second feature image and the third feature image into a CNN classification model to obtain a first prediction result.

[0101] In this embodiment, with reference to Figure 3 , the CNN classification model comprises a feature extraction module and a first fusion module, and the feature extraction module comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer and a feature fusion layer.

[0102] In the embodiment, the first feature extraction layer, the second feature extraction layer and the third feature extraction layer have the same structure, and refer to Figure 4 , the first feature extraction layer includes four independent channels and a channel merging layer, wherein: the first channel includes a 1x1 convolution layer; the second channel includes a 1x1 convolution layer and a 3x3 convolution layer in cascade; the third channel includes a 1x1 convolution layer and a 5x5 convolution layer in cascade; the fourth channel includes a 3x3 pooling layer and a 1x1 convolution layer in cascade, and the 1x1 convolution layer of the first channel, the 3x3 convolution layer of the second channel, the 5x5 convolution layer of the third channel and the 1x1 convolution layer of the fourth channel are all connected to the channel merging layer.

[0103] In the embodiment, the four independent channels can extract information of the first feature image, the second feature image and the third feature image under different spatial sizes, improving the integrity and comprehensiveness of information extraction; moreover, the second channel and the third channel both first use a 1x1 convolution layer to process the input first feature image, second feature image and third feature image, which can reduce the number of input channels and reduce the complexity of the CNN classification model.

[0104] Moreover, the feature fusion layer is a 2x2 pooling layer, and in the embodiment, the CNN classification model can effectively extract the features of the input first feature image, second feature image and third feature image by using the convolution and pooling of the feature extraction module, which not only simplifies the complexity of the CNN classification model, but also reduces the risk of overfitting.

[0105] In the embodiment, step S2 includes the following sub-steps:

[0106] S21: input the first feature image into the first feature extraction layer and the feature fusion layer respectively to obtain the first feature F 1-1 and the second feature F 2-1 of the first feature image respectively;

[0107] S22: input the second feature image into the second feature extraction layer and the feature fusion layer respectively to obtain the first feature F 1-2 and the second feature F 2-2 of the second feature image respectively;

[0108] S23: input the third feature image into the third feature extraction layer and the feature fusion layer respectively to obtain the first feature F 1-3 and the second feature F 2-3 of the third feature image respectively;

[0109] S24: input the first feature F 1-1 and the second feature F 2-1 of the first feature image into the first fusion module to obtain a first prediction probability p 1-1 of the first feature image; input the first feature F 1-2 and the second feature F 2-2 of the second feature image into the first fusion module to obtain a first prediction probability p 1-2 of the second feature image; input the first feature F 1-3 and the second feature F 2-3 of the third feature image into the first fusion module to obtain a first prediction probability p 1-3 of the third feature image; the first prediction probability p 1-1 of the first feature image, the first prediction probability p 1-2 of the second feature image and the first prediction probability p 1-3 of the third feature image form the first prediction result.

[0110] S3: input the first feature image, the second feature image and the third feature image into an ML classification model to obtain a second prediction result.

[0111] Referring to Figure 5 , the ML classification model comprises an SVM classification model, a logistic regression classification model, a decision tree classification model and a second fusion module, because the logistic regression classification model can simply and efficiently realize a classification problem, and can cope with the problem of multiple collinearity through L2 regularization; the decision tree classification model can intuitively make decisions and process nonlinear features; the SVM classification model can process high-dimensional features, and its classification surface does not depend on all data, which can better avoid overfitting problems. It should be noted that in the present application, the SVM classification model, the logistic regression classification model and the decision tree classification model are all conventional classification models in the field, and the present embodiment does not limit them.

[0112] In the present embodiment, step S3 comprises the following sub-steps:

[0113] S31: input the first feature image into the SVM classification model, the logistic regression classification model and the decision tree classification model respectively to obtain a first classification result C 1-1 , a second classification result C 2-1 and a third classification result C 3-1The second feature image is input into the SVM classification model, the logistic regression classification model, and the decision tree classification model, respectively, to obtain the first classification result C of the second feature image. 1-2 Second classification result C 2-2 And the third classification result C 3-2 The third feature image is input into the SVM classification model, the logistic regression classification model, and the decision tree classification model, respectively, to obtain the first classification result C of the third feature image. 1-3 Second classification result C 2-3 And the third classification result C 3-3 ;

[0114] S32: The first classification result C of the first feature image... 1-1 Second classification result C 2-1 And the third classification result C 3-1 The second fusion module is input to obtain the second prediction probability p of the first feature image. 2-1 The first classification result C of the second feature image 1-2 Second classification result C 2-2 And the third classification result C 3-2 Inputting the second fusion module yields the second predicted probability p of the second feature image. 2-2 The first classification result C of the third feature image 1-3 Second classification result C 2-3 And the third classification result C 3-3 Inputting the second fusion module yields the second prediction probability p of the third feature image. 2-3 The second prediction probability p of the first feature image 2-1 The second prediction probability p of the second feature image 2-2 and the second prediction probability p of the third feature image 2-3 This forms the second prediction result.

[0115] S4: Use the first prediction result and the second prediction result to obtain the probability that the image to be identified belongs to a predetermined classification category.

[0116] In this embodiment, step S4 includes the following sub-steps:

[0117] S41: The first prediction probability p of the first feature image 1-1 Second prediction probability p 2-1 The first prediction probability p of the second feature image 1-2 Second prediction probability p 2-2 and the first prediction probability p of the third feature image 1-3and a second prediction probability p 2-3 The weighting processing is performed to obtain a prediction result P0 of the to-be-identified image belonging to a predetermined classification category, and P0 = 1 x p 1-1 + 2 x p 2-1 + 3 x p 1-2 + 4 x p 2-2 + 5 x p 1-3 + 6 x p 2-3 wherein: ω1, ω2, ω3, ω4, ω5 and ω6 are weights of p 1-1 , p 2-1 , p 1-2 , p 2-2 , p 1-3 and p 2-3 , and ω1+2+3+4+ω5+6 = 1.

[0118] S42: The prediction result P0 of the to-be-identified image belonging to a predetermined classification category is scaled by using an activation function to obtain a probability P of the to-be-identified image belonging to a predetermined classification category, and P = σ (P0).

[0119] Based on the image identification method provided in the embodiment, the prediction is made with reference to Figure 6 The embodiment further provides an image identification device, which comprises:

[0120] An acquisition module 1 is configured to acquire a to-be-identified image, and pre-process the to-be-identified image to obtain a first feature image, a second feature image and a third feature image.

[0121] A first prediction module 2 is configured to input the first feature image, the second feature image and the third feature image into a CNN classification model to obtain a first prediction result.

[0122] A second prediction module 3 is configured to input the first feature image, the second feature image and the third feature image into an ML classification model to obtain a second prediction result.

[0123] A third prediction module 4 is configured to obtain a probability of the to-be-identified image belonging to a predetermined classification category by using the first prediction result and the second prediction result.

[0124] In the embodiment, the acquisition module 1 comprises an acquisition unit 11 and a pre-processing unit 12, wherein: (1) the acquisition unit 11 is configured to acquire the to-be-identified image; and (2) the pre-processing unit 12 is configured to pre-process the to-be-identified image.

[0125] Moreover, the preprocessing unit 12 comprises an image denoising unit 121, an image enhancement unit 122, an image contour fitting unit 123, an image normalization unit 124 and an image extraction unit 125, wherein: the image denoising unit 121 performs denoising processing on the to-be-recognized image by using an anisotropic diffusion filtering algorithm to obtain a first preprocessed image; the image enhancement unit 122 performs enhancement processing on the first preprocessed image by using an improved histogram equalization algorithm to obtain a second preprocessed image; the image contour fitting unit 123 performs contour fitting processing on the second preprocessed image by using a Canny algorithm to obtain a third preprocessed image; the image normalization unit 124 performs normalization processing on the third preprocessed image to obtain a fourth preprocessed image; and the image extraction unit 125 extracts the first feature image, the second feature image and the third feature image from the fourth preprocessed image.

[0126] In this embodiment, the first prediction module 2 comprises a first feature extraction unit 21, a second feature extraction unit 22, a third feature extraction unit 23 and a first fusion unit 24, wherein:

[0127] The first feature extraction unit 21 inputs the first feature image into the first feature extraction layer and the feature fusion layer respectively to obtain first features F 1-1 and second features F 2-1 of the first feature image respectively.

[0128] The second feature extraction unit 22 inputs the second feature image into the second feature extraction layer and the feature fusion layer respectively to obtain first features F 1-2 and second features F 2-2 of the second feature image respectively.

[0129] The third feature extraction unit 23 inputs the third feature image into the third feature extraction layer and the feature fusion layer respectively to obtain first features F 1-3 and second features F 2-3 of the third feature image respectively.

[0130] The first fusion unit 24 inputs the first features F 1-1 and the second features F 2-1 of the first feature image into the first fusion module to obtain a first prediction probability p 1-1 of the first feature image; inputs the first features F 1-2 and the second features F 2-2 of the second feature image into the first fusion module to obtain a first prediction probability p 1-2 of the second feature image; and inputs the first features F 1-3Second feature F 2-3 Inputting into the first fusion module yields the first predicted probability p of the third feature image. 1-3 .

[0131] In this embodiment, the second prediction module 3 includes a classification unit 31 and a second fusion unit 32, wherein:

[0132] The classification unit 31 inputs the first feature image into the SVM classification model, the logistic regression classification model, and the decision tree classification model respectively, and obtains the first classification result C of the first feature image respectively. 1-1 Second classification result C 2-1 And the third classification result C 3-1 The second feature image is input into the SVM classification model, the logistic regression classification model, and the decision tree classification model, respectively, to obtain the first classification result C of the second feature image. 1-2 Second classification result C 2-2 And the third classification result C 3-2 The third feature image is input into the SVM classification model, the logistic regression classification model, and the decision tree classification model, respectively, to obtain the first classification result C of the third feature image. 1-3 Second classification result C 2-3 And the third classification result C 3-3 .

[0133] The second fusion unit 32 combines the first classification result C of the first feature image. 1-1 Second classification result C 2-1 And the third classification result C 3-1 The second fusion module is input to obtain the second prediction probability p of the first feature image. 2-1 The first classification result C of the second feature image 1-2 Second classification result C 2-2 And the third classification result C 3-2 Inputting the second fusion module yields the second predicted probability p of the second feature image. 2-2 The first classification result C of the third feature image 1-3 Second classification result C 2-3 And the third classification result C 3-3 Inputting the second fusion module yields the second prediction probability p of the third feature image. 2-3 The second prediction probability p of the first feature image 2-1 The second prediction probability p of the second feature image 2-2 and the second prediction probability p of the third feature image 2-3 This forms the second prediction result.

[0134] In the embodiment, the third prediction module 4 comprises a prediction unit 41 and a scaling unit 42, wherein the prediction unit 41 performs weighting processing on the first prediction probability p 1-1 and the second prediction probability p 2-1 of the first feature image, the first prediction probability p 1-2 and the second prediction probability p 2-2 of the second feature image, and the first prediction probability p 1-3 and the second prediction probability p 2-3 of the third feature image to obtain a prediction result P0 of the to-be-identified image belonging to a predetermined classification category, and P0 = ω1 × p 1-1 + ω2 × p 2-1 + ω3 × p 1-2 + ω4 × p 2-2 + ω5 × p1-3+ ω6 × p 2-3 , wherein ω1, ω2, ω3, ω4, ω5 and ω6 are weights of p 1-1 , p 2-1 , p 1-2 , p 2-2 , p 1-3 and p 2-3 , and ω1+2+3+4+ω5+6=1; the scaling unit 42 performs scaling processing on the prediction result p0 of the to-be-identified image belonging to the predetermined classification category by using an activation function to obtain a probability P of the to-be-identified image belonging to the predetermined classification category, and P = f (p0).

[0135] Based on the image recognition method provided in the embodiment, the present application further provides an image recognition device, which can be a server, a desktop computing device or a mobile computing device (for example, a laptop computer, a handheld computing device, a tablet computer, a netbook, etc.), or a terminal device, comprising:

[0136] a processor;

[0137] a memory for storing a computer program executed by the processor;

[0138] wherein the processor implements the image recognition method provided in the embodiment when executing the computer program.

[0139] Based on the image recognition method provided in the embodiment, the present application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed to implement the image recognition method provided in the present application.

[0140] The application can take the form of a computer program product implemented on one or more storage media (including but not limited to magnetic storage media, CD-ROMs, optical storage media, etc.) having a computer program code embodied in the medium. The computer usable storage medium includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0141] Embodiment two

[0142] Based on the image recognition method provided in embodiment one, the embodiment provides an application of the image recognition method in Alzheimer's disease prediction. Specifically, the image recognition method in embodiment one is used for Alzheimer's disease prediction, which includes the following steps:

[0143] S1: obtaining an image to be recognized, and pre-processing the image to be recognized to obtain a first feature image, a second feature image and a third feature image, wherein the image to be recognized is a coherent light tomography image, the first feature image is an NFL image, the second feature image is a GCL image, and the third feature image is an IPL image.

[0144] The retina is the extracranial extension of the central nervous system, which is the only neural and microvascular tissue that can be directly and non-invasively observed. It can be divided into ten layers from inside to outside, wherein the thickness of the nerve fiber layer (NFL) reflects the state of the retinal ganglion cells; the thickness of the ganglion cell layer (GCL) reflects the state of the bipolar cells; and the thickness of the inner plexiform layer (IPL) reflects the state of the amacrine cells. Moreover, existing research has shown that AD has a significant correlation with the thickness of NFL, GCL and IPL, indicating that NFL, GCL and IPL can be used as markers for AD prediction. Therefore, in the present application, the NFL image, the GCL image and the IPL image are obtained from the coherent light tomography image, which can accurately predict AD.

[0145] The optical coherence tomography (OCT) can present the multi-layer biological structure of the retina non-contact, non-invasively and quickly, so as to extract the NFL image, the GCL image and the IPL image from the OCT image. Moreover, in the embodiment, the OCT image is derived from a clinical trial or an open source database.

[0146] In the embodiment, the edge of the OCT image is fuzzy, which affects the accuracy of the extraction of the NFL image, the GCL image and the IPL image. Therefore, in the embodiment, the OCT image needs to be preprocessed to obtain an image with clearer edge, so as to improve the accuracy of the extraction of the NFL image, the GCL image and the IPL image.

[0147] In the embodiment, the process of preprocessing the OCT image is the same as the process of preprocessing the to-be-recognized image in the first embodiment, and includes the following sub-steps:

[0148] S11: The anisotropic diffusion filtering algorithm is used to perform denoising processing on the OCT image to obtain a first preprocessed image.

[0149] In order to reduce the influence of noise on the edge detection of the OCT image, the OCT image needs to be denoised to prevent false detection caused by noise. In the embodiment, the anisotropic diffusion filtering algorithm is used to denoise the OCT image, because the anisotropic diffusion filtering algorithm mainly performs smoothing processing on the OCT image to achieve denoising of the OCT image, and can remove the noise of the OCT image while preserving or even enhancing the edge of the OCT image.

[0150] In the embodiment, when the anisotropic diffusion filtering algorithm is used to denoise the OCT image, the OCT image is first regarded as a heat field, and each pixel point is regarded as a heat flow. Then, it is determined whether to diffuse to the surrounding according to the relationship between each pixel point and the surrounding pixel points. For example, when the difference between a certain pixel point and the surrounding pixel points is large, it is indicated that the pixel point is likely to be a boundary, and the pixel point does not diffuse in this direction to preserve the boundary of the OCT image, so as to obtain the first preprocessed image.

[0151] Specifically, when the anisotropic diffusion filtering algorithm is used to denoise the OCT image, iterative processing is needed, that is, I tdenotes an image of the OCT image after t iterations, I t+1 denotes an image of the OCT image after (t+1) iterations, then:

[0152]

[0153] wherein (1) are all divergence, respectively, partial derivatives of the OCT image after t iterations in N (north), S (south), E (east) and W (west) four directions, and the calculation formula is:

[0154] x, y respectively represent the values of the OCT image after t iterations in the x direction and the y direction.

[0155] (2) cN x,y , S x,y , E x,y , W x,y are the thermal conductivity coefficients of N (north), S (south), E (east) and W (west) four directions, respectively, and the calculation formula is: And, the values of cN x,y , S x,y , E x,y , W x,y are all small, because when the difference between a certain pixel point and the surrounding pixel points is large, the pixel point may be a boundary, and the surrounding pixel points are not possible to diffuse in this direction, so as to retain the boundary. In addition, k is a constant related to the thermal conductivity coefficient cN x,y , S x,y , E x,y , W x,y , the larger the value, the smoother the image, the less likely to retain the boundary.

[0156] (3) λ is a constant related to the thermal conductivity coefficient cN x,y , D x,y , E x,y , W x,y , the larger the value, the smoother the image.

[0157] S12: using the improved histogram equalization algorithm to enhance the first preprocessed image to obtain a second preprocessed image.

[0158] Since the direct adoption of the histogram equalization algorithm to enhance the first pre-processed image will result in the gray dynamic range of the first pre-processed image not being effectively improved, which makes only part of the gray scale of the enhanced first pre-processed image contain effective information, and influences the enhancement effect of the first pre-processed image, therefore, the improved histogram equalization algorithm is used to enhance the first pre-processed image in the application.

[0159] Moreover, the improved histogram equalization algorithm is used to enhance the first pre-processed image, which includes the following steps:

[0160] S121: the first pre-processed image is subjected to gray scale transformation by using the histogram equalization algorithm to obtain a first gray scale image, and the first gray scale image includes L gray scales, and the gray scale range of each gray scale is 0, 1, 2, 3, …, L-1 respectively;

[0161] S122: the L gray scales of the first gray scale image are subjected to gray scale mapping according to a gray scale mapping formula respectively to obtain the second pre-processed image, and the gray scale mapping formula is: Wherein: T k is the gray scale value of the kth gray scale in the second pre-processed image; S max is the maximum gray scale value of the first pre-processed image; S min is the minimum gray scale value of the first pre-processed image; S k is the gray scale value of the kth gray scale in the first pre-processed image; a is a brightness compensation coefficient, and 0≤a≤1.

[0162] S13: the second pre-processed image is subjected to contour fitting processing by using the Canny algorithm to obtain a third pre-processed image.

[0163] In the embodiment, step S13 includes the following sub-steps:

[0164] S131: the second pre-processed image is subjected to gray scale processing to obtain a second gray scale image; let Gray(x, y) represent the value of the second gray scale image, then Gray(x, y)=0.299+0.587+0.114, wherein: R, G and B are respectively the red channel value, the blue channel value and the green channel value corresponding to the second pre-processed image; x is the numerical value of the second gray scale image in the x direction; y is the numerical value of the second gray scale image in the y direction.

[0165] S132: the second gray scale image is subjected to Gaussian filtering processing to obtain a filtered image, and let K represent the Gaussian kernel when the second gray scale image is subjected to Gaussian filtering processing, then Wherein: σ is the standard deviation, and the Gaussian filtering processing on the second gray image can reduce the influence of noise on the edge detection result of the second preprocessed image.

[0166] S133: Gradient amplitude G and gradient direction θ of the pixel point of the filtered image are calculated by using Sobel operator, and G = sqrt(G x 2 + G y 2 ), Wherein: G x is the gradient of the pixel point of the filtered image in the x direction, and G y is the gradient of the pixel point of the filtered image in the y direction, and * is the convolution symbol; A is the 3*3 window in the filtered image.

[0167] Since the edge of the second preprocessed image can point to various directions, four operators are used to detect the horizontal edge, vertical edge and diagonal edge of the second preprocessed image in the embodiment, and Sobel operator is used to return the gradient G x of the pixel point of the filtered image in the x direction and the gradient G y of the pixel point of the filtered image in the y direction.

[0168] S134: The edges of the filtered image are subjected to non-maximum suppression processing by using the gradient amplitude G of the pixel point of the filtered image to determine the edge points, and the contour fitting image is obtained.

[0169] Non-maximum suppression is an edge thinning technique, which functions to thin the edge. After the gradient calculation of the filtered image, the edge extraction of the filtered image based on the gradient value is still very fuzzy, and all the gradient values other than the local maximum value can be suppressed to 0 by using non-maximum suppression, and the method of non-maximum suppression for each pixel point in the image is that the gradient of the current pixel point is compared with the gradients of the two pixel points along the positive and negative gradient directions, if the gradient of the current pixel point is greater than the gradients of the two pixel points along the positive and negative gradient directions, the pixel point is retained as the edge point, otherwise the pixel point will be suppressed.

[0170] Specifically in the embodiment, reference is made to Figure 2, g1, g2, g3, g4 are all pixel points, the diagonal line in the figure represents the gradient direction of the pixel point c, the pixel point dTmp1 and the pixel point dTmp2 represent two pixel points along the positive and negative gradient directions respectively, and the gradients of the pixel point dTmp1 and the pixel point dTmp2 are calculated in a linear difference manner, specifically: G(dTmp1) = w1*G(g2) + (1-w1)*G(g1), G(dTmp2) = w2*G(g3) + (1-w2)*G(g4), wherein: distance(dTmp1, g2) represents the distance between the pixel point dTmp1 and the pixel point g2, distance(g1, g2) represents the distance between the pixel point g1 and the pixel point g2, distance(dTmp1, g3) represents the distance between the pixel point dTmp2 and the pixel point g3, and distance(g3, g4) represents the distance between the pixel point g3 and the pixel point g4.

[0171] When G(c)>(dTmp1) and G(c)>(dTmp2), the pixel point c is retained and is a boundary point of the contour fitting image; when G(c)≤G(dTmp1) or G(c)≤G(dTmp2), the pixel point c is suppressed.

[0172] S135: Double-threshold detection is performed on the contour fitting image, and pixel points with pixel values lower than a low threshold among edge points of the contour fitting image are suppressed to obtain the third preprocessed image.

[0173] Since the image fitting image is an image subjected to maximum value suppression processing, its boundary can accurately represent the actual edge of the image, but there are still edge pixels caused by noise and color changes. In order to solve these miscellaneous responses, it is necessary to filter the edge pixels by gradient value, and include edge pixels with higher gradient value. Specifically, in this embodiment, a double-threshold algorithm is used to perform double-threshold detection on the contour fitting image, and pixel points with pixel values lower than a low threshold among edge points of the contour fitting image are suppressed to obtain the third preprocessed image.

[0174] When the double-threshold algorithm is used to perform double-threshold detection on the contour-fitted image, a plurality of threshold values are first determined; then, for each threshold value, the occurrence probability of each segmentation characteristic value is obtained by using a histogram, and the segmentation characteristic values are divided into two classes by using the threshold value, and the intra-class variance and the inter-class variance g of each class are calculated; then, the threshold value with the largest inter-class variance g is taken as the optimal high threshold value, and the threshold value with the smallest intra-class variance is taken as the optimal low threshold value; finally, the gradient of the edge pixel point is compared with the optimal high threshold value and the optimal low threshold value respectively, and when the gradient of the edge pixel point is less than the optimal high threshold value and greater than the optimal low threshold value, the edge pixel point is retained; when the gradient of the edge pixel point is not less than the optimal high threshold value or the gradient of the edge pixel point is not greater than the optimal low threshold value, the edge pixel point is suppressed.

[0175] S14: performing normalization processing on the third preprocessed image to obtain a fourth preprocessed image.

[0176] Since the positions of the NFL image, the GCL image and the IPL image in different OCT images are often not in the same region, in order to improve the accuracy of extracting the NFL image, the GCL image and the IPL image, the third preprocessed image needs to be normalized so that the NFL image, the GCL image and the IPL image are located in the same region, thereby improving the accuracy of extracting the NFL image, the GCL image and the IPL image.

[0177] S15: extracting the NFL image, the GCL image and the IPL image from the fourth preprocessed image.

[0178] S2: inputting the NFL image, the GCL image and the IPL image into a CNN classification model to obtain a first prediction result.

[0179] In this embodiment, with reference to Figure 3 , the CNN classification model comprises a feature extraction module and a first fusion module, and the feature extraction module comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer and a feature fusion layer.

[0180] In this embodiment, the first feature extraction layer, the second feature extraction layer and the third feature extraction layer have the same structure, with reference to Figure 4, the first feature extraction layer comprises four independent channels and a channel merging layer, wherein: the first channel comprises a 1x1 convolutional layer; the second channel comprises a 1x1 convolutional layer and a 3x3 convolutional layer in cascade; the third channel comprises a 1x1 convolutional layer and a 5x5 convolutional layer in cascade; the fourth channel comprises a 3x3 pooling layer and a 1x1 convolutional layer in cascade, and the 1x1 convolutional layer of the first channel, the 3x3 convolutional layer of the second channel, the 5x5 convolutional layer of the third channel and the 1x1 convolutional layer of the fourth channel are all connected to the channel merging layer.

[0181] The four independent channels can extract information of the NFL image, the GCL image and the IPL image in different spatial sizes, thereby improving the integrity and comprehensiveness of information extraction; moreover, the second channel and the third channel both use a 1x1 convolutional layer to process the input NFL image, GCL image and IPL image, which can reduce the number of input channels and reduce the complexity of the CNN classification model.

[0182] Moreover, the feature fusion layer is a 2x2 pooling layer, and in the embodiment, the CNN classification model can effectively extract features of the input NFL image, GCL image and IPL image by using convolution and pooling of the feature extraction module, thereby simplifying the complexity of the CNN classification model and reducing the risk of overfitting.

[0183] Specifically in the embodiment, step S2 comprises the following sub-steps:

[0184] S21: inputting the NFL image into the first feature extraction layer and the feature fusion layer respectively to obtain first features F 1-1 and second NFL features F 2-1 of the NFL image respectively;

[0185] S22: inputting the GCL image into the second feature extraction layer and the feature fusion layer respectively to obtain first features F 1-2 and second GCL features F 2-2 of the GCL image respectively;

[0186] S23: inputting the IPL image into the third feature extraction layer and the feature fusion layer respectively to obtain first features F 1-2 and second NFL features F 2-3 of the IPL image respectively;

[0187] S24: inputting the first features F 1-1 and the second NFL features F 2-1Inputting into the first fusion module yields the first predicted probability p of the NFL image. 1-1 The first feature F of the GCL image 1-2 Second GCL feature F 2-2 Inputting into the first fusion module yields the first predicted probability p of the GCL image. 1-2 The first feature F of the IPL image 1-2 Second NFL feature F 2-3 Inputting into the first fusion module yields the first prediction probability p of the IPL image. 1-3 The first predicted probability p of the NFL image 1-1 The first prediction probability p of the GCL image 1-2 and the first prediction probability p of the IPL image 1-3 The first prediction result is formed.

[0188] S3: Input the NFL image, the GCL image, and the IPL image into the ML classification model to obtain the second prediction result.

[0189] Reference Figure 5 The ML classification model includes an SVM classification model, a logistic regression classification model, a decision tree classification model, and a second fusion module. This is because the logistic regression classification model can simply and efficiently solve classification problems, and can address multicollinearity issues through L2 regularization; the decision tree classification model can intuitively make decisions and handle nonlinear features; the SVM classification model can handle high-dimensional features, and its classification surface does not depend on all data, thus effectively avoiding overfitting. It should be noted that in this invention, the SVM classification model, the logistic regression classification model, and the decision tree classification model are all conventional classification models in the field, and this embodiment does not limit them.

[0190] In this embodiment, step S3 includes the following sub-steps:

[0191] S31: Input the NFL image into the SVM classification model, the logistic regression classification model, and the decision tree classification model respectively, and obtain the first classification result C of the NFL image respectively. 1-1 Second classification result C 2-1 And the third classification result C 3-1 The GCL image is input into the SVM classification model, the logistic regression classification model, and the decision tree classification model, respectively, to obtain the first classification result C of the GCL image. 1-2 Second classification result C 2-2 And the third classification result C3-2 ; inputting the IPL image into the SVM classification model, the logistic regression classification model and the decision tree classification model respectively to obtain first classification results C 1-3 , second classification results C 2-3 and third classification results C 3-3 of the GCL image respectively.

[0192] S32: inputting the first classification results C 1-1 , the second classification results C 2-1 and the third classification results C 3-1 of the NFL image into the second fusion module to obtain second prediction probabilities p 2-1 of the NFL image; inputting the first classification results C 1-2 , the second classification results C 2-2 and the third classification results C 3-2 of the GCL image into the second fusion module to obtain second prediction probabilities p 2-2 of the GCL image; inputting the first classification results C 1-3 , the second classification results C 2-3 and the third classification results C 3-3 of the IPL image into the second fusion module to obtain second prediction probabilities p 2-3 of the IPL image; and the second prediction probabilities p 2-1 of the NFL image, the second prediction probabilities p 2-2 of the GCL image and the second prediction probabilities p 2-3 of the IPL image form the second prediction results.

[0193] S4: obtaining the probability that the to-be-identified image belongs to Alzheimer's disease by using the first prediction results and the second prediction results.

[0194] In the embodiment, step S4 includes the following sub-steps:

[0195] S41: performing weighted processing on the first prediction probabilities p 1-1 and the second prediction probabilities p 2-1 of the NFL image, the first prediction probabilities p 1-2 and the second prediction probabilities p 2-2 of the GCL image, and the first prediction probabilities p 1-3 and the second prediction probabilities p 2-3 of the IPL image to obtain a prediction result P0 that the to-be-identified image belongs to Alzheimer's disease, and P0 = ω1 × p 1-1 + ω2 × p 2-1 + ω3 × p 1-2 + ω4 × p 2-2+ ω5 * p 1-3 + ω6 * p 2-3 wherein: ω1, ω2, ω3, ω4, ω5 and ω6 are weights of p 1-1 , p 2-1 , p 1-3 , p 2-3 , p 1-4 and p 2-4 , and ω1+ω2+ω3+ω4+ω5+ω6=1.

[0196] S42: scaling the prediction result P0 of the to-be-identified image belonging to Alzheimer's disease by using an activation function to obtain a probability P of the to-be-identified image belonging to Alzheimer's disease, and P0 < P < 1.

[0197] In order to verify the accuracy of the image recognition method for predicting Alzheimer's disease in the embodiment one, the present application uses the retinal coherence tomography images of 267 AD patients as to-be-identified images, and uses the logistic regression classification model, the SVM classification model, the ML classification model, the CNN classification model, the random forest classification model and the image recognition method in the embodiment one to predict Alzheimer's disease, and calculates the prediction accuracy by using the prediction result and the true result, referring to Figure 6 It can be seen that the prediction accuracy of the image recognition method for predicting Alzheimer's disease in the embodiment one is much higher than that of other prediction methods, which proves that the accuracy of the image recognition method for predicting Alzheimer's disease in the embodiment one is high.

[0198] The above-described embodiments only express the preferred embodiments of the present application, which are described in detail and specifically, but should not be understood as the limitation of the scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, and the present application also intends to include these modifications and improvements.

Claims

1. An image recognition method applied to Alzheimer's disease prediction, characterized in that, include: S1: Obtain the image to be identified, and sequentially perform anisotropic diffusion filtering algorithm, improved histogram equalization algorithm, Canny algorithm and normalization preprocessing on the image to be identified to obtain a fourth preprocessed image, and extract a first feature image, a second feature image and a third feature image from the fourth preprocessed image; wherein: the image to be identified is a coherent optical tomography image, the first feature image is an NFL image, the second feature image is a GCL image, and the third feature image is an IPL image; S2: Input the first feature image, the second feature image, and the third feature image into the CNN classification model respectively to obtain the first predicted probability of the first feature image. The first prediction probability of the second feature image The first prediction probability of the third feature image This forms the first prediction result; S3: Input the first feature image, the second feature image, and the third feature image into the ML classification model respectively to obtain the second predicted probability of the first feature image. The second prediction probability of the second feature image The second prediction probability of the third feature image This forms the second prediction result; S4: Use the first prediction result and the second prediction result to obtain the probability that the image to be identified belongs to a predetermined classification category.

2. The image recognition method according to claim 1, characterized in that, Step S1, the preprocessing of the image to be identified includes: S11: The image to be identified is denoised using an anisotropic diffusion filtering algorithm to obtain a first preprocessed image; S12: The first preprocessed image is enhanced using an improved histogram equalization algorithm to obtain a second preprocessed image; S13: Use the Canny algorithm to perform contour fitting on the second preprocessed image to obtain the third preprocessed image; S14: Normalize the third preprocessed image to obtain the fourth preprocessed image; S15: Extract the first feature image, the second feature image, and the third feature image from the fourth preprocessed image.

3. The image recognition method according to claim 2, characterized in that, Step S12 includes: S121: The first preprocessed image is subjected to grayscale transformation using a histogram equalization algorithm to obtain a first grayscale image, wherein the first grayscale image includes... One grayscale layer; S122: For the first grayscale image Each gray level is mapped to gray level according to the gray level mapping formula to obtain the second preprocessed image, and the gray level mapping formula is: ,in: For the second preprocessed image, the first The grayscale values ​​of each grayscale layer; The maximum grayscale value of the first preprocessed image; The minimum grayscale value of the first preprocessed image; For the first preprocessed image, the first The grayscale values ​​of each grayscale layer; This is the brightness compensation coefficient, and .

4. The image recognition method according to claim 2, characterized in that, Step S13 includes: S131: Perform grayscale processing on the second preprocessed image to obtain a second grayscale image; let The value of the second grayscale image is then... ,in: , and These are the red channel values, blue channel values, and green channel values ​​corresponding to the second preprocessed image, respectively. For the second grayscale image in The numerical value of the direction; For the second grayscale image in The numerical value of the direction; S132: Perform Gaussian filtering on the second grayscale image to obtain a filtered image, let... The value of the filtered image is then... ,in: For parameters; S133: Calculate the gradient magnitude of the pixels in the filtered image using the Sobel operator. and gradient direction ,and: , ,in: For the pixels of the filtered image in The gradient in the direction, and ; For the pixels of the filtered image in The gradient in the direction, and ; The symbol for convolution; S134: Utilize the gradient magnitude of the pixels in the filtered image. Non-maximum suppression processing is applied to the edges of the filtered image to determine edge points, resulting in a contour-fitted image. S135: Perform double threshold detection on the contour fitting image, and suppress pixels with pixel values ​​lower than the low threshold in the edge points of the contour fitting image to obtain the third preprocessed image.

5. The image recognition method according to any one of claims 1-4, characterized in that, In step S2, the CNN classification model includes a feature extraction module and a first fusion module. The feature extraction module includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, and a feature fusion layer. Step S2 includes: S21: Input the first feature image into the first feature extraction layer and the feature fusion layer respectively to obtain the first feature of the first feature image. Second feature ; S22: Input the second feature image into the second feature extraction layer and the feature fusion layer respectively to obtain the first feature of the second feature image. Second feature ; S23: Input the third feature image into the third feature extraction layer and the feature fusion layer respectively to obtain the first feature of the third feature image. Second feature ; S24: Transfer the first feature of the first feature image Second feature Inputting the first fusion module yields the first predicted probability of the first feature image. ; the first feature of the second feature image Second feature The first fusion module is input to obtain the first predicted probability of the second feature image. The first feature of the third feature image Second feature Inputting into the first fusion module yields the first predicted probability of the third feature image. The first predicted probability of the first feature image The first prediction probability of the second feature image and the first prediction probability of the third feature image The first prediction result is formed.

6. The image recognition method according to claim 5, characterized in that, In step S3, the ML classification model includes an SVM classification model, a logistic regression classification model, a decision tree classification model, and a second fusion module; Step S3 includes: S31: Input the first feature image into the SVM classification model, the logistic regression classification model, and the decision tree classification model respectively, and obtain the first classification result of the first feature image respectively. Second classification results and third classification results The second feature image is input into the SVM classification model, the logistic regression classification model, and the decision tree classification model, respectively, to obtain the first classification result of the second feature image. Second classification results and third classification results The third feature image is input into the SVM classification model, the logistic regression classification model, and the decision tree classification model, respectively, to obtain the first classification result of the third feature image. Second classification results and third classification results ; S32: The first classification result of the first feature image... Second classification results and third classification results The second fusion module is input to obtain the second predicted probability of the first feature image. The first classification result of the second feature image; Second classification results and third classification results The second fusion module is input to obtain the second predicted probability of the second feature image. The first classification result of the third feature image; Second classification results and third classification results The second fusion module is input to obtain the second prediction probability of the third feature image. The second prediction probability of the first feature image The second prediction probability of the second feature image and the second prediction probability of the third feature image This forms the second prediction result.

7. The image recognition method according to claim 6, characterized in that, Step S4 includes: S41: First prediction probability of the first feature image Second prediction probability The first prediction probability of the second feature image Second prediction probability and the first prediction probability of the third feature image. Second prediction probability After weighted processing, the prediction result of whether the image to be identified belongs to the predetermined classification category is obtained. ,and ,in: , , , , and They are respectively , , , , and The weights, and; ; S42: Utilize an activation function to predict whether the image to be identified belongs to a predetermined classification category. The image to be identified is scaled to obtain the probability that it belongs to a predetermined classification category. ,and .

8. An image recognition device for predicting Alzheimer's disease, characterized in that, include: An acquisition module is used to acquire an image to be identified, and sequentially perform anisotropic diffusion filtering algorithm, improved histogram equalization algorithm, Canny algorithm and normalization preprocessing on the image to be identified to obtain a fourth preprocessed image, and extract a first feature image, a second feature image and a third feature image from the fourth preprocessed image; wherein: the image to be identified is a coherent optical tomography image, the first feature image is an NFL image, the second feature image is a GCL image, and the third feature image is an IPL image; The first prediction module is used to input the first feature image, the second feature image, and the third feature image into the CNN classification model respectively to obtain the first prediction probability of the first feature image. The first prediction probability of the second feature image The first prediction probability of the third feature image This forms the first prediction result; The second prediction module is used to input the first feature image, the second feature image, and the third feature image into the ML classification model respectively to obtain the second prediction probability of the first feature image. The second prediction probability of the second feature image The second prediction probability of the third feature image This forms the second prediction result; The third prediction module is used to obtain the probability that the image to be identified belongs to a predetermined classification category using the first prediction result and the second prediction result.

9. An image recognition device, characterized in that, include: processor; Memory for storing computer programs executed by the processor; The processor executes the computer program to implement the image recognition method according to any one of claims 1-7.

Images