Training method and electronic device for ophthalmic disease prediction model

Abstract

The present application provides a training method and electronic device for an ophthalmic disease prediction model, the method comprising: obtaining ophthalmic image data and label data corresponding to the ophthalmic image data; preprocessing the ophthalmic image data to obtain multiple image blocks at different scale levels; learning the spatial offset of each sampling point on each image block through a convolution operation of a stacked adaptive gradient modulation convolution layer, adjusting the original convolution kernel weights of the stacked adaptive gradient modulation convolution layer based on the spatial offset, transforming the multiple image blocks at different scale levels to obtain a morphological enhancement feature map; inputting the morphological enhancement feature map into a hybrid attention feature extraction module to obtain a microlesion enhancement feature map and reducing the dimension to obtain a feature vector; inputting the feature vector and label data into a fully connected classifier to output a disease category prediction probability, until a training stop condition is met, at which point the multi-branch convolutional neural network completes training to form an ophthalmic disease prediction model, thereby improving the accuracy of identifying microlesions.

Description

Technical Field

[0001] The present application relates to the field of medical data processing technology, and in particular to a training method and electronic equipment for an ophthalmic disease prediction model. Background Art

[0002] Although existing ophthalmic image processing methods can provide certain auxiliary diagnosis to some extent, they still have many shortcomings, especially in the processing of image data, identification of lesion features and accuracy of models.

[0003] With the continuous development of medical imaging technology, the quality and resolution of ophthalmic images have been greatly improved, providing richer information for disease diagnosis. However, these high-resolution images also bring new challenges, especially in processing tiny lesions in the image, complex lesion morphology, and image consistency under different lighting conditions. Traditional methods of simple downsampling and global normalization often lose detailed information or cannot eliminate lighting differences caused by equipment or shooting angles. These problems make it difficult to accurately identify lesion areas in ophthalmic images, especially tiny lesions such as microaneurysms and exudates. Traditional methods perform poorly in detecting these tiny lesions. Summary of the Invention

[0004] The present application provides a training method and electronic equipment for an ophthalmic disease prediction model, which can improve the accuracy of identifying micro-lesions such as microaneurysms and exudates.

[0005] The first aspect of the present application provides a training method for an ophthalmic disease prediction model, which is applied to the training of a multi-branch convolutional neural network, which includes a stacked adaptive gradient modulated convolution layer, a hybrid attention feature extraction module and a fully connected classifier. The method includes: obtaining ophthalmic image data and label data corresponding to the ophthalmic image data; preprocessing the ophthalmic image data to obtain image blocks of multiple scale levels; learning the spatial offset of each sampling point on each image block through the convolution operation of the stacked adaptive gradient modulated convolution layer, adjusting the original convolution kernel weights of the stacked adaptive gradient modulated convolution layer based on the spatial offset, and transforming the image blocks of multiple scale levels to obtain a morphological enhancement feature map; inputting the morphological enhancement feature map into the hybrid attention feature extraction module to obtain a small lesion enhancement feature map and reducing the dimension to obtain a feature vector; inputting the feature vector and the label data into the fully connected classifier, and outputting the disease category prediction probability. When the training stop condition is met, the multi-branch convolutional neural network completes the training to form an ophthalmic disease prediction model.

[0006] In some embodiments of the first aspect, the multi-branch convolutional neural network also includes a cross-scale feature fusion module, which adjusts the original convolution kernel weights of the stacked adaptive gradient modulation convolution layer based on the spatial offset, and transforms multiple image blocks at different scale levels to obtain a morphological enhancement feature map, including: sampling based on the spatial offset to obtain an offset eigenvalue of each sampling point after offset; performing an inner product of the offset eigenvalue and the transpose of the learnable weight vector to obtain a score for each sampling point; normalizing the scores of all sampling points by a Softmax function to obtain a dynamic modulation coefficient for each sampling point; multiplying the dynamic modulation coefficient of each sampling point, the original convolution kernel weight of the corresponding stacked adaptive gradient modulation convolution layer, and the offset eigenvalue item by item, and summing the products of all sampling points to generate a first feature map, the first feature map recording the feature activation strength of all sampling points; inputting multiple first feature maps corresponding to the image blocks at multiple scale levels into the cross-scale feature fusion module, and outputting a morphological enhancement feature map.

[0007] In some embodiments of the first aspect, the hybrid attention feature extraction module includes a channel attention weight determination module, a spatial attention weight determination module, and a matching module. The morphological enhancement feature map is input into the hybrid attention feature extraction module to obtain a small lesion enhancement feature map and the dimensionality is reduced to obtain a feature vector. The channel attention weight determination module includes a first fully connected layer and a second fully connected layer. The channel attention weight determination module performs the following steps:

[0008] a.1 Perform global average pooling on the morphological enhancement feature map in the channel dimension to obtain the global response value of each channel;

[0009] a.2 Use the ReLU function to map the product of the global response value and the weight matrix of the first fully connected layer into a value in the non-negative semi-axis interval;

[0010] a.3 Use the Sigmoid function to transform the product of the value mapped to the non-negative semi-axis interval and the weight matrix of the second fully connected layer into a channel attention weight vector;

[0011] The following steps are performed by the spatial attention weight determination module:

[0012] b.1 Perform maximum pooling and global average pooling on the morphological enhancement feature map in the channel dimension to obtain two two-dimensional spatial mapping maps;

[0013] b.2. Concatenate the two 2D spatial maps and perform a convolution operation to obtain the second feature map.

[0014] b.3 Use the Sigmoid function to transform the second feature map into a spatial weight map;

[0015] b.4 Weight each spatial position in the morphological enhancement feature map according to the spatial weight map to obtain the spatial attention weight map.

[0016] Perform the following steps to match the module:

[0017] c.1 Broadcast and match the channel attention weight vector and the spatial attention weight map to the same dimension as the morphological enhancement feature map;

[0018] c.2. Multiply the reduced channel attention weight vector and spatial attention weight map with the morphological enhancement feature map to obtain the channel attention weighted feature map and spatial attention weighted feature map, respectively.

[0019] c.3 Add the channel attention weighted feature map, spatial attention weighted feature map, and morphological enhancement feature map to obtain the small lesion enhancement feature map and reduce the dimension to obtain the feature vector.

[0020] In some embodiments of the first aspect, a small lesion enhancement feature map and dimensionality reduction to obtain a feature vector include: performing a 1x1 convolution operation on the small lesion enhancement feature map to generate a response map; binarizing the response map with a preset threshold to obtain a mask map, the mask map being used to locate the lesion area to be enhanced in the small lesion enhancement feature map; multiplying the inverse of the mask map by the image block element-by-element to obtain an original retained area image; inputting the coverage area of ​​the mask map into a preset lightweight generation network to obtain generated content; multiplying the generated content and the mask map element-by-element to obtain a synthesized mask area image; adding the original retained area image and the synthesized mask area image to obtain an enhanced small lesion enhancement feature map and reducing the dimensionality to obtain a feature vector.

[0021] In some embodiments of the first aspect, the feature vector and label data are input into a fully connected classifier, and the disease category prediction probability is output until the training stop condition is met. The multi-branch convolutional neural network completes the training, including: dividing the feature vector into anchor sample features, similar positive sample features and negative sample features according to the label data; calculating the feature similarity between the anchor sample features and the similar positive sample features and negative sample features respectively; comparing the feature similarities to obtain the feature comparison loss of each ophthalmic image data; determining the disease category according to the disease category prediction probability output by the fully connected classifier; calculating the classification loss of each ophthalmic image data by weighted cross entropy of the classification confidence score corresponding to the disease category prediction probability and the label data, wherein the category weight in the calculation is determined according to the inverse square root of the number of samples of the ophthalmic image data corresponding to the disease category; linearly weighting the feature comparison loss and the classification loss to obtain the total loss function value of each ophthalmic image data; when the downward trend of the total loss function value is less than the preset trend value, determining that the multi-branch convolutional neural network has completed training.

[0022] In some embodiments of the first aspect, the method further includes: in each training step, calculating the L2 norm of the loss gradient of each ophthalmic image data on the target layer parameters based on the total loss function value, and taking the average of the calculation results as the connection importance index of the target layer parameters at the current iteration number; generating a sparse mask of the target layer parameters based on the connection importance index; calculating the gradient of the total loss function value with respect to the target layer parameters; performing element-wise multiplication of the sparse mask and the gradient, multiplying the result by a preset learning rate, and then subtracting the result from the matrix of the target layer parameters to obtain updated target layer parameters, wherein the parameters whose connection importance index in the updated target layer parameters is higher than the preset index are updated, and the parameters whose connection importance index is lower than or equal to the preset index remain unchanged.

[0023] In some embodiments of the first aspect, the method further includes: freezing the output function of the hybrid feature extraction module, updating only the parameters of the fully connected classifier head until the total loss function value is less than a preset value, and completing the parameter update; unfreezing the output function of the hybrid feature extraction module, updating the fully connected classifier head and the adaptive gradient modulated convolution layer of the hybrid feature extraction module, and completing the parameter update until the total loss function value is less than the preset value; unfreezing all layers of the multi-branch convolutional neural network, reducing the learning rate to one tenth of the preset learning rate, and updating the parameters of all layers with a smaller step size until the total loss function value is less than the preset value, and completing the parameter update.

[0024] In some embodiments of the first aspect, obtaining ophthalmic image data and label data corresponding to the ophthalmic image data includes: obtaining ophthalmic image data and labeling the ophthalmic image data to obtain label data, wherein the label data is labeled with a location and a category of a lesion area; preprocessing the ophthalmic image data to obtain multiple image blocks at different scale levels, including: at different scale levels, according to a preset sliding step size, cropping the ophthalmic image data through a sliding window to obtain multiple image blocks at different scales, wherein the size of the sliding window is determined by a scale index corresponding to the scale level; for each image block, with a target pixel in the image block as the center, extracting a neighborhood window of a preset size around the target pixel, calculating the mean and standard deviation of all pixel values ​​in the neighborhood window, subtracting the mean from the target pixel value, and then dividing the result by the sum of the standard deviation and a very small constant, thereby converting the multiple image blocks at different scale levels into a normalized multiple image blocks at different scale levels.

[0025] In some embodiments of the first aspect, the method further includes: acquiring new ophthalmic image data and inputting it into an ophthalmic disease prediction model, and outputting a disease category prediction probability.

[0026] A second aspect of the present application provides an electronic device, comprising: a processor and a memory; the memory is coupled to the processor, the memory is used to store computer program code, and the processor calls the computer program code to enable the electronic device to execute the method of the first aspect.

[0027] It can be understood that the training method and electronic device of the ophthalmic disease prediction model provided by the present application obtain image blocks of multiple scale levels by preprocessing ophthalmic image data, retaining the lesion features from macro to micro, solving the problem of loss of details in conventional downsampling methods, and especially being able to capture multi-level information of subtle lesion areas, thereby improving the recognition ability of complex lesions. Then, through the convolution operation of the stacked adaptive gradient modulation convolution layer, the spatial offset of each sampling point on each image block is learned, and the original convolution kernel weights of the stacked adaptive gradient modulation convolution layer are adjusted based on the spatial offset, which can improve the adaptability to changes in lesion morphology and enhance the recognition of differences in the shape of exudates. Finally, the morphological enhancement feature map is input into the hybrid attention feature extraction module to obtain a small lesion enhancement feature map and the dimensionality is reduced to obtain a feature vector for subsequent prediction, further enhancing the recognition ability of small lesions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0029] Figure 1 A schematic diagram of the structure of a multi-branch convolutional neural network provided in an embodiment of the present application;

[0030] Figure 2 A schematic diagram of a flow chart of a method for training an ophthalmic disease prediction model provided in an embodiment of the present application;

[0031] Figure 3 This is a schematic diagram showing the comparative evaluation results of the training method for the ophthalmic disease prediction model provided in the embodiment of the present application and other methods;

[0032] Figure 4 Schematic diagram 2 of comparative evaluation results of the training method for the ophthalmic disease prediction model provided in an embodiment of the present application and other methods;

[0033] Figure 5 Schematic diagram 3 of comparative evaluation results of the training method for the ophthalmic disease prediction model provided in the embodiment of the present application and other methods;

[0034] Figure 6 Schematic diagram 4 of comparative evaluation results of the training method for the ophthalmic disease prediction model provided in an embodiment of the present application and other methods;

[0035] Figure 7 Schematic diagram 5 of comparative evaluation results of the training method for the ophthalmic disease prediction model provided in the embodiment of the present application and other methods;

[0036] Figure 8Schematic diagram six of comparative evaluation results of the training method for the ophthalmic disease prediction model provided in an embodiment of the present application and other methods;

[0037] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0038] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0039] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different drawings represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this application.

[0040] The terms "first", "second", etc. involved in this application are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.

[0041] The prior art is further described below:

[0042] Chinese invention patent application number CN202410829627.1 proposes a machine learning-based method for discovering anti-ER stress glaucoma IOP-lowering drugs. The method involves constructing an anti-ER stress compound library using a target-independent strategy; training a machine learning model for drug screening by extracting multiple drug physicochemical and structural characteristics to train a machine learning model, then computer-screening multiple marketed and preclinical drugs to select anti-ER stress compounds; and validating the glaucoma disease phenotype by validating the selected anti-ER stress compounds in pre-defined glaucoma cell and animal models. This method aims to explore new glaucoma IOP-lowering drugs, expand the mechanisms of action of existing drugs, establish a target-independent drug discovery model, and fully explore phenotype-based glaucoma drug discovery strategies.

[0043] The above technical solution has the following problems that still need to be further solved:

[0044] 1. Existing technologies usually rely on simple downsampling to process high-resolution ophthalmic images, but this process will lead to the loss of subtle lesion features, especially in the detection of small targets such as microaneurysms. The loss of details will affect the accuracy of the model.

[0045] 2. Traditional normalization methods are usually based on global features and cannot effectively eliminate the lighting differences in fundus images caused by different shooting angles or equipment, resulting in insufficient image contrast. Especially when the contrast of the lesion area is low, the traditional method is less effective.

[0046] 3. Most convolutional neural networks have difficulty distinguishing the texture differences between fundus blood vessels and lesion areas, and are insufficiently sensitive to tiny lesions. Existing technologies often rely on fixed convolution kernel weights and lack the ability to adapt to changes in the morphology of different lesions, especially the obvious differences in the morphology of exudates in diabetic retinopathy.

[0047] 4. In ophthalmic image data, the number of samples of certain lesions (such as glaucoma) is scarce. Traditional oversampling methods usually lead to overfitting and cannot effectively handle imbalanced data. In addition, the poor generalization ability of the model caused by overfitting makes the trained model unable to perform stably in actual clinical applications.

[0048] The following specific embodiments are used to describe in detail the technical solution of the present application and how the technical solution of the present application solves the technical problem. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0049] See also Figure 1 , Figure 1 This is a schematic diagram of the structure of a branched convolutional neural network used in the training method of the ophthalmic disease prediction model provided in this application. The branched convolutional neural network 100 includes at least a stacked adaptive gradient modulated convolution layer 10, a cross-scale feature fusion module 20, a hybrid attention feature extraction module 30, a global average pooling layer 40, and a fully connected classifier 51. In one embodiment, the branched convolutional neural network 100 has three parallel scale processing branches. Each branch inputs an image block of the corresponding scale level (s=1, 2, 3), extracts features through the stacked adaptive gradient modulated convolution layer 10, and the branches interact with each other through the cross-scale feature fusion module 20. The fused features are input into the hybrid attention feature extraction module 30 for further optimization, and finally the dimensionality is reduced by the global average pooling layer 40, connected to the fully connected classifier 51, and the predicted probability of the disease category is output.

[0050] See also Figure 1 and 2 , Figure 2 This is a flow chart of the training method of the ophthalmic disease prediction model provided in this application. The execution subject of the training method can be an electronic device. Figure 2 As shown, the training method may include the following steps:

[0051] Step S110: Obtain ophthalmic image data and label data corresponding to the ophthalmic image data.

[0052] Specifically, ophthalmic image data is acquired through specialized medical equipment, with a focus on diagnosing retinopathy. Clinical-grade fundus cameras are used to acquire high-resolution, color ophthalmic image data under varying lighting conditions. This data covers both healthy individuals and patients with various fundus lesions, such as diabetic retinopathy and glaucoma, ensuring comprehensive coverage of these lesions. The acquisition process strictly adheres to medical imaging standards, retaining the original RAW format data to maintain detailed integrity. Each ophthalmic image is accompanied by basic patient information (age, medical history, etc.) and acquisition parameters (device model, focal length, etc.). All ophthalmic image data undergoes preliminary quality screening to remove severely blurred or occluded samples, forming an ophthalmic image dataset.

[0053] Next, the ophthalmic image data is labeled to obtain labeled data, which is annotated with the location and category of the lesion area. Through manual labeling, experienced ophthalmologists professionally annotate the collected ophthalmic image data. Based on internationally recognized lesion grading standards, such as the ETDRS diabetic retinopathy grading system, the location and category of lesions such as microaneurysms, hemorrhages, and hard exudates are accurately calibrated in the ophthalmic image data. The labeled results are saved as polygonal masks, and the lesion severity level is recorded. This creates structured labeled data that corresponds one-to-one with the ophthalmic image data, providing supervision information for subsequent classification model training.

[0054] Step S120: Pre-process the ophthalmic image data to obtain multiple image blocks at different scale levels.

[0055] Specifically, preprocessing includes multi-scale cropping and normalization. In one embodiment of multi-scale cropping, at different scale levels, the ophthalmic image data is cropped at equal intervals in the horizontal and vertical directions using a sliding window according to a preset sliding step size to obtain multiple image blocks of different scales, wherein the size of the sliding window is determined by the scale index corresponding to the scale level. For example, the multi-scale cropping formula is expressed as: Where, Expressed as Image blocks at different scale levels, Represented as ophthalmic image data; Expressed as an image cropping function; is the scale level; is the window size, and , is the sliding step length, and . Indicates rounding down.

[0056] In one implementation of the normalization, for each image block, a neighborhood window of a preset size is extracted around the target pixel in the image block, and the mean and standard deviation of all pixel values ​​in the neighborhood window are calculated. The target pixel value is subtracted from the mean and then divided by the sum of the standard deviation and a very small constant, thereby converting multiple image blocks of different scales into multiple normalized image blocks of different scale levels. For example, the normalization formula is expressed as Where, For the At each scale level, the image block At the target pixel coordinates The pixel value after dynamic area normalization is performed at ; For The mean of the neighborhood window centered on is calculated as ; is a local neighborhood window, such as, Neighborhood window of is the coordinate of the target pixel, relative to The offset, such as ; For The standard deviation of the neighborhood window centered on ; is a small constant that prevents division by zero, such as, In this way, each pixel can be normalized, thereby achieving normalization of each image block.

[0057] It is understandable that ophthalmic image data has the characteristics of high resolution, large scale differences in lesion areas, and low local contrast. Conventional methods will lose subtle lesion features by direct downsampling, and global normalization has difficulty eliminating illumination differences. Through the implementation of multiple-degree cropping, lesion features from macro to micro are retained. Subsequently, dynamic area normalization is performed through the normalization implementation to eliminate illumination differences while enhancing edge contrast. Local area normalization is performed on each pixel point of the image block at each scale. A multi-scale adaptive cropping strategy is adopted, combined with dynamic area normalization, to retain key lesion areas and enhance local contrast.

[0058] like Figure 3As shown in the figure, in the evaluation experiment, the multi-scale cropping strategy and dynamic area normalization processing were verified to improve the detection effect of various ophthalmic lesions. The horizontal axis in the figure covers five typical lesion types (including diabetic retinopathy, glaucoma, etc.) and healthy samples, and the vertical axis compares the four preprocessing methods. The thermal color block intuitively reflects the classification accuracy, and the color from light to dark represents increasing performance. The experimental results show that the method of this application (dynamic normalization) presents the darkest color block in all lesion types, especially in glaucoma detection. The improvement is most significant, indicating that: 1) multi-scale cropping effectively retains multi-level features from macroscopic blood vessels to microscopic exudates; 2) dynamic area normalization significantly overcomes the problem of illumination differences in fundus images through local contrast enhancement; 3) the technical advantages are more prominent in structurally complex lesions (such as changes in the morphology of the optic cup in glaucoma), proving that this preprocessing strategy is irreplaceable for maintaining the integrity of lesion features.

[0059] Step S130: Through the convolution operation of the stacked adaptive gradient modulation convolution layer, the spatial offset of each sampling point on each image block is learned, the original convolution kernel weight of the stacked adaptive gradient modulation convolution layer is adjusted based on the spatial offset, and the image blocks of multiple different scale levels are transformed to obtain a morphological enhancement feature map.

[0060] Specifically, the morphological enhancement feature map refers to a feature map that enhances the morphology of the lesion area. The expression of the convolution operation can be , where For the The spatial offset of the sampling points, , This example uses a 3x3 convolution operation. The number of output channels of the convolution kernel of the adaptive gradient modulation convolution layer is 9. The output corresponds to the offset of each position on the convolution kernel. The offset vector corresponding to each position under the convolution kernel (that is, the spatial offset of the sampling point) is output.

[0061] Then, the dynamic modulation coefficient of each sampling point is determined according to the spatial offset, and the original convolution kernel weight of the adaptive gradient modulation convolution layer is adjusted according to the dynamic modulation coefficient. The image blocks of multiple scale levels are transformed to obtain the morphological enhancement feature map.

[0062] For example, the expression of the dynamic modulation coefficient is: , where is the dynamic modulation coefficient, is the transpose of the learnable weight vector, The sampling point is at position The offset characteristic value at . is the current position coordinate, For the The air offset of the sampling points guides the adaptive deformation of the convolution kernel. is the sampling point index, For the The spatial offset of each sampling point. It can be understood that the formula in this example represents the spatial offset of each sampling point. , using the sampling function F() to calculate the offset position Sampling is performed at the sampling points to obtain the offset characteristic value of each sampling point after offset Next, the offset eigenvalue and the transpose of the learnable weight vector Take the inner product and get the score of each sampling point . Normalize the scores of all sampling points through the Softmax function , get the dynamic modulation coefficient of each sampling point , reflecting the weight contribution of each sampling point in the convolution output.

[0063] Exemplarily, the output expression of the first feature map is , where is the feature activation intensity of the first feature map at sampling point p, n is the sampling point index, , is the original convolution kernel weight. It can be understood that the formula in this example represents the dynamic modulation coefficient of each sampling point , the original convolution kernel weights of the corresponding adaptive gradient modulation convolution layer and offset eigenvalues After performing item-by-item multiplication, the products of all sampling points are summed to generate the feature activation intensity of the first feature map at the sampling point p. The feature activation intensity is calculated for all sampling points to generate the first feature map, which records the feature activation intensity of all sampling points.

[0064] Finally, multiple first feature maps corresponding to image blocks at multiple scale levels are input into the cross-scale feature fusion module to output a morphologically enhanced feature map.

[0065] Understandably, the fixed weights of traditional convolution kernels result in poor adaptability to changes in lesion morphology, particularly in diabetic retinopathy, where exudate shapes vary significantly. In step S130, the spatial offset of each sampling point on each image block is learned through the convolution operation of the stacked adaptive gradient modulated convolution layers. The original convolution kernel weights of the stacked adaptive gradient modulated convolution layers are adjusted based on the spatial offset, improving adaptability to changes in lesion morphology and enhancing recognition of differences in exudate shape.

[0066] Step S140: Input the morphological enhancement feature map into the hybrid attention feature extraction module to obtain a small lesion enhancement feature map and reduce the dimension to obtain a feature vector.

[0067] Specifically, the micro-lesion enhancement feature map refers to a feature map that enhances the texture difference between the fundus blood vessels and the lesion area and strengthens the response of the lesion area. Figure 1 As shown, the hybrid attention feature extraction module 30 includes a channel attention weight determination module 31, a spatial attention weight determination module 32 and a matching module 33. The channel attention weight determination module includes a first fully connected layer and a second fully connected layer (not shown).

[0068] For example, the expression of the channel attention weight determination module is Where, is the channel attention weight vector, is the Sigmoid activation function, is the weight matrix of the first fully connected layer, is the weight matrix of the second fully connected layer, is the global average pooling, It is the morphological enhancement feature map, in which the pixel values ​​correspond to , is the ReLU activation function. It can be understood that this formula indicates that the following steps are performed by the channel attention weight determination module:

[0069] a.1 Perform global average pooling on the morphological enhancement feature map in the channel dimension to obtain the global response value of each channel .

[0070] a.2 Use the ReLU function to convert the global response value The product of the weight matrix of the first fully connected layer Mapped to values ​​in the non-negative semi-axis interval.

[0071] a.3 Use the Sigmoid function to map the values ​​mapped to the non-negative semi-axis interval to the weight matrix of the second fully connected layer The product is transformed into the channel attention weight vector .

[0072] For example, the expression of the spatial attention weight determination module is Where, is the spatial attention weight map, For a 3×3 convolution operation (i.e. Figure 1 As shown, the spatial attention weight determination module 32 includes an adaptive gradient modulated convolution layer 321); is the maximum pooling, It can be understood that the example formula represents the following steps performed by the spatial attention weight determination module.

[0073] b.1 Perform maximum pooling and global average pooling on the morphological enhancement feature map in the channel dimension to obtain two two-dimensional spatial mapping maps and .

[0074] b.2 Concatenate the two two-dimensional spatial mapping images and perform a convolution operation to obtain the second feature map.

[0075] b.3 Use the Sigmoid function to transform the second feature map into a spatial weight map.

[0076] b.4 Weight each spatial position in the morphological enhancement feature map according to the spatial weight map to obtain the spatial attention weight map .

[0077] For example, the expression of the matching module is , where Enhanced feature map for small lesions. is element-wise multiplication, This is a broadcast addition. The formula expression in this example performs the following steps through the matching module:

[0078] c.1 Pair of channel attention weight vectors and spatial attention weight map Broadcast matching is performed separately to the morphological enhancement feature map Same dimensions.

[0079] c.2. Combine the channel attention weight vector and spatial attention weight map after dimensionality reduction with the morphological enhancement feature map Multiply them to get the channel attention weighted feature map and spatial attention weighted feature maps .

[0080] c.3 Add the channel attention weighted feature map, spatial attention weighted feature map and morphological enhancement feature map to obtain the microlesion enhancement feature map And reduce the dimension to get the feature vector.

[0081] Understandably, conventional convolutional neural networks have difficulty distinguishing between the texture differences between fundus vessels and lesion areas, and are insensitive to tiny lesions. This application uses a channel-spatial hybrid attention mechanism (i.e., a spatial attention weight determination module and a spatial attention weight determination module) to dynamically fuse multi-dimensional features. By combining channel importance evaluation with spatial region focusing, it enhances the detection capability of small targets such as microaneurysms.

[0082] like Figure 4As shown in the figure, in the evaluation experiment, the hybrid attention mechanism was evaluated for its breakthrough in detecting tiny lesions. The horizontal axis uses a logarithmic scale to display the lesion size (10μm-320μm), and the vertical axis is the detection accuracy. The four curves represent different attention configurations, and the shaded area represents the 95% confidence interval. The experimental results show that: 1) The curve of this application (purple) is at the top throughout the entire process, and forms an obvious "performance highland" in the detection range of 10-40μm microaneurysms; 2) The width of the confidence interval is significantly narrower than that of the baseline method, indicating higher prediction stability; 3) In the detection of extremely small lesions (10μm), this application has an improvement of more than 30% compared with the traditional method. The experimental results show the synergistic effect of the channel-space dual-path attention mechanism. The channel weight focuses on the texture difference between blood vessels and lesions, and the spatial weight enhances the response of the lesion area. The fusion of the two enables the model to have the ability to capture "microscopic" lesions.

[0083] Step S140: Input the feature vector and label data into a fully connected classifier and output the disease category prediction probability. When the training stop condition is met, the multi-branch convolutional neural network completes the training and forms an ophthalmic disease prediction model.

[0084] It can be understood that a new ophthalmic image is input into the ophthalmic disease prediction model, and the disease category prediction probability is output. The category corresponding to the maximum prediction probability is used as the classification category. Specifically, the new ophthalmic image data is first input into the ophthalmic disease prediction model, and the image undergoes forward propagation. During this process, the model extracts high-level features from the image and classifies it based on these features. The input image is first cropped and normalized at multiple scales to ensure that the image quality is consistent with that during training; then, through the trained convolutional neural network and hybrid attention feature extraction module, the model will identify and locate the diseased area in the image;

[0085] Finally, through the fully connected classifier, the model outputs the disease category prediction probability of the image to determine the disease category such as diabetic retinopathy, glaucoma, etc.

[0086] In the above technical solution, by preprocessing ophthalmic image data to obtain image blocks at multiple scale levels, the lesion features from macro to micro are retained, solving the problem of loss of details in conventional downsampling methods. In particular, it can capture multi-level information of subtle lesion areas, improving the recognition ability of complex lesions. Then, through the convolution operation of the stacked adaptive gradient modulation convolutional layer, the spatial offset of each sampling point on each image block is learned. Based on the spatial offset, the original convolution kernel weights of the stacked adaptive gradient modulation convolutional layer are adjusted, which can improve the adaptability to changes in lesion morphology and enhance the recognition of differences in exudate shape. Finally, the morphological enhancement feature map is input into the hybrid attention feature extraction module to obtain the micro lesion enhancement feature map and the dimensionality is reduced to obtain the feature vector for subsequent prediction, further enhancing the recognition ability of micro lesions.

[0087] In some embodiments, in step c3, the process of enhancing the feature map of small lesions and reducing the dimension to obtain a feature vector includes transforming the feature map of small lesions into a mask map, performing adversarial generation enhancement based on the mask map and the image block to obtain an enhanced feature map of small lesions.

[0088] For example, the mask image transformation expression is , where is a binary mask image, To perform binarization with a threshold of 0.5, This example formula indicates that the following steps S210 and S220 can be performed:

[0089] Step S210: Enhance the feature map of small lesions Perform a 1x1 convolution operation to generate a response map .

[0090] Step S220: Analyze the response graph using a preset threshold Perform binarization to obtain the mask image ,The mask image is used to locate the lesion area that needs to be enhanced in the ,microlesion enhancement feature map.

[0091] For example, the formula for generating the enhanced micro-lesion enhancement feature map is: , where This is the enhanced feature map of the micro-lesion. is the image block, It is a preset lightweight generation network with 3 residual blocks. This example formula indicates that the following steps S230 to S250 can be performed:

[0092] Step S230: Invert the mask image With image blocks Multiply element by element to get the original retained area image .

[0093] Step S240: Covering the area of ​​the mask image Input the preset lightweight generation network to get the generated content .

[0094] Step S250: Multiply the generated content and the mask image element by element to obtain a composite mask area image .

[0095] Step S260: Add the original retained area image and the synthetic mask area image to obtain the enhanced micro-lesion enhancement feature map And reduce the dimension to get the feature vector.

[0096] It is understandable that ophthalmic image data has an extremely imbalanced class problem. For example, glaucoma samples are scarce, and traditional oversampling can easily lead to overfitting. This application uses data enhancement based on random erasing of lesion areas and generative adversarial methods to solve the problem of small sample learning by retaining the background authenticity while generating diverse lesion morphologies.

[0097] In some embodiments, step S140: inputting the feature vector and label data into a fully connected classifier and outputting the disease category prediction probability until the training stop condition is met, the multi-branch convolutional neural network completes the training, comprising the following steps:

[0098] Step S141: According to the label data, the feature vector is divided into anchor sample features, similar positive sample features and negative sample features.

[0099] Step S142: Calculate the feature similarity between the anchor sample feature and similar positive sample features and negative sample features respectively.

[0100] Step S143: comparing feature similarities to obtain feature comparison losses of each ophthalmic image data.

[0101] The above steps S141 to S143 can be expressed by the following formula: Where, is the feature comparison loss function, which outputs the feature comparison loss of each ophthalmic image data. is the cosine similarity, is the temperature coefficient, such as, , is the anchor sample feature, is the positive sample feature of the same type, is the negative sample feature, is the sample index; is the number of negative samples; Represents the logarithmic function, the default is 10 as the base, Represents an exponential function with a natural constant as its base.

[0102] It is understandable that a single cross entropy loss is difficult to distinguish between similar fundus lesion categories, such as hard exudates and soft exudates. This application adopts a loss function of joint metric learning and classification. By calculating the feature similarity between the anchor sample and its similar samples, and comparing them with the similarity of all its negative samples, the contrast loss of the current sample is calculated using the Softmax structure to ensure that the model brings the features of similar samples closer and pushes the features of different classes farther away, thereby enhancing the discriminability.

[0103] Step S144: Determine the disease category based on the disease category prediction probability output by the fully connected classifier.

[0104] Step S145: The classification confidence score corresponding to the disease category prediction probability is weighted by the label data to calculate the classification loss of each ophthalmic image data, wherein the category weight in the calculation is determined according to the inverse square root of the number of samples of the ophthalmic image data corresponding to the disease category.

[0105] The above steps S144 to S145 can be expressed by the following formula.

[0106] .

[0107] Where, is the classification loss function, which outputs the classification loss of each ophthalmic image data. is the category weight, which is calculated as follows: , Disease category The number of samples, is the category index, is the total number of categories, For category The label data, For the model category The logit output of , logit is the original output value of the last layer of the model without activation function processing, which represents the classification confidence score.

[0108] It can be understood that the classification confidence score corresponding to the predicted probability of each disease category is weighted by the cross entropy calculation of the label data, where the category weight is determined according to the inverse square root of the number of samples of the ophthalmic image data corresponding to the disease category to alleviate the sample imbalance. The Sigmoid activation function is used to map the logit output of the model to [0,1], corresponding to the predicted probability of the category.

[0109] Step S146: linearly weight the feature contrast loss and the classification loss to obtain the total loss function value of each ophthalmic image data. For example, the total loss function value is determined by the formula:

[0110] , where is the total loss function, output the total loss function value, is the balancing factor, such as .

[0111] Step S147: When the downward trend of the total loss function value is less than a preset trend value, it is determined that the multi-branch convolutional neural network has completed training.

[0112] In some embodiments, the training method of the present application further includes optimizing the parameters of each layer structure in the multi-branch convolutional neural network using a dynamic sparsity strategy, such as optimizing the original convolution kernel weights of the adaptive gradient modulated convolution layer, the learnable weight vectors in the cross-scale feature fusion module, and the weight matrices of the first and second fully connected layers in the channel attention weight determination module. That is, the training method of the present application further includes the following steps:

[0113] Step S310: In each training step, the L2 norm of the loss gradient of each ophthalmic image data on the target layer parameter is calculated according to the total loss function value, and the average value of the calculation results is taken as the connection importance index of the target layer parameter at the current iteration number.

[0114] For example, the expression formula of the connection importance of the lth layer at iteration number t is: , where For the Layer in iteration number The connection importance index of is the sample set in the training batch (i.e., multiple ophthalmic image data in the training batch), For the The gradient of the loss with respect to the layer parameters, For the The total loss function value of ophthalmic image data, is the L2 norm.

[0115] Step S330: Generate a sparse mask of the target layer parameters according to the connection importance index.

[0116] Exemplarily, the sparse mask generation is represented as , where For the sparse mask of the layer, Indicates u= before selection The first layer of the maximum value in the iteration number The connection importance index of The number of connections to reserve; To round down, keep the ratio, such as, , For the Layer parameter amount.

[0117] Step S340: Calculate the gradient of the total loss function value with respect to the target layer parameters.

[0118] Step S350: perform element-by-element multiplication of the sparse mask and the gradient, multiply the result by the preset learning rate, and then subtract the result from the matrix of the target layer parameters to obtain the updated target layer parameters. Among the updated target layer parameters, the parameters whose connection importance index is higher than the preset index are updated, and the parameters whose connection importance index is lower than or equal to the preset index remain unchanged.

[0119] Steps S340 to S350 can be performed using the formula Expressed as, where For the Layer The weight matrix at the iteration, For the Layer The weight matrix at the iteration, To preset the learning rate, control the parameter update step size, is element-wise multiplication, is the total loss function value of the current batch of samples for the The gradient of the layer parameters, is the total loss function value of the current batch of samples. In some embodiments, the training method of the present application further includes optimizing the parameters of each layer structure in the multi-branch convolutional neural network by using a phased progressive fine-tuning method, that is, the training method of the present application further includes the following steps:

[0120] Step S410: Freeze the output function of the hybrid feature extraction module, and only update the parameters of the fully connected classifier head until the total loss function value is less than a preset value, completing the parameter update.

[0121] For example, step S410 can be expressed as , where is the optimal parameter of the fully connected classifier head (that is, the parameter obtained by the last update), Parameter optimization process to minimize the total loss function; is the total loss function value; is the frozen hybrid feature extraction module output, is the label data.

[0122] It can be understood that only the parameters of the classifier head are updated and the output of the hybrid feature extraction module is frozen. The optimization goal is to minimize the total loss function under the current structure, laying the foundation for the classifier to learn the initial discrimination boundary of the global feature space.

[0123] Step S420: Unfreeze the output function of the hybrid feature extraction module, update the fully connected classifier head and the adaptive gradient modulated convolution layer of the hybrid feature extraction module, and complete the parameter update when the total loss function value is less than the preset value.

[0124] For example, Figure 1As shown, the hybrid feature extraction module 30 further includes an adaptive gradient modulated convolution layer 321, and step S420 can be expressed as , where are the optimal parameters of the partially unfrozen layers (i.e., the fully connected classifier head and the adaptive gradient modulated convolutional layers of the hybrid feature extraction module), This is the enhanced feature map of the tiny lesion after partial thawing.

[0125] It can be understood that unfreezing the adaptive gradient modulated convolution layer of the hybrid feature extraction module allows the network to gradually restore part of its capacity while maintaining the stability of shallow features. The updated parameters continue to optimize the classification performance based on richer features by minimizing the total loss function.

[0126] Step S430: Unfreeze all layers of the multi-branch convolutional neural network, reduce the learning rate to one tenth of the preset learning rate, and update the parameters of all layers with a smaller step size until the total loss function value is less than the preset value, completing the parameter update.

[0127] For example, step S430 can also be expressed as Expressed as, where is the optimal parameter after fine-tuning the entire network; This is the enhanced feature map of tiny lesions after unfreezing the entire network.

[0128] It can be understood that in the final stage, all layers of the network are unfrozen and the learning rate is reduced to one tenth of the initial learning rate. In this way, all parameters are fine-tuned with a smaller step size while the entire network is trainable, further improving the model performance and stabilizing the optimization results.

[0129] It is understandable that direct end-to-end training is prone to falling into local optimality. The embodiment of the present application designs the above three-stage optimization strategy, which gradually releases network capacity and avoids distortion of deep features.

[0130] like Figures 5 to 8 The following uses multiple evaluation experiments to further illustrate the superiority of the training method provided by combining all the above embodiments of this application. Figure 5As shown, the ability of different classification methods to identify common eye diseases is evaluated. A horizontal comparison with mainstream models (residual networks 50, efficient networks, dense networks 121, and the visual transformer base model) verifies the universality of this technology across multiple lesion scenarios. The horizontal axis of the figure shows five typical ophthalmic conditions (diabetic retinopathy, glaucoma, etc.) and healthy samples, while the vertical axis represents classification accuracy. Different colored bars correspond to the performance of each method. The experimental results show that the training method proposed in this application maintains the highest accuracy bar across all disease categories, with the largest lead in diabetic retinopathy. This demonstrates that the multi-scale adaptive cropping fully preserves microscopic lesion features, the hybrid attention mechanism accurately focuses on the lesion area, and the dynamic gradient modulated convolution's ability to adapt to the morphology of irregular lesions such as exudates, all contribute to improved discrimination accuracy for complex lesions.

[0131] like Figure 6 As shown in the figure, by monitoring the changes in validation set accuracy during training, the optimization efficiency and stability of each method are analyzed. The horizontal axis represents training rounds, the vertical axis represents validation accuracy, and the different line types represent the convergence trajectory of each method. This technical curve (solid line) exhibits three key characteristics: a steep rise in the initial stage (rapid convergence), gentle fluctuations in the middle stage (stable optimization), and a peak plateau (optimal performance) in the late stage. This shows that the dynamic sparse connection training strategy selects high-contribution connections by gradient amplitude, avoiding interference from redundant features. Combined with the three-stage progressive fine-tuning strategy, it solidifies basic features before releasing network capacity, effectively overcoming the local optimal trap caused by end-to-end training and achieving efficient and stable model optimization.

[0132] like Figure 7 As shown in the figure, the feature dimensionality reduction visualization technology is used to intuitively analyze the discriminability of features learned by different methods. The left and right sub-figures show the feature distribution of conventional methods and this technology respectively. The horizontal and vertical axes are dimensionality reduction coordinates, and the point colors represent disease categories. The boundaries of the five categories of samples in the left figure are blurred and interpenetrate each other, especially diabetic retinopathy (red dots) and glaucoma (blue dots) seriously overlap. The right figure shows an ideal distribution with high cohesion and inter-class separation. Various samples form independent and compact clusters, indicating that the channel-space hybrid attention mechanism enhances the feature response of small targets such as microaneurysms through dual-path weight focusing. The joint metric loss function brings similar samples closer and pushes different samples away at the feature level, essentially improving the discriminability of the feature space.

[0133] like Figure 8As shown, the method's generalization ability under limited data is evaluated by simulating a scenario where rare lesion samples are scarce in clinical practice. The horizontal axis represents the proportion of training samples, and the vertical axis represents the key indicators of microaneurysm detection. When the sample proportion is lower than 40%, the performance of conventional methods drops sharply (the dotted line drops sharply), while the curve of this technology (asterisk line) always remains high and stable, and still maintains a significant advantage at a sample size of 10%, indicating the effectiveness of the lesion perception data enhancement strategy. It locates the lesion area based on the attention response map, and adopts an adversarial generation mechanism to synthesize diverse lesion morphologies on the real background, breaking through the overfitting limitations of traditional oversampling, so that the model can obtain discrimination capabilities close to the full amount of data for small sample diseases such as glaucoma.

[0134] In summary, this application retains the lesion features from macro to micro through sliding window cropping and combines image block extraction at different scales, solves the problem of loss of details in conventional downsampling methods, and is especially able to capture multi-level information of subtle lesion areas, thereby improving the ability to recognize complex lesions.

[0135] Through local area normalization processing, the problem of illumination difference is solved, the edge contrast is enhanced, the defects of traditional global normalization are overcome, and the identifiability of tiny lesions is effectively improved. It has obvious advantages, especially in the detection of lesions such as glaucoma.

[0136] The hybrid attention feature extraction module that combines channel and spatial dimensions can more accurately focus on the lesion areas in ophthalmic images, especially in the detection of small targets such as microaneurysms. By dynamically focusing on features in different dimensions, the detection ability of tiny lesions is improved.

[0137] By combining random erasure with the lesion area enhancement technology of generative adversarial networks, this application effectively solves the problem of class imbalance in ophthalmic images, especially in the detection of small sample lesions such as glaucoma, significantly improving the generalization ability of the model and breaking through the overfitting problem caused by traditional oversampling.

[0138] Figure 9 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 9 As shown, the electronic device 1 includes:

[0139] Processor 11, memory 12 and bus 13;

[0140] The memory 12 is used to store the computer program code of the processor 11;

[0141] The processor 11 is configured to execute the technical solution of the training method of the ophthalmic disease prediction model in any of the aforementioned method embodiments by executing the computer program code.

[0142] Optionally, the memory 12 can be independent or integrated with the processor 11.

[0143] The memory 12 is connected to the processor 11 via a bus 13 and enables communication between them.

[0144] Optionally, the memory 12 may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0145] Bus 13 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the figure uses only one thick line, but this does not imply that there is only one bus or only one type of bus.

[0146] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0147] The electronic device 1 is used to execute the technical solution provided in any of the aforementioned method embodiments, and its implementation principles and technical effects are similar and will not be repeated here.

[0148] In some embodiments, the electronic device 1 includes a desktop computer, a server, a smart phone, a laptop computer, or the like.

[0149] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the training method for the ophthalmic disease prediction model as described above is implemented.

[0150] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A training method for an ophthalmic disease prediction model, characterized in that: The method is applied to the training of a multi-branch convolutional neural network, wherein the multi-branch convolutional neural network includes a stacked adaptive gradient modulated convolution layer, a hybrid attention feature extraction module, a cross-scale feature fusion module, and a fully connected classifier, and the method includes: Obtaining ophthalmic image data and label data corresponding to the ophthalmic image data; Preprocessing ophthalmic image data to obtain multiple image blocks at different scale levels; Through the convolution operation of the stacked adaptive gradient modulated convolutional layer, the spatial offset of each sampling point on each image block is learned. Based on the spatial offset, the offset feature value of each sampling point after offset is obtained by sampling. Taking the inner product of the offset feature value and the transpose of the learnable weight vector to obtain the score of each sampling point; Normalize the scores of all sampling points through the Softmax function to obtain the dynamic modulation coefficient of each sampling point; After performing item-by-item multiplication of the dynamic modulation coefficient of each sampling point, the original convolution kernel weight of the corresponding stacked adaptive gradient modulated convolution layer, and the offset eigenvalue, the products are summed for all sampling points to generate a first feature map, where the first feature map records the feature activation intensities of all sampling points; Inputting multiple first feature maps corresponding to image blocks at multiple scale levels into a cross-scale feature fusion module, and outputting a morphologically enhanced feature map; Inputting the morphological enhancement feature map into a hybrid attention feature extraction module to obtain a small lesion enhancement feature map and reducing the dimension to obtain a feature vector, wherein the hybrid attention feature extraction module includes a channel attention weight determination module, a spatial attention weight determination module and a matching module, the channel attention weight determination module is used to convert the morphological enhancement feature map into a channel attention weight vector, and the spatial attention weight determination module is used to convert the morphological enhancement feature map into a spatial attention weight map; The following steps are performed by the matching module: c.1 Broadcast and match the channel attention weight vector and the spatial attention weight map to the same dimension as the morphological enhancement feature map; c.

2. Multiply the reduced channel attention weight vector and spatial attention weight map with the morphological enhancement feature map to obtain the channel attention weighted feature map and spatial attention weighted feature map, respectively. c.3 Add the channel attention weighted feature map, spatial attention weighted feature map, and morphological enhancement feature map to obtain the microlesion enhancement feature map and reduce the dimension to obtain the feature vector; The feature vector and label data are input into a fully connected classifier, and the disease category prediction probability is output. When the training stop condition is met, the multi-branch convolutional neural network completes the training to form the ophthalmic disease prediction model.

2. The method according to claim 1, characterized in that The step of inputting the morphological enhancement feature map into a hybrid attention feature extraction module to obtain a microlesion enhancement feature map and reducing the dimension to obtain a feature vector comprises: The channel attention weight determination module includes a first fully connected layer and a second fully connected layer, and the following steps are performed by the channel attention weight determination module: a.1 Perform global average pooling on the morphological enhancement feature map in the channel dimension to obtain the global response value of each channel; a.2 Use the ReLU function to map the product of the global response value and the weight matrix of the first fully connected layer into a value in the non-negative semi-axis interval; a.3 Use the Sigmoid function to transform the product of the value mapped to the non-negative semi-axis interval and the weight matrix of the second fully connected layer into a channel attention weight vector; The following steps are performed by the spatial attention weight determination module: b.1 Perform maximum pooling and global average pooling on the morphological enhancement feature map in the channel dimension to obtain two two-dimensional spatial mapping maps; b.

2. Concatenate the two 2D spatial maps and perform a convolution operation to obtain the second feature map. b.3 Use the Sigmoid function to transform the second feature map into a spatial weight map; b.4 Weight each spatial position in the morphological enhancement feature map according to the spatial weight map to obtain a spatial attention weight map.

3. The method according to claim 1, characterized in that The micro-lesion enhancement feature map and dimension reduction to obtain a feature vector include: Perform 1x1 convolution operation on the micro-lesion enhancement feature map to generate a response map; Binarizing the response image with a preset threshold to obtain a mask image, wherein the mask image is used to locate the lesion area that needs to be enhanced in the micro-lesion enhancement feature image; The original retained area image is obtained by multiplying the inverse of the mask image with the image block element by element; Input the coverage area of ​​the mask image into the preset lightweight generation network to obtain the generated content; Multiplying the generated content and the mask image element by element to obtain a composite mask area image; The original retained area image and the synthetic mask area image are added together to obtain the enhanced micro-lesion enhancement feature map and the dimensionality is reduced to obtain the feature vector.

4. The method according to any one of claims 1 to 3, characterized in that: Inputting the feature vector and label data into a fully connected classifier and outputting the disease category prediction probability until the training stop condition is met, the multi-branch convolutional neural network completes the training, including: According to the label data, the feature vector is divided into anchor sample features, similar positive sample features and negative sample features; Calculate the feature similarity between the anchor sample features and the features of similar positive samples and negative samples respectively; Comparing the feature similarity, we can obtain the feature comparison loss of each ophthalmic image data; Determine the disease category based on the disease category prediction probability output by the fully connected classifier; The classification loss of each ophthalmic image data is calculated by weighting the classification confidence score corresponding to the disease category prediction probability and the label data. The category weight in the calculation is determined by the inverse square root of the number of samples of the ophthalmic image data corresponding to the disease category. The feature contrast loss and classification loss are linearly weighted to obtain the total loss function value of each ophthalmic image data; When the downward trend of the total loss function value is less than a preset trend value, it is determined that the multi-branch convolutional neural network has completed training.

5. The method according to claim 4, characterized in that The method further comprises: In each training step, the L2 norm of the loss gradient of each ophthalmic image data on the target layer parameters is calculated according to the total loss function value, and the average value of the calculated results is taken as the connection importance index of the target layer parameters at the current iteration number; generating a sparse mask of target layer parameters according to the connection importance index; Calculate the gradient of the total loss function value with respect to the target layer parameters; The sparse mask and the gradient are element-wise multiplied and multiplied with the preset learning rate, and then subtracted from the matrix of the target layer parameters to obtain updated target layer parameters. Among the updated target layer parameters, the parameters whose connection importance index is higher than the preset index are updated, and the parameters whose connection importance index is lower than or equal to the preset index remain unchanged.

6. The method according to claim 5, characterized in that The method further comprises: Freeze the output function of the hybrid feature extraction module and only update the parameters of the fully connected classifier head until the total loss function value is less than the preset value. Unfreeze the output function of the hybrid feature extraction module and update the fully connected classifier head and the adaptive gradient modulated convolution layer of the hybrid feature extraction module until the total loss function value is less than the preset value, completing the parameter update; All layers of the multi-branch convolutional neural network are unfrozen, and the learning rate is reduced to one tenth of the preset learning rate. The parameters of all layers are updated with a smaller step size until the total loss function value is less than the preset value, completing the parameter update.

7. The method according to claim 1, characterized in that The obtaining of ophthalmic image data and label data corresponding to the ophthalmic image data includes: Acquiring ophthalmic image data, and labeling the ophthalmic image to obtain label data, wherein the label data is labeled with the location and category of the lesion area; The preprocessing of the ophthalmic image data obtains a plurality of image blocks at different scale levels, including: At different scale levels, the ophthalmic image data is cropped using a sliding window according to a preset sliding step size to obtain multiple image blocks of different scales, wherein the size of the sliding window is determined by the scale index corresponding to the scale level; For each image block, a neighborhood window of a preset size around the target pixel in the image block is extracted with the target pixel as the center, and the mean and standard deviation of all pixel values ​​in the neighborhood window are calculated. The target pixel value is subtracted from the mean and then divided by the sum of the standard deviation and a very small constant, and multiple image blocks of different scales are converted into normalized image blocks of the multiple different scale levels.

8. The method according to claim 1, characterized in that The method also includes: New ophthalmic image data is acquired and input into the ophthalmic disease prediction model, and the disease category prediction probability is output.

9. An electronic device, characterized in that: include: processor and memory; The memory is coupled to the processor, and the memory is used to store computer program code. The processor calls the computer program code to enable the electronic device to execute the method according to any one of claims 1 to 8.

Images