Prediction method and system for retinopathy symptoms of premature infants

Abstract

The invention relates to the field of image processing, in particular to a premature infant retinopathy symptom prediction method and system. The method comprises the following steps: acquiring a plurality of angle retina images and risk factor information; performing noise reduction and redundancy elimination processing; constructing a feature extraction network; carrying out feature fusion on the plurality of angle features to obtain a fused feature; extracting features of the risk factor information through a stacked auto-encoder to obtain risk factor features; performing feature splicing on the fusion feature and the risk factor feature to obtain a final feature; and obtaining a prediction result according to the final features. According to the method, a plurality of retina images shot from different angles are adopted, features obtained in the images are spliced with risk factor features, and the influence of corrected gestational age on the retinopathy of the premature infant during treatment is considered, so that the intelligent prediction effect of the retinopathy symptom of the premature infant is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more specifically, to a method and system for predicting symptoms of retinopathy of prematurity. Background Art

[0002] In recent years, with the advancement of medical technology, the survival rate of premature infants has continued to increase, but at the same time, the incidence of retinopathy has also been on the rise, so early prediction and intervention of the disease are particularly critical. With the rapid development of medical imaging and artificial intelligence technology, new possibilities have been provided for the prediction of retinopathy of prematurity. Through high-resolution fundus imaging technology, the structure and changes of retinal blood vessels can be clearly observed, providing an objective basis for the identification of lesions. At the same time, based on artificial intelligence algorithms such as deep learning, fundus images can be automatically analyzed and processed to extract feature information related to lesions, so as to achieve early prediction and diagnosis of lesions. However, the current method mainly relies on the single information of retinal images, but fails to combine retinal images with risk factors, which has obvious limitations. Although relying solely on retinal images for prediction can capture the visual characteristics of lesions, it ignores other important risk factors, such as the birth weight and gestational age of premature infants, which can easily lead to misdiagnosis or missed diagnosis.

[0003] The prior art discloses an automatic zoning and identification method for retinopathy of prematurity based on an attention mechanism and a deep supervision strategy. The method first preprocesses the two-dimensional retinal fundus color image, including bilinear interpolation downsampling and mean reduction processing, and performs online data amplification operations to enhance data diversity. Then, the spatial channel attention module SACAB uses the SE compression excitation module as its channel attention mechanism, automatically obtains the importance of each channel through learning, accurately locates the lesion, and enhances the feature extraction effect. However, this method uses a single picture and ignores the pathological characteristics of the patient's eyeball at multiple angles, resulting in incomplete diagnosis. Summary of the invention

[0004] The purpose of the present invention is to disclose a prediction method and system for symptoms of retinopathy of prematurity with better prediction effect.

[0005] In order to achieve the above object, the present invention provides a method for predicting symptoms of retinopathy of prematurity, comprising:

[0006] Obtain retinal images from multiple angles and risk factor information;

[0007] Performing noise reduction and redundancy removal processing on the retinal images at multiple angles to obtain processed retinal images at multiple angles;

[0008] Construct a feature extraction network. The feature extraction network consists of two parts. The first part is three modules consisting of a downsampling layer, a ConvNeXt layer, and an adaptive three-channel attention layer. The second part consists of a downsampling layer and a special ConvNeXt layer. The special ConvNext layer is inserted with the RSG mechanism.

[0009] Extract features from the processed retinal images at multiple angles through a feature extraction network to obtain features at multiple angles;

[0010] The risk factor information is extracted through stacked autoencoders to obtain risk factor features;

[0011] The features of multiple angles are fused to obtain fused features; the fused features and risk factor features are spliced ​​to obtain the final features;

[0012] According to the final features, the prediction results are obtained.

[0013] Furthermore, the retinal images at multiple angles are subjected to noise reduction and redundancy removal processing, and the processed retinal images at multiple angles include:

[0014] A window is slid on the retinal images at multiple angles, pixel values ​​of the retinal images at multiple angles in the window are sorted and the median is taken as a new value, and the retinal images at multiple angles in the window are processed to obtain a new pixel value image;

[0015] Then the new pixel value image is fused with the retinal images at multiple angles to obtain the denoised image;

[0016] A circular mask similar in size to the eyeball is multiplied pixel by pixel by the denoised image, so that pixels with a mask value of 0 become the specified background color, while pixels with a value of 1 retain their original values ​​to obtain retinal images at multiple angles after processing.

[0017] Furthermore, the feature extraction network consists of two parts. The first part is three modules consisting of a downsampling layer, a ConvNeXt layer, and an adaptive three-channel attention layer. The second part consists of a downsampling layer and a special ConvNeXt layer. The special ConvNext layer is inserted with the RSG mechanism.

[0018] x ′ = [ATBA (ConvNeXt block ×n (downsampling(x)))] ×3

[0019] EnConvNeXt(x)

[0020] =ConvNeXt block ×2(RSG(ConvNeXt block(downsampling(x ′ ))))

[0021] Among them, EnConvNeXt is the feature extraction network, ATBA is the adaptive three-channel attention, and downsampling is the downsampling operation.

[0022] Furthermore, the adaptive three-channel attention layer includes three channels, specifically:

[0023] The adaptive three-channel attention layer adaptively prioritizes the channel containing the key information of the current task in the first channel, while the second channel adaptively adjusts its spatial focus to capture the key information; in addition, in order to capture the global dependency characteristics of the retinal feature map, the first stage of the third channel performs feature mapping through three 1×1 convolution operations to obtain a set of rich intermediate features; in the second stage, the attention weights are calculated using the query and key, and these weights are used to weight and aggregate the values ​​to form the attention aggregation features;

[0024] The adaptive three-channel attention layer is formulated as follows:

[0025] X ATBA =Sumw(CA(X Cb ),SA(X Cb ),SelfAttention(X Cb ))

[0026] Among them, Sumw(·) represents the weighted average operation, X ATBA represents the output of the adaptive three-channel attention, and X Cb Generation of EnConvNeXt network output.

[0027] Furthermore, the RSG mechanism includes a center estimation part, a comparison part, and a vector conversion part;

[0028] The center estimation part is used to estimate a set of centers in each class. The formula of the center estimation part is as follows:

[0029] γ 1 =f(A 1 ave(x 1 )+b 1 )

[0030] Given a feature map x of an input sample l , l is the category label of the sample; then the feature map of the input sample is globally averaged pooled, and then a set of parameters A l and b l Perform a linear transformation; finally, use the softmax function f to calculate the output probability distribution γl , this distribution is used to assign samples to the closest center in the class to which they belong;

[0031] The comparison module is used to ensure that the feature displacement does not contain any frequent class related information during the sample generation process; the formula of the comparison module is as follows:

[0032] γ * =f(A * ave(h(cat[x1,x2]))+b * )

[0033] Where x1, x2 are feature maps of any two input samples from the same mini-batch, cat(·) represents the cascade operation along the channel dimension, h(·) is the convolution layer, and γ * Used to indicate whether two samples belong to the same category;

[0034] The vector transformation is used to calculate the feature displacement of each frequent class sample according to the estimated center and use it to generate new samples of the rare class; the vector transformation formula is as follows:

[0035]

[0036] in, represents a sample in the frequent class l, is with The central feature map of the same size is subtracted to get x fd-freq Indicates the feature displacement of frequent class samples;

[0037] The generation process of new rare class samples is:

[0038]

[0039] in, is the convolution operation, represents samples in the rare class l, is a newly generated sample in this class.

[0040] Furthermore, it also includes

[0041] The parameters of the RSG mechanism are optimized through the sample comparison part loss center estimation part and the maximum vector loss function. The purpose of the center estimation with sample comparison part loss is to update the center of each class and optimize the comparison part and the center estimation part; the maximum vector loss optimizes the parameters of the vector transformation module and ensures that the newly generated samples can enlarge the feature rare class space. The overall loss function formula is as follows:

[0042] L total =L cls +λ1L CESC+λ2L MV

[0043] Where L total is the overall loss function, L CESC is the central estimation function of the sample contrast loss, L MV is the maximum vector loss function, λ1, λ1 is the parameter, L cls It is a classification loss function, which is used to measure the accuracy of classifying samples and ensure that the model can correctly distinguish samples of different categories.

[0044] Furthermore, the stacked autoencoder is composed of multiple autoencoders stacked on top of each other, and each layer contains an encoder and a decoder; the encoder compresses the input into a low-dimensional space, and the decoder reconstructs the original input; through layer-by-layer training, deep features are extracted, and feature extraction and dimensionality reduction are performed on unlabeled data through self-supervised learning.

[0045] Furthermore, multiple angle features are fused to obtain fused features including:

[0046]

[0047] X resh =resh ape(avg(X ECN ))

[0048] X conv =conv 1×1 (X ECN )

[0049] Among them, M(·) is the fusion weight, the range of fusion weight is between 0 and 1, X MMFF is the fusion feature, X ECN For multiple angle features, X resh is the average feature, X conv is the multi-view image feature.

[0050] Furthermore, the fusion features and risk factor features are spliced ​​to obtain the final features including:

[0051] Reduce the dimension of the fused features to a one-dimensional vector to obtain a fused one-dimensional vector;

[0052] Reduce the dimension of risk factor characteristics to a one-dimensional vector, and obtain a one-dimensional vector of risk factors;

[0053] The fusion one-dimensional vector and the risk factor one-dimensional vector are concatenated to obtain the final feature.

[0054] In addition, the present invention also provides a prediction system for symptoms of retinopathy of prematurity, comprising:

[0055] Acquisition module: acquires retinal images from multiple angles and risk factor information;

[0056] Processing module: performing noise reduction and redundancy removal processing on retinal images at multiple angles to obtain processed retinal images at multiple angles;

[0057] Network construction module: Construct a feature extraction network. The feature extraction network consists of two parts. The first part is three modules consisting of a downsampling layer, a ConvNeXt layer, and an adaptive three-channel attention layer. The second part consists of a downsampling layer and a special ConvNeXt layer. The special ConvNext layer is inserted with the RSG mechanism.

[0058] Image feature extraction module: extract features from the processed retinal images at multiple angles through a feature extraction network to obtain features at multiple angles;

[0059] Risk factor feature extraction module: extracts features from risk factor information through stacked autoencoders to obtain risk factor features;

[0060] Feature fusion and splicing module: fuses features from multiple angles to obtain fused features; splices fused features and risk factor features to obtain final features;

[0061] Prediction module: Get the prediction result based on the final features.

[0062] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0063] This method uses multiple retinal images taken from different angles, and integrates three channel attentions through adaptive three-channel attention to enhance the recognition and analysis of key features in retinal images, and makes up for the lack of sample size through the RSG mechanism. This method also considers the impact of risk factors on retinopathy of prematurity by splicing the features obtained in the image with the features of risk factors, thereby improving the intelligent prediction effect of retinopathy of prematurity symptoms. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of the method for predicting symptoms of retinopathy of prematurity described in Example 1;

[0065] Figure 2 This is a schematic diagram of a feature extraction network described in Example 2;

[0066] Figure 3 This is a block diagram of a system for predicting symptoms of retinopathy of prematurity as described in Example 2; DETAILED DESCRIPTION

[0067] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0068] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0069] Embodiment 1:

[0070] The present invention provides a method for predicting symptoms of retinopathy of prematurity, comprising:

[0071] S1: Obtain retinal images from multiple angles and risk factor information;

[0072] S2: performing noise reduction and redundancy removal processing on the retinal images at multiple angles to obtain processed retinal images at multiple angles;

[0073] S3: Construct a feature extraction network. The feature extraction network consists of two parts. The first part is three modules consisting of a downsampling layer, a ConvNeXt layer, and an adaptive three-channel attention layer. The second part consists of a downsampling layer and a special ConvNeXt layer. The special ConvNext layer is inserted with the RSG mechanism.

[0074] S4: extracting features from the processed retinal images at multiple angles through a feature extraction network to obtain features at multiple angles;

[0075] S5: extracting features from the risk factor information through stacked autoencoders to obtain risk factor features;

[0076] S6: Fusing features from multiple angles to obtain fused features; and concatenating fused features with risk factor features to obtain final features;

[0077] S7: According to the final features, the prediction results are obtained.

[0078] This embodiment uses multiple retinal images taken from different angles, and integrates three channel attentions through adaptive three-channel attention to enhance the recognition and analysis of key features in retinal images, and compensates for the lack of sample quantity through the RSG mechanism. This embodiment takes into account the impact of risk factors on retinopathy of prematurity by splicing the features obtained in the image with the features of risk factors, thereby improving the intelligent prediction effect of retinopathy of prematurity symptoms.

[0079] Embodiment 2:

[0080] This embodiment further discloses on the basis of the first embodiment:

[0081] The retinal images at multiple angles are subjected to noise reduction and redundancy removal processing, and the processed retinal images at multiple angles include:

[0082] A window is slid on the retinal images at multiple angles, pixel values ​​of the retinal images at multiple angles in the window are sorted and the median is taken as a new value, and the retinal images at multiple angles in the window are processed to obtain a new pixel value image;

[0083] Then the new pixel value image is fused with the retinal images at multiple angles to obtain the denoised image;

[0084] A circular mask similar in size to the eyeball is multiplied pixel by pixel by the denoised image, so that pixels with a mask value of 0 become the specified background color, while pixels with a value of 1 retain their original values ​​to obtain retinal images at multiple angles after processing.

[0085] like Figure 2 The feature extraction network shown consists of two parts. The first part is three modules consisting of a downsampling layer, a ConvNeXt layer, and an adaptive three-channel attention layer. The second part consists of a downsampling layer and a special ConvNeXt layer. The special ConvNext layer is inserted with the RSG mechanism.

[0086] x ′ = [ATBA (ConvNeXt block ×n (downsampling(x)))] ×3

[0087] EnConvNeXt(x)

[0088] =ConvNeXt block ×2 (RSG(ConvNeXt block(downsampling(x ′ ))))

[0089] Among them, EnConvNeXt is the feature extraction network, ATBA is the adaptive three-channel attention, and downsampling is the downsampling operation.

[0090] The adaptive three-channel attention layer consists of three channels, specifically:

[0091] The adaptive three-channel attention layer adaptively prioritizes the channel containing the key information of the current task in the first channel, while the second channel adaptively adjusts its spatial focus to capture the key information; in addition, in order to capture the global dependency characteristics of the retinal feature map, the first stage of the third channel performs feature mapping through three 1×1 convolution operations to obtain a set of rich intermediate features; in the second stage, the attention weights are calculated using the query and key, and these weights are used to weight and aggregate the values ​​to form the attention aggregation features;

[0092] The adaptive three-channel attention layer is formulated as follows:

[0093] X ATBA =Sumw(CA(X Cb ),SA(X Cb ),SelfAttention(X Cb ))

[0094] Among them, Sumw(·) represents the weighted average operation, X ATBA represents the output of the adaptive three-channel attention, and X Cb Generation of EnConvNeXt network output.

[0095] The RSG mechanism includes a center estimation part, a comparison part, and a vector conversion part;

[0096] The center estimation part is used to estimate a set of centers in each class. The formula of the center estimation part is as follows:

[0097] γ l =f(A l ave(x l )+b l )

[0098] Given a feature map x of an input sample l , l is the category label of the sample; then the feature map of the input sample is globally averaged pooled, and then a set of parameters A l and b l Perform a linear transformation; finally, use the softmax function f to calculate the output probability distribution γ l , this distribution is used to assign samples to the closest center in the class to which they belong;

[0099] The comparison module is used to ensure that the feature displacement does not contain any frequent class related information during the sample generation process; the formula of the comparison module is as follows:

[0100] γ * =f(A * ave(h(cat[x1,x2]))+b * )

[0101] Where x1, x2 are feature maps of any two input samples from the same mini-batch, cat(·) represents the cascade operation along the channel dimension, h(·) is the convolution layer, and γ * Used to indicate whether two samples belong to the same category;

[0102] The vector transformation is used to calculate the feature displacement of each frequent class sample according to the estimated center and use it to generate new samples of the rare class; the vector transformation formula is as follows:

[0103]

[0104] in, represents a sample in the frequent class l, is with The central feature map of the same size is subtracted to get x fd-freq Indicates the feature displacement of frequent class samples;

[0105] The generation process of new rare class samples is:

[0106]

[0107] in, is the convolution operation, represents samples in the rare class l, is a newly generated sample in this class.

[0108] Also includes

[0109] The parameters of the RSG mechanism are optimized through the sample comparison part loss center estimation part and the maximum vector loss function. The purpose of the center estimation with sample comparison part loss is to update the center of each class and optimize the comparison part and the center estimation part; the maximum vector loss optimizes the parameters of the vector transformation module and ensures that the newly generated samples can enlarge the feature rare class space. The overall loss function formula is as follows:

[0110] L total =L cls +λ1L CESC +λ2L MV

[0111] Where L total is the overall loss function, L CEsC is the central estimation function of the sample contrast loss, L MV is the maximum vector loss function, λ1, λ1 is the parameter, L cls It is a classification loss function, which is used to measure the accuracy of classifying samples and ensure that the model can correctly distinguish samples of different categories.

[0112] The stacked autoencoder is composed of multiple autoencoders stacked on top of each other. Each layer contains an encoder and a decoder. The encoder compresses the input into a low-dimensional space, and the decoder reconstructs the original input. Through layer-by-layer training, deep features are extracted, and feature extraction and dimensionality reduction are performed on unlabeled data through self-supervised learning.

[0113] The processed retinal images at multiple angles are passed through a feature extraction network to extract features, and the features at multiple angles are obtained, including:

[0114] Adaptive three-channel attention layer: Adaptive three-channel attention integrates three attention mechanisms, which are responsible for adaptively prioritizing key information channels, adjusting spatial focus, and capturing global dependent features. By using these three attention mechanisms as parallel branches with adaptive weights and dynamically adjusting the attention weight of each branch, it achieves diversified capture and integration of features, thereby improving multiple angles of features.

[0115] The expressiveness and classification performance of features;

[0116] RSG mechanism: To address the problem of insufficient number of recurrence samples, new recurrence samples are generated through the RSG mechanism to expand the data set, thereby improving the ability to identify and extract features.

[0117] This embodiment uses multiple retinal images taken from different angles, and integrates three channel attentions through adaptive three-channel attention to enhance the recognition and analysis of key features in retinal images, and compensates for the lack of sample quantity through the RSG mechanism. This embodiment takes into account the impact of risk factors on retinopathy of prematurity by splicing the features obtained in the image with the features of risk factors, thereby improving the intelligent prediction effect of retinopathy of prematurity symptoms.

[0118] Embodiment three:

[0119] This embodiment further discloses on the basis of the second embodiment:

[0120] The features of multiple angles are fused to obtain the following fusion features:

[0121]

[0122] X resh =resh ape(avg(X ECN ))

[0123] X conv =conv 1×1 (X ECN )

[0124] Among them, M(·) is the fusion weight, the range of fusion weight is between 0 and 1, X MMFF is the fusion feature, X ECN For multiple angle features, X resh is the average feature, X conv is the multi-view image feature.

[0125] The fusion features and risk factor features are spliced ​​together to obtain the final features including:

[0126] Reduce the dimension of the fused features to a one-dimensional vector to obtain a fused one-dimensional vector;

[0127] Reduce the dimension of risk factor characteristics to a one-dimensional vector, and obtain a one-dimensional vector of risk factors;

[0128] The fusion one-dimensional vector and the risk factor one-dimensional vector are concatenated to obtain the final feature.

[0129] This embodiment uses multiple retinal images taken from different angles, and integrates three channel attentions through adaptive three-channel attention to enhance the recognition and analysis of key features in retinal images, and compensates for the lack of sample quantity through the RSG mechanism. This embodiment takes into account the impact of risk factors on retinopathy of prematurity by splicing the features obtained in the image with the features of risk factors, thereby improving the intelligent prediction effect of retinopathy of prematurity symptoms.

[0130] Embodiment 4:

[0131] This embodiment provides Figure 3 A prediction system for symptoms of retinopathy of prematurity is shown, comprising:

[0132] Acquisition module: acquires retinal images from multiple angles and risk factor information;

[0133] Processing module: performing noise reduction and redundancy removal processing on retinal images at multiple angles to obtain processed retinal images at multiple angles;

[0134] Network construction module: Construct a feature extraction network. The feature extraction network consists of two parts. The first part is three modules consisting of a downsampling layer, a ConvNeXt layer, and an adaptive three-channel attention layer. The second part consists of a downsampling layer and a special ConvNeXt layer. The special ConvNext layer is inserted with the RSG mechanism.

[0135] Image feature extraction module: extract features from the processed retinal images at multiple angles through a feature extraction network to obtain features at multiple angles;

[0136] Risk factor feature extraction module: extracts features from risk factor information through stacked autoencoders to obtain risk factor features;

[0137] Feature fusion and splicing module: fuses features from multiple angles to obtain fused features; splices fused features and risk factor features to obtain final features;

[0138] Prediction module: Get the prediction result based on the final features.

[0139] This embodiment uses multiple retinal images taken from different angles, and integrates three channel attentions through adaptive three-channel attention to enhance the recognition and analysis of key features in retinal images, and compensates for the lack of sample quantity through the RSG mechanism. This embodiment considers the effect of the corrected gestational age during treatment on retinopathy of prematurity by splicing the features obtained in the image with the risk factor features, thereby improving the intelligent prediction effect of retinopathy of prematurity symptoms.

[0140] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for predicting symptoms of retinopathy of prematurity, characterized in that: include: Obtain retinal images from multiple angles and risk factor information; Performing noise reduction and redundancy removal processing on the retinal images at multiple angles to obtain processed retinal images at multiple angles; Construct a feature extraction network. The feature extraction network consists of two parts. The first part is three modules consisting of a downsampling layer, a ConvNeXt layer, and an adaptive three-channel attention layer. The second part consists of a downsampling layer and a special ConvNeXt layer. The special ConvNext layer is inserted with the RSG mechanism. Extract features from the processed retinal images at multiple angles through a feature extraction network to obtain features at multiple angles; The risk factor information is extracted through stacked autoencoders to obtain risk factor features; Fusing multiple angle features to obtain fused features; The fusion features and risk factor features are concatenated to obtain the final features; According to the final features, the prediction results are obtained.

2. The method for predicting symptoms of retinopathy of prematurity according to claim 1, characterized in that: The retinal images at multiple angles are subjected to noise reduction and redundancy removal processing, and the processed retinal images at multiple angles include: A window is slid on the retinal images at multiple angles, pixel values ​​of the retinal images at multiple angles in the window are sorted and the median is taken as a new value, and the retinal images at multiple angles in the window are processed to obtain a new pixel value image; Then the new pixel value image is fused with the retinal images at multiple angles to obtain the denoised image; A circular mask similar in size to the eyeball is multiplied pixel by pixel by the denoised image, so that pixels with a mask value of 0 become the specified background color, while pixels with a value of 1 retain their original values ​​to obtain retinal images at multiple angles after processing.

3. The method for predicting symptoms of retinopathy of prematurity according to claim 2, characterized in that: The feature extraction network includes: x ′ =[ATBA(ConvNeXt block ×n (downsampling(x)))] ×3 EnConvNeXt(x) =ConvNeXt block ×2 (RSG(ConvNeXt block(downsampling(x ′ ))))Among them, EnConvNeXt is the feature extraction network, ATBA is the adaptive three-channel attention, and downsampling is the downsampling operation.

4. The method for predicting symptoms of retinopathy of prematurity according to claim 3, characterized in that: The adaptive three-channel attention layer consists of three channels, specifically: The adaptive three-channel attention layer adaptively prioritizes the channel containing the key information of the current task in the first channel, while the second channel adaptively adjusts its spatial focus to capture the key information; in addition, in order to capture the global dependency characteristics of the retinal feature map, the first stage of the third channel performs feature mapping through three 1×1 convolution operations to obtain a set of rich intermediate features; In the second stage, attention weights are calculated using the query and key, and these weights are used to weight and aggregate the values ​​to form attention aggregate features; The adaptive three-channel attention layer is formulated as follows: X ATBA =Sum(CA(X Cb ),EVEN Cb ),SelfAttention(X Cb )) Among them, Sumw(·) represents the weighted average operation, X ATBA represents the output of the adaptive three-channel attention, and X Cb Generation of EnConvNeXt network output.

5. The method for predicting symptoms of retinopathy of prematurity according to claim 1, characterized in that: The RSG mechanism includes a center estimation part, a comparison part, and a vector conversion part; The center estimation part is used to estimate a set of centers in each class. The formula of the center estimation part is as follows: c l =f(A l ave(x l )+b l ) Given a feature map x of an input sample l , l is the category label of the sample; then the feature map of the input sample is globally averaged pooled, and then a set of parameters A l and b l Perform a linear transformation; finally, use the softmax function f to calculate the output probability distribution γ 1 , this distribution is used to assign samples to the closest center in the class to which they belong; The comparison module is used to ensure that the feature displacement does not contain any frequent class related information during the sample generation process; the formula of the comparison module is as follows: γ * =f(A * ave(h(cat[x1,x2]))+b * ) Where x1, x2 are feature maps of any two input samples from the same mini-batch, cat(·) represents the cascade operation along the channel dimension, h(·) is the convolution layer, and γ * Used to indicate whether two samples belong to the same category; The vector transformation is used to calculate the feature displacement of each frequent class sample according to the estimated center and use it to generate new samples of the rare class; the vector transformation formula is as follows: in, represents a sample in the frequent class l, is with The central feature map of the same size is subtracted to get x fd-freq Indicates the feature displacement of frequent class samples; The generation process of new rare class samples is: in, is the convolution operation, represents samples in the rare class l, is a newly generated sample in this class.

6. The method for predicting symptoms of retinopathy of prematurity according to claim 5, characterized in that: Also includes The parameters of the RSG mechanism are optimized through the sample comparison part loss center estimation part and the maximum vector loss function. The purpose of the center estimation with sample comparison part loss is to update the center of each class and optimize the comparison part and the center estimation part; the maximum vector loss optimizes the parameters of the vector transformation module and ensures that the newly generated samples can enlarge the feature rare class space. The overall loss function formula is as follows: L total =L cls +λ1L CEsC +λ2L MV Where L total is the overall loss function, L CESC is the central estimation function of the sample contrast loss, L MV is the maximum vector loss function, λ1, λ1 is the parameter, L cls It is a classification loss function, which is used to measure the accuracy of classifying samples and ensure that the model can correctly distinguish samples of different categories.

7. The method for predicting symptoms of retinopathy of prematurity according to claim 1, characterized in that: The stacked autoencoder is composed of multiple autoencoders stacked on top of each other. Each layer contains an encoder and a decoder. The encoder compresses the input into a low-dimensional space, and the decoder reconstructs the original input. Through layer-by-layer training, deep features are extracted, and feature extraction and dimensionality reduction are performed on unlabeled data through self-supervised learning.

8. The method for predicting symptoms of retinopathy of prematurity according to claim 1, characterized in that: The features of multiple angles are fused to obtain the following fusion features: X resh =resh ape(avg(X ECN )) X conv =conv 1×1 (X ECN ) Among them, M(·) is the fusion weight, the range of fusion weight is between 0 and 1, X MMFF is the fusion feature, X ECN For multiple angle features, X resh is the average feature, X conv is the multi-view image feature.

9. The artificial intelligence prediction method for retinopathy of prematurity symptoms according to claim 1, characterized in that: The fusion features and risk factor features are spliced ​​together to obtain the final features including: Reduce the dimension of the fused features to a one-dimensional vector to obtain a fused one-dimensional vector; Reduce the dimension of risk factor characteristics to a one-dimensional vector, and obtain a one-dimensional vector of risk factors; The fusion one-dimensional vector and the risk factor one-dimensional vector are concatenated to obtain the final feature.

10. A system for predicting symptoms of retinopathy of prematurity, characterized in that: include: Acquisition module: acquires retinal images from multiple angles and risk factor information; Processing module: performing noise reduction and redundancy removal processing on retinal images at multiple angles to obtain processed retinal images at multiple angles; Network construction module: Construct a feature extraction network. The feature extraction network consists of two parts. The first part is three modules consisting of a downsampling layer, a ConvNeXt layer, and an adaptive three-channel attention layer. The second part consists of a downsampling layer and a special ConvNeXt layer. The special ConvNext layer is inserted with the RSG mechanism. Image feature extraction module: extract features from the processed retinal images at multiple angles through a feature extraction network to obtain features at multiple angles; Risk factor feature extraction module: extracts features from risk factor information through stacked autoencoders to obtain risk factor features; Feature fusion and splicing module: fuses features from multiple angles to obtain fused features; splices fused features and risk factor features to obtain final features; Prediction module: Get the prediction result based on the final features.

Images